Go發起HTTP2.0請求流程分析(後篇)——標頭壓縮

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來自公衆號：新世界雜貨鋪"}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"閱讀建議"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這是HTTP2.0系列的最後一篇，筆者推薦閱讀順序如下:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/6107cc8ccba566d1bcb4b2159","title":""},"content":[{"type":"text","text":"Go中的HTTP請求之——HTTP1.1請求流程分析"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/cb50c604d02c1ad0863e6a6d9","title":""},"content":[{"type":"text","text":"Go發起HTTP2.0請求流程分析(前篇)"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/d4c93658b5760c35d44ad0678","title":""},"content":[{"type":"text","text":"Go發起HTTP2.0請求流程分析(中篇)——數據幀&流控制"}]}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"回顧"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在前篇"},{"type":"codeinline","content":[{"type":"text","text":"(*http2ClientConn).roundTrip"}]},{"type":"text","text":"方法中提到了寫入請求header，而在寫入請求header之前需要先編碼（源碼見https://github.com/golang/go/blob/master/src/net/http/h2_bundle.go#L7947）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在中篇"},{"type":"codeinline","content":[{"type":"text","text":"(*http2ClientConn).readLoop"}]},{"type":"text","text":"方法中提到了"},{"type":"codeinline","content":[{"type":"text","text":"ReadFrame()"}]},{"type":"text","text":"方法，該方法會讀取數據幀，如果是"},{"type":"codeinline","content":[{"type":"text","text":"http2FrameHeaders"}]},{"type":"text","text":"數據幀，會調用"},{"type":"codeinline","content":[{"type":"text","text":"(*http2Framer).readMetaFrame"}]},{"type":"text","text":"對讀取到的數據幀解碼（源碼見https://github.com/golang/go/blob/master/src/net/http/h2_bundle.go#L2725）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲標頭壓縮具有較高的獨立性，所以筆者基於上面提到的編/解碼部分的源碼自己實現了一個可以獨立運行的小例子。本篇將基於自己實現的例子進行標頭壓縮分析（完整例子見https://github.com/Isites/go-coder/blob/master/http2/hpack-example/main.go）。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"開門見山"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HTTP2使用 HPACK 壓縮格式壓縮請求和響應標頭元數據，這種格式採用下面兩種技術壓縮："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"通過靜態哈夫曼代碼對傳輸的標頭字段進行編碼，從而減小數據傳輸的大小。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"單個連接中，client和server共同維護一個相同的標頭字段索引列表（筆者稱爲HPACK索引列表），此列表在之後的傳輸中用作編解碼的參考。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本篇不對哈夫曼編碼做過多的闡述，主要對雙端共同維護的索引列表進行分析。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HPACK 壓縮上下文包含一個靜態表和一個動態表：靜態表在規範中定義，並提供了一個包含所有連接都可能使用的常用 HTTP 標頭字段的列表；動態表最初爲空，將根據在特定連接內交換的值進行更新。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"HPACK索引列表"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"認識靜/動態表需要先認識"},{"type":"codeinline","content":[{"type":"text","text":"headerFieldTable"}]},{"type":"text","text":"結構體，動態表和靜態表都是基於它實現的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"type headerFieldTable struct {\n\t// As in hpack, unique ids are 1-based. The unique id for ents[k] is k + evictCount + 1.\n\tents []HeaderField\n\tevictCount uint64\n\n\t// byName maps a HeaderField name to the unique id of the newest entry with the same name.\n\tbyName map[string]uint64\n\n\t// byNameValue maps a HeaderField name/value pair to the unique id of the newest\n\tbyNameValue map[pairNameValue]uint64\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面將對上述的字段分別進行描述："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"ents"}]},{"type":"text","text":"：entries的縮寫，代表着當前已經索引的Header數據。在headerFieldTable中，每一個Header都有一個唯一的Id，以"},{"type":"codeinline","content":[{"type":"text","text":"ents[k]"}]},{"type":"text","text":"爲例，該唯一id的計算方式是"},{"type":"codeinline","content":[{"type":"text","text":"k + evictCount + 1"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"evictCount"}]},{"type":"text","text":"：已經從ents中刪除的條目數。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"byName"}]},{"type":"text","text":"：存儲具有相同Name的Header的唯一Id，最新Header的Name會覆蓋老的唯一Id。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"byNameValue"}]},{"type":"text","text":"：以Header的Name和Value爲key存儲對應的唯一Id。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對字段的含義有所瞭解後，接下來對headerFieldTable幾個比較重要的行爲進行描述。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"headerFieldTable).addEntry"},{"type":"text","text":"：添加Header實體到表中"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (t *headerFieldTable) addEntry(f HeaderField) {\n\tid := uint64(t.len()) + t.evictCount + 1\n\tt.byName[f.Name] = id\n\tt.byNameValue[pairNameValue{f.Name, f.Value}] = id\n\tt.ents = append(t.ents, f)\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，計算出Header在headerFieldTable中的唯一Id，並將其分別存入"},{"type":"codeinline","content":[{"type":"text","text":"byName"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"byNameValue"}]},{"type":"text","text":"中。最後，將Header存入"},{"type":"codeinline","content":[{"type":"text","text":"ents"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲使用了append函數，這意味着"},{"type":"codeinline","content":[{"type":"text","text":"ents[0]"}]},{"type":"text","text":"存儲的是存活最久的Header。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"headerFieldTable).evictOldest"},{"type":"text","text":"：從表中刪除指定個數的Header實體"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (t *headerFieldTable) evictOldest(n int) {\n\tif n > t.len() {\n\t\tpanic(fmt.Sprintf(\"evictOldest(%v) on table with %v entries\", n, t.len()))\n\t}\n\tfor k := 0; k < n; k++ {\n\t\tf := t.ents[k]\n\t\tid := t.evictCount + uint64(k) + 1\n\t\tif t.byName[f.Name] == id {\n\t\t\tdelete(t.byName, f.Name)\n\t\t}\n\t\tif p := (pairNameValue{f.Name, f.Value}); t.byNameValue[p] == id {\n\t\t\tdelete(t.byNameValue, p)\n\t\t}\n\t}\n\tcopy(t.ents, t.ents[n:])\n\tfor k := t.len() - n; k < t.len(); k++ {\n\t\tt.ents[k] = HeaderField{} // so strings can be garbage collected\n\t}\n\tt.ents = t.ents[:t.len()-n]\n\tif t.evictCount+uint64(n) < t.evictCount {\n\t\tpanic(\"evictCount overflow\")\n\t}\n\tt.evictCount += uint64(n)\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一個for循環的下標是從0開始的，也就是說刪除Header時遵循先進先出的原則。刪除Header的步驟如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"刪除"},{"type":"codeinline","content":[{"type":"text","text":"byName"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"byNameValue"}]},{"type":"text","text":"的映射。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"將第n位及其之後的Header前移。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"將倒數的n個Header置空，以方便垃圾回收。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"改變ents的長度。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"增加"},{"type":"codeinline","content":[{"type":"text","text":"evictCount"}]},{"type":"text","text":"的數量。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"headerFieldTable).search"},{"type":"text","text":"：從當前表中搜索指定Header並返回在當前表中的Index（此處的"},{"type":"codeinline","content":[{"type":"text","text":"Index"}]},{"type":"text","text":"和切片中的下標含義是不一樣的）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (t *headerFieldTable) search(f HeaderField) (i uint64, nameValueMatch bool) {\n\tif !f.Sensitive {\n\t\tif id := t.byNameValue[pairNameValue{f.Name, f.Value}]; id != 0 {\n\t\t\treturn t.idToIndex(id), true\n\t\t}\n\t}\n\tif id := t.byName[f.Name]; id != 0 {\n\t\treturn t.idToIndex(id), false\n\t}\n\treturn 0, false\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果Header的Name和Value均匹配，則返回當前表中的Index且"},{"type":"codeinline","content":[{"type":"text","text":"nameValueMatch"}]},{"type":"text","text":"爲true。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果僅有Header的Name匹配，則返回當前表中的Index且"},{"type":"codeinline","content":[{"type":"text","text":"nameValueMatch"}]},{"type":"text","text":"爲false。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果Header的Name和Value均不匹配，則返回0且"},{"type":"codeinline","content":[{"type":"text","text":"nameValueMatch"}]},{"type":"text","text":"爲false。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"headerFieldTable).idToIndex"},{"type":"text","text":"：通過當前表中的唯一Id計算出當前表對應的Index"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (t *headerFieldTable) idToIndex(id uint64) uint64 {\n\tif id <= t.evictCount {\n\t\tpanic(fmt.Sprintf(\"id (%v) <= evictCount (%v)\", id, t.evictCount))\n\t}\n\tk := id - t.evictCount - 1 // convert id to an index t.ents[k]\n\tif t != staticTable {\n\t\treturn uint64(t.len()) - k // dynamic table\n\t}\n\treturn k + 1\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"靜態表："},{"type":"codeinline","content":[{"type":"text","text":"Index"}]},{"type":"text","text":"從1開始，且Index爲1時對應的元素爲"},{"type":"codeinline","content":[{"type":"text","text":"t.ents[0]"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"動態表: "},{"type":"codeinline","content":[{"type":"text","text":"Index"}]},{"type":"text","text":"也從1開始，但是Index爲1時對應的元素爲"},{"type":"codeinline","content":[{"type":"text","text":"t.ents[t.len()-1]"}]},{"type":"text","text":"。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"靜態表"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"靜態表中包含了一些每個連接都可能使用到的Header。其實現如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"var staticTable = newStaticTable()\nfunc newStaticTable() *headerFieldTable {\n\tt := &headerFieldTable{}\n\tt.init()\n\tfor _, e := range staticTableEntries[:] {\n\t\tt.addEntry(e)\n\t}\n\treturn t\n}\nvar staticTableEntries = [...]HeaderField{\n\t{Name: \":authority\"},\n\t{Name: \":method\", Value: \"GET\"},\n\t{Name: \":method\", Value: \"POST\"},\n // 此處省略代碼\n\t{Name: \"www-authenticate\"},\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的"},{"type":"codeinline","content":[{"type":"text","text":"t.init"}]},{"type":"text","text":"函數僅做初始化"},{"type":"codeinline","content":[{"type":"text","text":"t.byName"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"t.byNameValue"}]},{"type":"text","text":"用。筆者在這裏僅展示了部分預定義的Header，完整預定義Header參見https://github.com/golang/go/blob/master/src/vendor/golang.org/x/net/http2/hpack/tables.go#L130。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"動態表"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"動態表結構體如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"type dynamicTable struct {\n\t// http://http2.github.io/http2-spec/compression.html#rfc.section.2.3.2\n\ttable headerFieldTable\n\tsize uint32 // in bytes\n\tmaxSize uint32 // current maxSize\n\tallowedMaxSize uint32 // maxSize may go up to this, inclusive\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"動態表的實現是基於"},{"type":"codeinline","content":[{"type":"text","text":"headerFieldTable"}]},{"type":"text","text":"，相比原先的基礎功能增加了表的大小限制，其他功能保持不變。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"靜態表和動態表構成完整的HPACK索引列表"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面介紹了動/靜態表中內部的Index和內部的唯一Id，而在一次連接中HPACK索引列表是由靜態表和動態表一起構成，那此時在連接中的HPACK索引是怎麼樣的呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"帶着這樣的疑問我們看看下面的結構："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a06c96380231681b43fc4b6d8f35f99.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖中藍色部分表示靜態表，黃色部分表示動態表。\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"H1...Hn"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"H1...Hm"}]},{"type":"text","text":"分別表示存儲在靜態表和動態表中的Header元素。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"在HPACK索引中靜態表部分的索引和靜態表的內部索引保持一致，動態表部分的索引爲動態表內部索引加上靜態表索引的最大值"},{"type":"text","text":"。在一次連接中Client和Server通過HPACK索引標識唯一的Header元素。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"HPACK編碼"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"衆所周知HTTP2的標頭壓縮能夠減少很多數據的傳輸，接下來我們通過下面的例子，對比一下編碼前後的數據大小："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"var (\n buf bytes.Buffer\n oriSize int\n)\nhenc := hpack.NewEncoder(&buf)\nheaders := []hpack.HeaderField{\n {Name: \":authority\", Value: \"dss0.bdstatic.com\"},\n {Name: \":method\", Value: \"GET\"},\n {Name: \":path\", Value: \"/5aV1bjqh_Q23odCf/static/superman/img/topnav/[email protected]\"},\n {Name: \":scheme\", Value: \"https\"},\n {Name: \"accept-encoding\", Value: \"gzip\"},\n {Name: \"user-agent\", Value: \"Go-http-client/2.0\"},\n {Name: \"custom-header\", Value: \"custom-value\"},\n}\nfor _, header := range headers {\n oriSize += len(header.Name) + len(header.Value)\n henc.WriteField(header)\n}\nfmt.Printf(\"ori size: %v, encoded size: %v\\n\", oriSize, buf.Len())\n//輸出爲：ori size: 197, encoded size: 111"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：在 HTTP2 中，請求和響應標頭字段的定義保持不變，僅有一些微小的差異：所有標頭字段名稱均爲小寫，請求行現在拆分成各個 "},{"type":"codeinline","content":[{"type":"text","text":":method"}]},{"type":"text","text":"、"},{"type":"codeinline","content":[{"type":"text","text":":scheme"}]},{"type":"text","text":"、"},{"type":"codeinline","content":[{"type":"text","text":":authority"}]},{"type":"text","text":" 和 "},{"type":"codeinline","content":[{"type":"text","text":":path"}]},{"type":"text","text":" 僞標頭字段。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上面的例子中，我們看到原來爲197字節的標頭數據現在只有111字節，減少了近一半的數據量！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"帶着一種 “臥槽，牛逼！”的心情開始對"},{"type":"codeinline","content":[{"type":"text","text":"henc.WriteField"}]},{"type":"text","text":"方法調試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (e *Encoder) WriteField(f HeaderField) error {\n\te.buf = e.buf[:0]\n\n\tif e.tableSizeUpdate {\n\t\te.tableSizeUpdate = false\n\t\tif e.minSize < e.dynTab.maxSize {\n\t\t\te.buf = appendTableSize(e.buf, e.minSize)\n\t\t}\n\t\te.minSize = uint32Max\n\t\te.buf = appendTableSize(e.buf, e.dynTab.maxSize)\n\t}\n\n\tidx, nameValueMatch := e.searchTable(f)\n\tif nameValueMatch {\n\t\te.buf = appendIndexed(e.buf, idx)\n\t} else {\n\t\tindexing := e.shouldIndex(f)\n\t\tif indexing {\n\t\t\te.dynTab.add(f) // 加入動態表中\n\t\t}\n\n\t\tif idx == 0 {\n\t\t\te.buf = appendNewName(e.buf, f, indexing)\n\t\t} else {\n\t\t\te.buf = appendIndexedName(e.buf, f, idx, indexing)\n\t\t}\n\t}\n\tn, err := e.w.Write(e.buf)\n\tif err == nil && n != len(e.buf) {\n\t\terr = io.ErrShortWrite\n\t}\n\treturn err\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經調試發現，本例中"},{"type":"codeinline","content":[{"type":"text","text":":authority"}]},{"type":"text","text":"，"},{"type":"codeinline","content":[{"type":"text","text":":path"}]},{"type":"text","text":"，"},{"type":"codeinline","content":[{"type":"text","text":"accept-encoding"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"user-agent"}]},{"type":"text","text":"走了"},{"type":"codeinline","content":[{"type":"text","text":"appendIndexedName"}]},{"type":"text","text":"分支；"},{"type":"codeinline","content":[{"type":"text","text":":method"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":":scheme"}]},{"type":"text","text":"走了"},{"type":"codeinline","content":[{"type":"text","text":"appendIndexed"}]},{"type":"text","text":"分支；"},{"type":"codeinline","content":[{"type":"text","text":"custom-header"}]},{"type":"text","text":"走了"},{"type":"codeinline","content":[{"type":"text","text":"appendNewName"}]},{"type":"text","text":"分支。這三種分支總共代表了兩種不同的編碼方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於本例中"},{"type":"codeinline","content":[{"type":"text","text":"f.Sensitive"}]},{"type":"text","text":"默認值爲false且Encoder給動態表的默認大小爲4096，按照"},{"type":"codeinline","content":[{"type":"text","text":"e.shouldIndex"}]},{"type":"text","text":"的邏輯本例中"},{"type":"codeinline","content":[{"type":"text","text":"indexing"}]},{"type":"text","text":"一直爲true（在筆者所使用的go1.14.2源碼中，client端尚未發現有使"},{"type":"codeinline","content":[{"type":"text","text":"f.Sensitive"}]},{"type":"text","text":"爲true的代碼）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"筆者對上面"},{"type":"codeinline","content":[{"type":"text","text":"e.tableSizeUpdate"}]},{"type":"text","text":"相關的邏輯不提的原因是控制"},{"type":"codeinline","content":[{"type":"text","text":"e.tableSizeUpdate"}]},{"type":"text","text":"的方法爲"},{"type":"codeinline","content":[{"type":"text","text":"e.SetMaxDynamicTableSizeLimit"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"e.SetMaxDynamicTableSize"}]},{"type":"text","text":"，而筆者在"},{"type":"codeinline","content":[{"type":"text","text":"(*http2Transport).newClientConn"}]},{"type":"text","text":"（此方法相關邏輯參見前篇）相關的源碼中發現了這樣的註釋："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"// TODO: SetMaxDynamicTableSize, SetMaxDynamicTableSizeLimit on\n// henc in response to SETTINGS frames?"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"筆者看到這裏的時候內心激動不已呀，產生了一種強烈的想貢獻代碼的慾望，奈何自己能力有限只能看着機會卻抓不住呀，只好含恨埋頭苦學（開個玩笑～，畢竟某位智者說過，寫的越少BUG越少😄）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"Encoder).searchTable"},{"type":"text","text":"：從HPACK索引列表中搜索Header，並返回對應的索引。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (e *Encoder) searchTable(f HeaderField) (i uint64, nameValueMatch bool) {\n\ti, nameValueMatch = staticTable.search(f)\n\tif nameValueMatch {\n\t\treturn i, true\n\t}\n\n\tj, nameValueMatch := e.dynTab.table.search(f)\n\tif nameValueMatch || (i == 0 && j != 0) {\n\t\treturn j + uint64(staticTable.len()), nameValueMatch\n\t}\n\n\treturn i, false\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"搜索順序爲，先搜索靜態表，如果靜態表不匹配，則搜索動態表，最後返回。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"索引Header表示法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此表示法對應的函數爲"},{"type":"text","marks":[{"type":"strong"}],"text":"appendIndexed"},{"type":"text","text":"，且該Header已經在索引列表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"該函數將Header在HPACK索引列表中的索引編碼，原先的Header最後僅用少量的幾個字節就可以表示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func appendIndexed(dst []byte, i uint64) []byte {\n\tfirst := len(dst)\n\tdst = appendVarInt(dst, 7, i)\n\tdst[first] |= 0x80\n\treturn dst\n}\nfunc appendVarInt(dst []byte, n byte, i uint64) []byte {\n\tk := uint64((1 << n) - 1)\n\tif i < k {\n\t\treturn append(dst, byte(i))\n\t}\n\tdst = append(dst, byte(k))\n\ti -= k\n\tfor ; i >= 128; i >>= 7 {\n\t\tdst = append(dst, byte(0x80|(i&0x7f)))\n\t}\n\treturn append(dst, byte(i))\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由"},{"type":"codeinline","content":[{"type":"text","text":"appendIndexed"}]},{"type":"text","text":"知，用索引頭字段表示法時，第一個字節的格式必須是"},{"type":"codeinline","content":[{"type":"text","text":"0b1xxxxxxx"}]},{"type":"text","text":"，即第0位必須爲"},{"type":"codeinline","content":[{"type":"text","text":"1"}]},{"type":"text","text":"，低7位用來表示值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果索引大於"},{"type":"codeinline","content":[{"type":"text","text":"uint64((1 << n) - 1)"}]},{"type":"text","text":"時，需要使用多個字節來存儲索引的值，步驟如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"第一個字節的最低n位全爲1。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"索引i減去uint64((1 << n) - 1)後，每次取低7位或上"},{"type":"codeinline","content":[{"type":"text","text":"0b10000000"}]},{"type":"text","text":"， 然後i右移7位並和128進行比較，判斷是否進入下一次循環。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"循環結束後將剩下的i值直接放入buf中。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用這種方法表示Header時，僅需要少量字節就可以表示一個完整的Header頭字段，最好的情況是一個字節就可以表示一個Header字段。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"增加動態表Header表示法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此種表示法對應兩種情況：一，Header的Name有匹配索引；二，Header的Name和Value均無匹配索引。這兩種情況分別對應的處理函數爲"},{"type":"codeinline","content":[{"type":"text","text":"appendIndexedName"}]},{"type":"text","text":"和"},{"type":"codeinline","content":[{"type":"text","text":"appendNewName"}]},{"type":"text","text":"。這兩種情況均會將Header添加到動態表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"appendIndexedName："},{"type":"text","text":" 編碼有Name匹配的Header字段。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func appendIndexedName(dst []byte, f HeaderField, i uint64, indexing bool) []byte {\n\tfirst := len(dst)\n\tvar n byte\n\tif indexing {\n\t\tn = 6\n\t} else {\n\t\tn = 4\n\t}\n\tdst = appendVarInt(dst, n, i)\n\tdst[first] |= encodeTypeByte(indexing, f.Sensitive)\n\treturn appendHpackString(dst, f.Value)\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這裏我們先看看"},{"type":"codeinline","content":[{"type":"text","text":"encodeTypeByte"}]},{"type":"text","text":"函數："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func encodeTypeByte(indexing, sensitive bool) byte {\n\tif sensitive {\n\t\treturn 0x10\n\t}\n\tif indexing {\n\t\treturn 0x40\n\t}\n\treturn 0\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面提到本例中indexing一直爲true，sensitive爲false，所以encodeTypeByte的返回值一直爲"},{"type":"codeinline","content":[{"type":"text","text":"0x40"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此時回到appendIndexedName函數，我們知道增加動態表Header表示法的第一個字節格式必須是"},{"type":"codeinline","content":[{"type":"text","text":"0xb01xxxxxx"}]},{"type":"text","text":"，即最高兩位必須是"},{"type":"codeinline","content":[{"type":"text","text":"01"}]},{"type":"text","text":"，低6位用於表示Header中Name的索引。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過"},{"type":"codeinline","content":[{"type":"text","text":"appendVarInt"}]},{"type":"text","text":"對索引編碼後，下面我們看看"},{"type":"codeinline","content":[{"type":"text","text":"appendHpackString"}]},{"type":"text","text":"函數如何對Header的Value進行編碼："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func appendHpackString(dst []byte, s string) []byte {\n\thuffmanLength := HuffmanEncodeLength(s)\n\tif huffmanLength < uint64(len(s)) {\n\t\tfirst := len(dst)\n\t\tdst = appendVarInt(dst, 7, huffmanLength)\n\t\tdst = AppendHuffmanString(dst, s)\n\t\tdst[first] |= 0x80\n\t} else {\n\t\tdst = appendVarInt(dst, 7, uint64(len(s)))\n\t\tdst = append(dst, s...)\n\t}\n\treturn dst\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"appendHpackString"}]},{"type":"text","text":"編碼時分爲兩種情況："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"哈夫曼編碼後的長度小於原Value的長度時，先用"},{"type":"codeinline","content":[{"type":"text","text":"appendVarInt"}]},{"type":"text","text":"將哈夫曼編碼後的最終長度存入buf，然後再將真實的哈夫曼編碼存入buf。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"哈夫曼編碼後的長度大於等於原Value的長度時，先用"},{"type":"codeinline","content":[{"type":"text","text":"appendVarInt"}]},{"type":"text","text":"將原Value的長度存入buf，然後再將原Value存入buf。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這裏需要注意的是存儲Value長度時僅用了字節的低7位，最高位爲1表示存儲的內容爲哈夫曼編碼，最高位爲0表示存儲的內容爲原Value。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"appendNewName："},{"type":"text","text":" 編碼Name和Value均無匹配的Header字段。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func appendNewName(dst []byte, f HeaderField, indexing bool) []byte {\n\tdst = append(dst, encodeTypeByte(indexing, f.Sensitive))\n\tdst = appendHpackString(dst, f.Name)\n\treturn appendHpackString(dst, f.Value)\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面提到"},{"type":"codeinline","content":[{"type":"text","text":"encodeTypeByte"}]},{"type":"text","text":"的返回值爲"},{"type":"codeinline","content":[{"type":"text","text":"0x40"}]},{"type":"text","text":"，所以我們此時編碼的第一個字節爲"},{"type":"codeinline","content":[{"type":"text","text":"0b01000000"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一個字節編碼結束後通過"},{"type":"codeinline","content":[{"type":"text","text":"appendHpackString"}]},{"type":"text","text":"先後對Header的Name和Value進行編碼。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"HPACK解碼"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面理了一遍HPACK的編碼過程，下面我們通過一個解碼的例子來理一遍解碼的過程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"// 此處省略HPACK編碼中的編碼例子\nvar (\n invalid error\n sawRegular bool\n // 16 << 20 from fr.maxHeaderListSize() from\n remainSize uint32 = 16 << 20\n)\nhdec := hpack.NewDecoder(4096, nil)\n// 16 << 20 from fr.maxHeaderStringLen() from fr.maxHeaderListSize()\nhdec.SetMaxStringLength(int(remainSize))\nhdec.SetEmitFunc(func(hf hpack.HeaderField) {\n if !httpguts.ValidHeaderFieldValue(hf.Value) {\n invalid = fmt.Errorf(\"invalid header field value %q\", hf.Value)\n }\n isPseudo := strings.HasPrefix(hf.Name, \":\")\n if isPseudo {\n if sawRegular {\n invalid = errors.New(\"pseudo header field after regular\")\n }\n } else {\n sawRegular = true\n // if !http2validWireHeaderFieldName(hf.Name) {\n // \tinvliad = fmt.Sprintf(\"invalid header field name %q\", hf.Name)\n // }\n }\n if invalid != nil {\n fmt.Println(invalid)\n hdec.SetEmitEnabled(false)\n return\n }\n size := hf.Size()\n if size > remainSize {\n hdec.SetEmitEnabled(false)\n // mh.Truncated = true\n return\n }\n remainSize -= size\n fmt.Printf(\"%+v\\n\", hf)\n // mh.Fields = append(mh.Fields, hf)\n})\ndefer hdec.SetEmitFunc(func(hf hpack.HeaderField) {})\nfmt.Println(hdec.Write(buf.Bytes()))\n// 輸出如下：\n// ori size: 197, encoded size: 111\n// header field \":authority\" = \"dss0.bdstatic.com\"\n// header field \":method\" = \"GET\"\n// header field \":path\" = \"/5aV1bjqh_Q23odCf/static/superman/img/topnav/[email protected]\"\n// header field \":scheme\" = \"https\"\n// header field \"accept-encoding\" = \"gzip\"\n// header field \"user-agent\" = \"Go-http-client/2.0\"\n// header field \"custom-header\" = \"custom-value\"\n// 111 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過最後一行的輸出可以知道確確實實從111個字節中解碼出了197個字節的原Header數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而這解碼的過程筆者將從"},{"type":"codeinline","content":[{"type":"text","text":"hdec.Write"}]},{"type":"text","text":"方法開始分析，逐步揭開它的神祕面紗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":" func (d *Decoder) Write(p []byte) (n int, err error) {\n // 此處省略代碼\n\tif d.saveBuf.Len() == 0 {\n\t\td.buf = p\n\t} else {\n\t\td.saveBuf.Write(p)\n\t\td.buf = d.saveBuf.Bytes()\n\t\td.saveBuf.Reset()\n\t}\n\n\tfor len(d.buf) > 0 {\n\t\terr = d.parseHeaderFieldRepr()\n\t\tif err == errNeedMore {\n\t\t\t// 此處省略代碼\n\t\t\td.saveBuf.Write(d.buf)\n\t\t\treturn len(p), nil\n\t\t}\n\t\t// 此處省略代碼\n\t}\n\treturn len(p), err\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在筆者debug的過程中發現解碼的核心邏輯主要在"},{"type":"codeinline","content":[{"type":"text","text":"d.parseHeaderFieldRepr"}]},{"type":"text","text":"方法裏。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (d *Decoder) parseHeaderFieldRepr() error {\n\tb := d.buf[0]\n\tswitch {\n\tcase b&128 != 0:\n\t\treturn d.parseFieldIndexed()\n\tcase b&192 == 64:\n\t\treturn d.parseFieldLiteral(6, indexedTrue)\n // 此處省略代碼\n\t}\n\treturn DecodingError{errors.New(\"invalid encoding\")}\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一個字節與上128不爲0只有一種情況，那就是b爲"},{"type":"codeinline","content":[{"type":"text","text":"0b1xxxxxxx"}]},{"type":"text","text":"格式的數據，綜合前面的編碼邏輯可以知道索引Header表示法對應的解碼方法爲"},{"type":"codeinline","content":[{"type":"text","text":"d.parseFieldIndexed"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一個字節與上192爲64也只有一種情況，那就是b爲"},{"type":"codeinline","content":[{"type":"text","text":"0b01xxxxxx"}]},{"type":"text","text":"格式的數據，綜合前面的編碼邏輯可以知道增加動態表Header表示法對應的解碼方法爲"},{"type":"codeinline","content":[{"type":"text","text":"d.parseFieldLiteral"}]},{"type":"text","text":"。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"索引Header表示法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過"},{"type":"codeinline","content":[{"type":"text","text":"(*Decoder).parseFieldIndexed"}]},{"type":"text","text":"解碼時，真實的Header數據已經在靜態表或者動態表中了，只要通過HPACK索引找到對應的Header就解碼成功了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (d *Decoder) parseFieldIndexed() error {\n\tbuf := d.buf\n\tidx, buf, err := readVarInt(7, buf)\n\tif err != nil {\n\t\treturn err\n\t}\n\thf, ok := d.at(idx)\n\tif !ok {\n\t\treturn DecodingError{InvalidIndexError(idx)}\n\t}\n\td.buf = buf\n\treturn d.callEmit(HeaderField{Name: hf.Name, Value: hf.Value})\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述方法主要有三個步驟："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"通過"},{"type":"codeinline","content":[{"type":"text","text":"readVarInt"}]},{"type":"text","text":"函數讀取HPACK索引。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"通過"},{"type":"codeinline","content":[{"type":"text","text":"d.at"}]},{"type":"text","text":"方法找到索引列表中真實的Header數據。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"將Header傳遞給最上層。"},{"type":"codeinline","content":[{"type":"text","text":"d.CallEmit"}]},{"type":"text","text":"最終會調用"},{"type":"codeinline","content":[{"type":"text","text":"hdec.SetEmitFunc"}]},{"type":"text","text":"設置的閉包，從而將Header傳遞給最上層。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"readVarInt"},{"type":"text","text":"：讀取HPACK索引"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func readVarInt(n byte, p []byte) (i uint64, remain []byte, err error) {\n\tif n < 1 || n > 8 {\n\t\tpanic(\"bad n\")\n\t}\n\tif len(p) == 0 {\n\t\treturn 0, p, errNeedMore\n\t}\n\ti = uint64(p[0])\n\tif n < 8 {\n\t\ti &= (1 << uint64(n)) - 1\n\t}\n\tif i < (1< 0 {\n\t\tb := p[0]\n\t\tp = p[1:]\n\t\ti += uint64(b&127) << m\n\t\tif b&128 == 0 {\n\t\t\treturn i, p, nil\n\t\t}\n\t\tm += 7\n\t\tif m >= 63 { // TODO: proper overflow check. making this up.\n\t\t\treturn 0, origP, errVarintOverflow\n\t\t}\n\t}\n\treturn 0, origP, errNeedMore\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由上述的readVarInt函數知，當第一個字節的低n爲不全爲1時，則低n爲代表真實的HPACK索引，可以直接返回。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當第一個字節的低n爲全爲1時，需要讀取更多的字節數來計算真正的HPACK索引。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"第一次循環時m爲0，b的低7位加上"},{"type":"codeinline","content":[{"type":"text","text":"(1< uint64(d.maxTableIndex()) {\n\t\treturn\n\t}\n\tdt := d.dynTab.table\n\treturn dt.ents[dt.len()-(int(i)-staticTable.len())], true\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引小於靜態表長度時，直接從靜態表中獲取Header數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引長度大於靜態表時，根據前面介紹的"},{"type":"text","marks":[{"type":"strong"}],"text":"HPACK索引列表"},{"type":"text","text":"，可以通過"},{"type":"codeinline","content":[{"type":"text","text":"dt.len()-(int(i)-staticTable.len())"}]},{"type":"text","text":"計算出i在動態表"},{"type":"codeinline","content":[{"type":"text","text":"ents"}]},{"type":"text","text":"的真實下標，從而獲取Header數據。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"增加動態表Header表示法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過"},{"type":"codeinline","content":[{"type":"text","text":"(*Decoder).parseFieldLiteral"}]},{"type":"text","text":"解碼時，需要考慮兩種情況。一、Header的Name有索引。二、Header的Name和Value均無索引。這兩種情況下，該Header都不存在於動態表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面分步驟分析"},{"type":"codeinline","content":[{"type":"text","text":"(*Decoder).parseFieldLiteral"}]},{"type":"text","text":"方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、讀取buf中的HPACK索引。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"nameIdx, buf, err := readVarInt(n, buf)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、 如果索引不爲0，則從HPACK索引列表中獲取Header的Name。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"ihf, ok := d.at(nameIdx)\nif !ok {\n return DecodingError{InvalidIndexError(nameIdx)}\n}\nhf.Name = ihf.Name"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3、如果索引爲0，則從buf中讀取Header的Name。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"hf.Name, buf, err = d.readString(buf, wantStr)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4、從buf中讀取Header的Value，並將完整的Header添加到動態表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"hf.Value, buf, err = d.readString(buf, wantStr)\nif err != nil {\n return err\n}\nd.buf = buf\nif it.indexed() {\n d.dynTab.add(hf)\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"(*"},{"type":"text","marks":[{"type":"strong"}],"text":"Decoder).readString"},{"type":"text","text":"： 從編碼的字節數據中讀取真實的Header數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"func (d *Decoder) readString(p []byte, wantStr bool) (s string, remain []byte, err error) {\n\tif len(p) == 0 {\n\t\treturn \"\", p, errNeedMore\n\t}\n\tisHuff := p[0]&128 != 0\n\tstrLen, p, err := readVarInt(7, p)\n\t// 省略校驗邏輯\n\tif !isHuff {\n\t\tif wantStr {\n\t\t\ts = string(p[:strLen])\n\t\t}\n\t\treturn s, p[strLen:], nil\n\t}\n\n\tif wantStr {\n\t\tbuf := bufPool.Get().(*bytes.Buffer)\n\t\tbuf.Reset() // don't trust others\n\t\tdefer bufPool.Put(buf)\n\t\tif err := huffmanDecode(buf, d.maxStrLen, p[:strLen]); err != nil {\n\t\t\tbuf.Reset()\n\t\t\treturn \"\", nil, err\n\t\t}\n\t\ts = buf.String()\n\t\tbuf.Reset() // be nice to GC\n\t}\n\treturn s, p[strLen:], nil\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 首先判斷字節數據是否是哈夫曼編碼（和前面的"},{"type":"codeinline","content":[{"type":"text","text":"appendHpackString"}]},{"type":"text","text":"函數對應），然後通過"},{"type":"codeinline","content":[{"type":"text","text":"readVarInt"}]},{"type":"text","text":"讀取數據的長度並賦值給"},{"type":"codeinline","content":[{"type":"text","text":"strLen"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果不是哈夫曼編碼，則直接返回"},{"type":"codeinline","content":[{"type":"text","text":"strLen"}]},{"type":"text","text":"長度的數據。如果是哈夫曼編碼，讀取"},{"type":"codeinline","content":[{"type":"text","text":"strLen"}]},{"type":"text","text":"長度的數據，並用哈夫曼算法解碼後再返回。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"驗證&總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在前面我們已經瞭解了HPACK索引列表，以及基於HPACK索引列表的編/解碼流程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面筆者最後驗證一下已經編解碼過後的Header，再次編解碼時的大小。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"go"},"content":[{"type":"text","text":"// 此處省略前面HAPACK編碼和HPACK解碼的demo\n// try again\nfmt.Println(\"try again: \")\nbuf.Reset()\nhenc.WriteField(hpack.HeaderField{Name: \"custom-header\", Value: \"custom-value\"}) // 編碼已經編碼過後的Header\nfmt.Println(hdec.Write(buf.Bytes())) // 解碼\n// 輸出：\n// ori size: 197, encoded size: 111\n// header field \":authority\" = \"dss0.bdstatic.com\"\n// header field \":method\" = \"GET\"\n// header field \":path\" = \"/5aV1bjqh_Q23odCf/static/superman/img/topnav/[email protected]\"\n// header field \":scheme\" = \"https\"\n// header field \"accept-encoding\" = \"gzip\"\n// header field \"user-agent\" = \"Go-http-client/2.0\"\n// header field \"custom-header\" = \"custom-value\"\n// 111 \n// try again:\n// header field \"custom-header\" = \"custom-value\"\n// 1 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由上面最後一行的輸出可知，解碼僅用了一個字節，即本例中編碼一個已經編碼過的Header也僅需一個字節。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"綜上：在一個連接上，client和server維護一個相同的HPACK索引列表，多個請求在發送和接收Header數據時可以分爲兩種情況。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"Header在HPACK索引列表裏面，可以不用傳輸真實的Header數據僅需傳輸HPACK索引從而達到標頭壓縮的目的。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"Header不在HPACK索引列表裏面，對大多數Header而言也僅需傳輸Header的Value以及Name的HPACK索引，從而減少Header數據的傳輸。同時，在發送和接收這樣的Header數據時會更新各自的HPACK索引列表，以保證下一個請求傳輸的Header數據儘可能的少。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，由衷的感謝將HTTP2.0系列讀完的讀者，真誠的希望各位讀者能夠有所收穫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果大家有什麼疑問可以在評論區和諧地討論，筆者看到了也會及時回覆，願大家一起進步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"注"},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1. 寫本文時， 筆者所用go版本爲: go1.14.2"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2. 索引Header表示法和增加動態表Header表示法均爲筆者自主命名，主要便於讀者理解。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"參考："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https://developers.google.com/web/fundamentals/performance/http2?hl=zh-cn"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}