TCP性能分析與調優策略

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"網絡傳輸"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳播延遲: 消息從發送端到接收端需要的時間，是信號傳播距離和速度的函數"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳輸延遲: 把消息中的所有比特轉移到鏈路中需要的時間，是消息長度和鏈路速率的函數"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"處理延遲: 處理分組首部、檢查位錯誤及確定分組目標所需的時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"排隊延遲: 到來的分組排隊等待處理的時間"}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CDN把距離縮短，以加快訪問速度"}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"延遲的最後一公里"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"延遲相當大一部分往往花在最後的幾公里，因爲客戶單端連接公網的方式和接入鏈路都比較差"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"TCP"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c7/c740e71f444886f60f16eef29b8b0815.png","alt":"web-sync","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"時延"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每個http連接都需要經過三次握手，從紐約向倫敦請求，啓動一次TCP連接，光三次握手至少要花56ms，向倫敦發送分組需要28ms，響應要28ms。可以看出三次握手帶來的延遲是非常大的"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"擁塞控制"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"擁塞崩潰"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可能是往返時間超過了所有主機的最大中斷間隔，於是相應的主機會產生越來越多的副本，使整個網絡陷入癱瘓，最終個交換節點的緩衝區被填滿，多出來的分組必須刪除"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"流量控制"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過縮放接收窗口（rwnd）的大小來控制流量的發送（可配）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/69/6984d9033ac74d83745072590e6f60bc.png","alt":"web-rwnd","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"慢啓動"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過一個動態可變的擁塞窗口（cwnd）大小來控制流量的發送，網絡可發送的最大數據取rwnd和cwnd的最小值"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7d/7d1082df30d4198922780cfd5e3a9bc3.png","alt":"web-slowstart","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖可見，一個請求需要經過220ms纔可以達到最大速率。因爲慢啓動限制了可用的吞吐量，對於小文件的傳輸是很不利的"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"慢啓動重啓：在連接空閒一段時間後，重置擁塞窗口到一個安全的默認值。毫無疑問，SSR對於長週期空閒而突發請求的TCP連接有很大的影響，建議服務器金庸SSR"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/22/22ff9c84b51b1f91a890c1d33bf7295d.png","alt":"web-cwnd","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"擁塞預防"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"慢啓動每次往返都會成倍提高傳輸的數據量，知道超過接收端的流量控制窗口或者有分組丟失爲止。此時擁塞預防算法接入"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/38/3874ce91ac4a2b71285c316b970c4417.png","alt":"web-defint","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"帶寬延遲積"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BDP表示數據鏈路的容量與其端到端延遲的乘積，結果就是任意時刻在途未確認的最大數據量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"發送端和接收端發送超過了未被確認的最大數據量，都會停下來等待對方的ACK，這就造成了數據缺口。爲了解決這個問題應該設置窗口足夠大，過小的窗口會限制連接的吞吐量。窗口的大小最小應該設置爲BDP"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"隊首阻塞"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP按序交付與可靠交付，如果有時丟包，那麼後續的包必須等到這個丟包的數據重發並接收，才能交付給應用程序，這就導致讀數據時會感覺延遲交付"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在應用程序不關係按序交付和可靠交付的情況下TCP並不是最好的選擇。例如音頻，丟了一個包可以在音頻中插入一個小小的間隙，就可以繼續處理後面的包，只要間隙足夠小，用戶就注意不到，而等待丟包可能導致音頻輸出產生無法預料的。相對而言，後者的用戶體驗更差."}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"調優"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"原因"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 三次握手增加了整整一次往返時間；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 慢啓動將被應用到每個新連接；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 流量及擁塞控制會影響所有連接的吞吐量；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 的吞吐量由當前擁塞窗口大小控制。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"方案"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把服務器內核升級到最新版本（Linux：3.2+）；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"確保 cwnd 大小爲 10；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"禁用空閒後的慢啓動；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"確保啓動窗口縮放；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"減少傳輸冗餘數據；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"壓縮要傳輸的數據；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把服務器放到離用戶近的地方以減少往返時間；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"盡最大可能重用已經建立的 TCP 連接。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"UDP"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"網絡地址轉換"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c8/c8dcb0a1f7a0af59f3040b675ebee82f.png","alt":"web-nat","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f3/f3fd4102285e9d629bbb62b6a8545b35.png","alt":"web-nataddr","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這三段地址只允許私網擁有，不允許公網擁有這些ip"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"連接狀體超時"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"中轉UDP的路由，由於UDP沒有連接和終止的概念，這導致中轉路由不知道什麼時候該刪除連接狀態。爲了解決這個問題，路由器會定期清理，路由狀態一旦清除，UDP則需要重新建立。解決方法是定期雙向發keep-alive分組。按理TCP有明確的連接狀態，路由器應當可以完整把握TCP的生命週期，但是路由器沒有這麼做，也同樣給TCP設置了超時清理的動作，這導致一個長時間不活躍的TCP，會無端端的連接斷開。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"P2P"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"STUN: Session Traversal Utilities for NAT 是一個協議，可以讓內網應用程序獲得一個外網ip和端口，STUN服務器架設在公網"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/32/32cc35f4eb835c19ef8ed061d8d58479.png","alt":"web-stun","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"各自內網的應用程序使用STUN後，就能獲得一個外網ip，STUN服務器通過keepalive方式保持路由不超時，各應用程序就能直接UDP通訊了"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TURN: Traversal Using Relays around NAT. 當內網不能使用NAT時，可以使用TURN服務器，應用程序通過TCP連接TURN服務器，服務器做消息中轉"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"libjingle是谷歌的一個實現了STUN/TURN/ICE的開源庫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"92% 的時間可以直接連接（STUN）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"8% 的時間要使用中繼器（TURN）。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ICE: Interactive Connectivity Establishment協議，能直連就直連，不能直連則使用STUN，再不行則使用TURN"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"設計原則"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序必須容忍各種因特網路徑條件；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該控制傳輸速度；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該對所有流量進行擁塞控制；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該使用與 TCP 相近的帶寬；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該準備基於丟包的重發計數器；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該不發送大於路徑 MTU 的數據報；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該處理數據報丟失、重複和重排；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該足夠穩定以支持 2 分鐘以上的交付延遲；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序應該支持 IPv4 UDP 校驗和，必須支持 IPv6 校驗和；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用程序可以在需要時使用 keep-alive（最小間隔 15 秒）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"建議使用WebRTC"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"帶寬/延遲與頁面加載時間的關係"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/37/37325461ebfc6f0605fac1ba15cfab2d.png","alt":"web-pageload","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"時延的原因"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP握手/流量擁塞控制/丟包/隊首擁塞"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"網站各資源的用戶體驗度量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過Navigation Timing/User Timing/resource timing來度量"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"瀏覽器的優化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"資源預取和排定優先次序"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DNS預解析"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP預連接"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"頁面預渲染"}]}]}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"服務器如何利用這些優化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CSS 和 JavaScript 等重要資源應該儘早在文檔中出現；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應該儘早交付 CSS，從而解除渲染阻塞並讓 JavaScript 執行；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"非關鍵性 JavaScript 應該推遲，以避免阻塞 DOM 和 CSSOM 構建；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HTML 文檔由解析器遞增解析，從而保證文檔可以間隙性發送，以求得最佳性能"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/dd/dd0c1330272079ddc30733b7813c861b.png","alt":"web-preload","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"http優化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"減少DNS查詢"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"減少HTTP請求"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用CDN"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"添加Expires首部並配置ETag標籤"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Gzip資源"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"避免HTTP重定向"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用持久化連接"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"keep alive和連接池的侷限"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於每個服務器的ip，客戶端維持一個長連接的連接池，如果有多個請求發往服務器，並超過連接池的數量，則會迫使客戶端必須等待連接池的空閒，而且啓用多個socket嚴重佔用系統資源。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"http協議的侷限"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每次發送http請求，都必須加上頭部，而頭部是沒有經過壓縮的，這直接導致頭部的長度有可能超過body的長度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}