TCP性能分析与调优策略

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"网络传输"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"传播延迟: 消息从发送端到接收端需要的时间，是信号传播距离和速度的函数"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"传输延迟: 把消息中的所有比特转移到链路中需要的时间，是消息长度和链路速率的函数"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"处理延迟: 处理分组首部、检查位错误及确定分组目标所需的时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"排队延迟: 到来的分组排队等待处理的时间"}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CDN把距离缩短，以加快访问速度"}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"延迟的最后一公里"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"延迟相当大一部分往往花在最后的几公里，因为客户单端连接公网的方式和接入链路都比较差"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"TCP"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c7/c740e71f444886f60f16eef29b8b0815.png","alt":"web-sync","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"时延"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每个http连接都需要经过三次握手，从纽约向伦敦请求，启动一次TCP连接，光三次握手至少要花56ms，向伦敦发送分组需要28ms，响应要28ms。可以看出三次握手带来的延迟是非常大的"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"拥塞控制"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"拥塞崩溃"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可能是往返时间超过了所有主机的最大中断间隔，于是相应的主机会产生越来越多的副本，使整个网络陷入瘫痪，最终个交换节点的缓冲区被填满，多出来的分组必须删除"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"流量控制"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过缩放接收窗口（rwnd）的大小来控制流量的发送（可配）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/69/6984d9033ac74d83745072590e6f60bc.png","alt":"web-rwnd","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"慢启动"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过一个动态可变的拥塞窗口（cwnd）大小来控制流量的发送，网络可发送的最大数据取rwnd和cwnd的最小值"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7d/7d1082df30d4198922780cfd5e3a9bc3.png","alt":"web-slowstart","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如图可见，一个请求需要经过220ms才可以达到最大速率。因为慢启动限制了可用的吞吐量，对于小文件的传输是很不利的"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"慢启动重启：在连接空闲一段时间后，重置拥塞窗口到一个安全的默认值。毫无疑问，SSR对于长周期空闲而突发请求的TCP连接有很大的影响，建议服务器金庸SSR"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/22/22ff9c84b51b1f91a890c1d33bf7295d.png","alt":"web-cwnd","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"拥塞预防"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"慢启动每次往返都会成倍提高传输的数据量，知道超过接收端的流量控制窗口或者有分组丢失为止。此时拥塞预防算法接入"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/38/3874ce91ac4a2b71285c316b970c4417.png","alt":"web-defint","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"带宽延迟积"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BDP表示数据链路的容量与其端到端延迟的乘积，结果就是任意时刻在途未确认的最大数据量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"发送端和接收端发送超过了未被确认的最大数据量，都会停下来等待对方的ACK，这就造成了数据缺口。为了解决这个问题应该设置窗口足够大，过小的窗口会限制连接的吞吐量。窗口的大小最小应该设置为BDP"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"队首阻塞"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP按序交付与可靠交付，如果有时丢包，那么后续的包必须等到这个丢包的数据重发并接收，才能交付给应用程序，这就导致读数据时会感觉延迟交付"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在应用程序不关系按序交付和可靠交付的情况下TCP并不是最好的选择。例如音频，丢了一个包可以在音频中插入一个小小的间隙，就可以继续处理后面的包，只要间隙足够小，用户就注意不到，而等待丢包可能导致音频输出产生无法预料的。相对而言，后者的用户体验更差."}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"调优"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"原因"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 三次握手增加了整整一次往返时间；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 慢启动将被应用到每个新连接；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 流量及拥塞控制会影响所有连接的吞吐量；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP 的吞吐量由当前拥塞窗口大小控制。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"方案"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把服务器内核升级到最新版本（Linux：3.2+）；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"确保 cwnd 大小为 10；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"禁用空闲后的慢启动；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"确保启动窗口缩放；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"减少传输冗余数据；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"压缩要传输的数据；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把服务器放到离用户近的地方以减少往返时间；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"尽最大可能重用已经建立的 TCP 连接。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"UDP"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"网络地址转换"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c8/c8dcb0a1f7a0af59f3040b675ebee82f.png","alt":"web-nat","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f3/f3fd4102285e9d629bbb62b6a8545b35.png","alt":"web-nataddr","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这三段地址只允许私网拥有，不允许公网拥有这些ip"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"连接状体超时"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"中转UDP的路由，由于UDP没有连接和终止的概念，这导致中转路由不知道什么时候该删除连接状态。为了解决这个问题，路由器会定期清理，路由状态一旦清除，UDP则需要重新建立。解决方法是定期双向发keep-alive分组。按理TCP有明确的连接状态，路由器应当可以完整把握TCP的生命周期，但是路由器没有这么做，也同样给TCP设置了超时清理的动作，这导致一个长时间不活跃的TCP，会无端端的连接断开。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"P2P"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"STUN: Session Traversal Utilities for NAT 是一个协议，可以让内网应用程序获得一个外网ip和端口，STUN服务器架设在公网"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/32/32cc35f4eb835c19ef8ed061d8d58479.png","alt":"web-stun","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"各自内网的应用程序使用STUN后，就能获得一个外网ip，STUN服务器通过keepalive方式保持路由不超时，各应用程序就能直接UDP通讯了"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TURN: Traversal Using Relays around NAT. 当内网不能使用NAT时，可以使用TURN服务器，应用程序通过TCP连接TURN服务器，服务器做消息中转"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"libjingle是谷歌的一个实现了STUN/TURN/ICE的开源库。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"92% 的时间可以直接连接（STUN）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"8% 的时间要使用中继器（TURN）。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ICE: Interactive Connectivity Establishment协议，能直连就直连，不能直连则使用STUN，再不行则使用TURN"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"设计原则"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序必须容忍各种因特网路径条件；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该控制传输速度；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该对所有流量进行拥塞控制；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该使用与 TCP 相近的带宽；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该准备基于丢包的重发计数器；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该不发送大于路径 MTU 的数据报；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该处理数据报丢失、重复和重排；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该足够稳定以支持 2 分钟以上的交付延迟；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序应该支持 IPv4 UDP 校验和，必须支持 IPv6 校验和；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应用程序可以在需要时使用 keep-alive（最小间隔 15 秒）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"建议使用WebRTC"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"带宽/延迟与页面加载时间的关系"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/37/37325461ebfc6f0605fac1ba15cfab2d.png","alt":"web-pageload","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"时延的原因"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP握手/流量拥塞控制/丢包/队首拥塞"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"网站各资源的用户体验度量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过Navigation Timing/User Timing/resource timing来度量"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"浏览器的优化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"资源预取和排定优先次序"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DNS预解析"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"TCP预连接"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"页面预渲染"}]}]}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"服务器如何利用这些优化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CSS 和 JavaScript 等重要资源应该尽早在文档中出现；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"应该尽早交付 CSS，从而解除渲染阻塞并让 JavaScript 执行；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"非关键性 JavaScript 应该推迟，以避免阻塞 DOM 和 CSSOM 构建；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HTML 文档由解析器递增解析，从而保证文档可以间隙性发送，以求得最佳性能"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/dd/dd0c1330272079ddc30733b7813c861b.png","alt":"web-preload","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"http优化"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"减少DNS查询"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"减少HTTP请求"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用CDN"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"添加Expires首部并配置ETag标签"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Gzip资源"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"避免HTTP重定向"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用持久化连接"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"keep alive和连接池的局限"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于每个服务器的ip，客户端维持一个长连接的连接池，如果有多个请求发往服务器，并超过连接池的数量，则会迫使客户端必须等待连接池的空闲，而且启用多个socket严重占用系统资源。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"http协议的局限"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每次发送http请求，都必须加上头部，而头部是没有经过压缩的，这直接导致头部的长度有可能超过body的长度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}