零拷贝原理的文章网上满天飞，但你知道如何使用零拷贝吗？

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"零拷贝是中间件相关面试中必考题，本文就和大家一起来总结一下NIO拷贝的原理，并结合Netty代码，从代码实现层面近距离观摩如何使用java实现零拷贝。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#48b378","name":"user"}}],"text":"1、零拷贝实现原理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"**“零拷贝”**其实包括两个层面的含义：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拷贝一份相同的数据从一个地方移动到另外一个地方的过程，叫拷贝。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"零希望在IO读写过程中，CPU控制的数据拷贝到次数为0。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在IO编程领域，当然是拷贝的次数越少越好，逐步优化，将其拷贝次数将为0，最大化的提高性能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那接下来我们循序渐进来看一下如何减少数据复制。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"接下来我们将以RocketMQ消息发送、消息读取场景来阐述IO读写过程中可能需要进行的数据复制与上下文切换。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.1 传统的IO读流程","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一次传统的IO读序列流程如下所示：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/cd/cd56790a335e5db1e978734afcdd252f.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"java应用中，如果要将从文件中读取数据，其基本的流程如下所示：","attrs":{}}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"当broker收到拉取请求时发起一次read系统调用，此时操作系统会进行一次上下文的切换，从用态间切换到内核态。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"通过直接存储访问器(DMA)从磁盘将数据加载到内核缓存区（DMA Copy，这个阶段不需要CPU参与，如果是阻塞型IO，该过程用户线程会处于阻塞状态）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"然后在CPU的控制下，将内核缓存区的数据copy到用户空间的缓存区(由于这个是操作系统级别的行为，通常这里指的内存缓存区，通常使用的是堆外内存)，这里将发生一次CPU复制与一次上下文切换（从内核态切换到用户态）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"将堆外内存中的数据复制到应用程序的堆内存，供应用程序使用，本次复制需要经过CPU控制。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"将数据加载到堆空间，需要传输到网卡，这个过程又要进入到内核空间，然后复制到sockebuffer，然后进入网卡协议引擎，从而进入到网络传输中。该部分会在接下来会详细介绍。","attrs":{}}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#6a737d","name":"user"}}],"text":"温馨提示：RocketMQ底层的工作机制并不是上述模型，是经过优化后的读写模型，本文将循序渐进的介绍优化过程。","attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2 传统的IO写流程","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一次传统的IO写入流程如下图所示：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/43/43450a516cc3e95f0034014c9d30c14c.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"核心关键步骤如下：","attrs":{}}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"在broker收到消息时首先会在堆空间中创建一个堆缓存区，用于存储用户需要写入的数据，然后需要将jvm堆内存中数据复制到操作系统内存(CPU COPY)","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"发起write系统调用，将用户空间中的数据复制到内存缓存区，**此过程发生一次上下文切换（用户态切换到内核态）**并进行一次CPU Copy。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"通过直接存储访问器(DMA)将内核空间的数据写入到磁盘，并返回结果，此过程发生一次DMA Copy 与一次上下文切换(内核态切换到用户态)","attrs":{}}]}]}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3 读写优化技巧","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从上面两张流程图，我们不能看出读写处理流程中存在太多复制，同样的数据需要被复制多次，造成性能损耗，故IO读写通常的优化方向主要为：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"减少复制次数、减少用户态/内核态切换次数。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1.3.1 引入堆外内存","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"jvm堆空间中数据要发送到内核缓存区，通常需要先将jvm堆空间中的数据拷贝到系统内存（","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"一个非官方的理解，用C语言实现的本地方法调用中，首先需要将堆空间中数据拷贝到C语言相关的存储结构","attrs":{}},{"type":"text","text":")，故提高性能的第一个措施：使用堆外内存。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不过堆外内存中的数据，通常还是需要从堆空间中获取，从这个角度来看，貌似提升的性能有限。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1.3.2 引入内存映射(MMap与write)","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过引入内存映射机制，减少用户空间与内核空间之间的数据复制，如下图所示：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a88eb153b1d9ae4727aaacd2edcc464.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"内存映射的核心思想就是将内核缓存区、用户空间缓存区映射到同一个物理地址上，可以减少用户缓存区与内核缓存区之间的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"数据拷贝","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但由于内存映射机制并不会减少上下文切换次数。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1.3.3 大名鼎鼎鼎sendfile","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Linux 2.1内核引入了sendfile函数用于将","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"文件通过socket传送","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#6a737d","name":"user"}},{"type":"strong","attrs":{}}],"text":"注意sendfile的传播方向：使用于将文件中的内容直接传播到Socket，通常使用客户端从服务端文件中读取数据，在服务端内部实现零拷贝。","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在1.3.1中介绍客户端从服务端读取消息的过程中，并没有展开介绍从服务端写入到客户端网络中的过程，接下来看看sendfile的数据拷贝图解：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/bc/bc2e696ce2b96ac1fcc43107d7681433.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"sendfile的主要特点是","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"在内核空间中通过DMA将数据从磁盘文件拷贝到内核缓存区，然后可以直接将内核缓存区中的数据在CPU控制下将数据复制到socket缓存区","attrs":{}},{"type":"text","text":"，最终在DMA的控制下将socketbufer中拷贝到协议引擎，然后经网卡传输到目标端。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"sendfile的优势(特点)：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"一次sendfile调用会只设计两次上下文切换，比read+write减少两次上下文切换。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"一次sendfile会存在3次copy,其中一次CPU拷贝，两次DMA拷贝。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1.3.4 Linux Gather","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Linux2.4内核引入了gather机制，用以消除最后一次CPU拷贝，即不再将内核缓存区中的数据拷贝到socketbuffer，而是将内存缓存区中的内存地址、需要读取数据的长度写入到socketbuffer中，然后DMA直接根据socketbuffer中存储的内存地址，直接从内核缓存区中的数据拷贝到协议引擎（注意，这次拷贝由DMA控制）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"从而实现真正的零拷贝。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#48b378","name":"user"}}],"text":"2、结合Netty谈零拷贝实战","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面讲述了“零拷贝”的实现原理，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"接下来将尝试从Netty源码去探究在代码层面如何使用“零拷贝”","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从网上的资料可以得知，在java nio提供的类库中真正能运用底层操作系统的零拷贝机制只有FileChannel的transferTo，而在Netty中也不出意料的对这种方式进行了封装，其类图如下：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0e/0e0dfde664fbec8dc34537927909dd12.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其主要的核心要点是FileRegion的transferTo方法，我们结合该方法再来介绍DefaultFileRegion各个核心属性的含义。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/68/68945b999ed472587975f8f0618f113d.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述代码并不复杂，我们不难得出如下观点：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"首先介绍DefaultFileRegion的核心属性含义：File f底层抽取数据来源的底层磁盘文件FileChannel file底层文件的文件通道。long position数据从通道中抽取的起始位置long count需要传递的总字节数long transfered已传递的字节数量。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"核心要点是调用java nio FileChannel的transferTo方法，底层调用的是操作系统的sendfile函数，即真正的零拷贝。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"调用一次transferTo方法并不一定能将需要的数据全部传输完成，故该方法返回已传输的字节数，是否需要再次调用该方法的判断方法：已传递的字节数是否等于需要传递的总字节数（transfered == count）","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来我们看一下FileRegion的transferTo在netty中的调用链，从而推断一下Netty中的零拷贝的触发要点。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6f/6f228207ca26adb36ed3521f5422071d.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Netty中代表两个类型的通道：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"EpollSocketChannel基于Epoll机制进行事件的就绪选择机制。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"NioSocketChannel","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"基于select机制的事件就绪选择。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Netty中调用通道Channel的flush或writeAndFlush方法，都会最终触发底层通道的网络写事件，如果待写入的对象是FileRegion，则会触发零拷贝机制，接下来我们对两个简单介绍一下：","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.1 EpollSocketChannel 通道零拷贝","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"写入的入口函数为如下：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/89/89c2e7e395ebe499720eb3ee2a9399d5.jpeg","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"核心思想为：如果待写入的消息是DefaultFileRegion，EpollSocketChannel将直接调用sendfile函数进行数据传递；如果是FileRegion类型，则按照约定调用FileRegion的transferTo进行数据传递，这种方式是否真正进行零拷贝取决于FileRegion的transferTo中是否调用了FileChannel的transferTo方法。","attrs":{}}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#6a737d","name":"user"}}],"text":"温馨提示：本文并没有打算详细分析Epoll机制以及编程实践。","attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.2 NioSocketChannel 通道零拷贝实现","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"实现入口为：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f6/f690f262540f48203e6e437761441875.png","alt":"在这里插入图片描述","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从这里可知，NioSocketChannel就是中规中矩的调用FileRegion的transferTo方法，是否真正实现了零拷贝，取决于底层是否调用了FileChannel的transferTo方法。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.3 零拷贝实践总结","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从Netty的实现中我们基本可以得出结论：是否是零拷贝，判断的依据是是否调用了FileChannel的transferTo方法，更准备的表述是底层是否调用了操作系统的sendfile函数，并且操作系统底层还需要支持gather机制，即linux的内核版本不低于2.4。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章首发于","attrs":{}},{"type":"link","attrs":{"href":"https://www.codingw.net/posts/e587b550.html","title":"","type":null},"content":[{"type":"text","text":"https://www.codingw.net/posts/e587b550.html","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule","attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者简介：丁威，《RocketMQ 技术内幕》一书作者、RocketMQ 开源社区优秀布道师，公众号「中间件兴趣圈」维护者，主打成体系剖析 Java 主流中间件，已发布 Kafka、RocketMQ、Dubbo、Sentinel、Canal、ElasticJob 等中间件 15 个专栏。","attrs":{}}]}]}