netty 服務端啓動

創建ServerBootstrap,該類是Netty服務端的啓動類
綁定Reactor線程池,EventLoopGroup,實際上就是EventLoop的數組.除了處理IO操作外,用戶提交的task和定時任務也是由EventLoop執行.避免了多線程競爭的情況
設置並綁定服務端Channel,對於Nio服務端,使用的是NioServerSocketChannel
鏈路建立是創建並初始化ChannelPipeline.其本質是一個處理網絡事件的職責鏈
初始化ChannelPipline完成之後,添加並設置ChannelHandler,
綁定並啓動監聽端口,在綁定之前系統會先進行一些初始化,完成之後會啓動監聽端口,並將ServerSocketChannel註冊到Selector上監聽客戶端連接
Selector輪詢.由Reactor線程NioEventLoop負責調度和執行Selector輪詢,選擇準備就緒的Channel集合.
當輪詢到準備就緒的Channel之後，由Reactor線程執行ChannelPipeline的相應方法，最終調度執行ChannelHandler
執行Netty系統ChannelHandler和用戶自定義的ChannelHandler。ChannelPipeline根據網絡事件的類型，調度並執行ChannelHandler。

源碼分析

服務端從bind 函數啓動。bind 函數最後訪問到了AbstractBootstrap的 dobind 函數，如下

private ChannelFuture doBind(final SocketAddress localAddress) {
  final ChannelFuture regFuture = initAndRegister();
  //省略其他函數
}

初始化

final ChannelFuture initAndRegister() {
        Channel channel = null;
        try {
            channel = channelFactory.newChannel();
            init(channel);
        //省略其他函數
    }

使用 channelFactory 的 newChannel 構造一個 channel，對於 Nio應用來說，這裏是NioServerSocketChannel，其使用了反射來獲取實例。
調用 init 函數

init 函數是一個抽象函數，被定義在了ServerBootstrap中

void init(Channel channel) throws Exception {
    final Map<ChannelOption<?>, Object> options = options0();
  ////設置 Socket 參數和 NioServerSocketChannel 的附加屬性
    synchronized (options) {
        setChannelOptions(channel, options, logger);
    }

    final Map<AttributeKey<?>, Object> attrs = attrs0();
    synchronized (attrs) {
        for (Entry<AttributeKey<?>, Object> e: attrs.entrySet()) {
            @SuppressWarnings("unchecked")
            AttributeKey<Object> key = (AttributeKey<Object>) e.getKey();
            channel.attr(key).set(e.getValue());
        }
    }
		//////
    ChannelPipeline p = channel.pipeline();

    final EventLoopGroup currentChildGroup = childGroup;
    final ChannelHandler currentChildHandler = childHandler;
    final Entry<ChannelOption<?>, Object>[] currentChildOptions;
    final Entry<AttributeKey<?>, Object>[] currentChildAttrs;
    synchronized (childOptions) {
        currentChildOptions = childOptions.entrySet().toArray(newOptionArray(0));
    }
    synchronized (childAttrs) {
        currentChildAttrs = childAttrs.entrySet().toArray(newAttrArray(0));
    }

    p.addLast(new ChannelInitializer<Channel>() {
        @Override
        public void initChannel(final Channel ch) throws Exception {
            final ChannelPipeline pipeline = ch.pipeline();
            ChannelHandler handler = config.handler();
            if (handler != null) {
                pipeline.addLast(handler);///將 AbstractBootstrap 的 Handler 添加到 ChannelPipeline 中
            }

            ch.eventLoop().execute(new Runnable() {
                @Override
                public void run() {
                  //添加用於註冊的 handler
                    pipeline.addLast(new ServerBootstrapAcceptor(
                            ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
                }
            });
        }
    });
}

前面一大堆其實都是設置連接參數，對於 Server 來說關鍵點在於添加了ServerBootstrapAcceptor這是一個內部類，並且其是server 的唯一一個handler。暫且略過不提。

註冊

init 後回到initAndRegister函數

final ChannelFuture initAndRegister() {
  //省略其他代碼
  ChannelFuture regFuture = config().group().register(channel);
  if (regFuture.cause() != null) {
    if (channel.isRegistered()) {
      channel.close();
    } else {
      channel.unsafe().closeForcibly();
    }
  }
  return regFuture;
}

從當前的執行線程中獲取一個並將 channel註冊到這個線程中。

對於 NioServerChannel其調用到了SingleThreadEventLoop的 register 函數

@Override
public ChannelFuture register(Channel channel) {
    return register(new DefaultChannelPromise(channel, this));
}

@Override
public ChannelFuture register(final ChannelPromise promise) {
    ObjectUtil.checkNotNull(promise, "promise");
    promise.channel().unsafe().register(this, promise);
    return promise;
}

對於 NioServerChannel 的 unsafe 函數，其在AbstractNioMessageChannel中實現

而 register 函數，則在AbstractUnsafe中定義，並被聲明爲 final

public final void register(EventLoop eventLoop, final ChannelPromise promise) {
  if (eventLoop == null) {
    throw new NullPointerException("eventLoop");
  }
  if (isRegistered()) {
    promise.setFailure(new IllegalStateException("registered to an event loop already"));
    return;
  }
  if (!isCompatible(eventLoop)) {
    promise.setFailure(
      new IllegalStateException("incompatible event loop type: " + eventLoop.getClass().getName()));
    return;
  }

  AbstractChannel.this.eventLoop = eventLoop;

  if (eventLoop.inEventLoop()) {
    register0(promise);
  } else {
    try {
      eventLoop.execute(new Runnable() {
        @Override
        public void run() {
          register0(promise);
        }
      });
    } catch (Throwable t) {
      logger.warn(
        "Force-closing a channel whose registration task was not accepted by an event loop: {}",
        AbstractChannel.this, t);
      closeForcibly();
      closeFuture.setClosed();
      safeSetFailure(promise, t);
    }
  }
        }

前面的函數是參數校驗，

關鍵代碼爲

AbstractChannel.this.eventLoop = eventLoop;

這行代碼將 evetloop（即指定的線程）綁定到了 channel 中。

此時調用線程仍然是用戶線程，因此會進入

eventLoop.execute(new Runnable() {
    @Override
    public void run() {
        register0(promise);
    }
});

從此刻開始，netty 線程和用戶線程就開始並行了。

隨後由 eventLoop 來執行register0函數

private void register0(ChannelPromise promise) {
    try {
        // check if the channel is still open as it could be closed in the mean time when the register
        // call was outside of the eventLoop
        if (!promise.setUncancellable() || !ensureOpen(promise)) {
            return;
        }
        boolean firstRegistration = neverRegistered;
        doRegister();
        neverRegistered = false;
        registered = true;

        // Ensure we call handlerAdded(...) before we actually notify the promise. This is needed as the
        // user may already fire events through the pipeline in the ChannelFutureListener.
        pipeline.invokeHandlerAddedIfNeeded();

        safeSetSuccess(promise);
        pipeline.fireChannelRegistered();
        // Only fire a channelActive if the channel has never been registered. This prevents firing
        // multiple channel actives if the channel is deregistered and re-registered.
        if (isActive()) {
            if (firstRegistration) {
                pipeline.fireChannelActive();
            } else if (config().isAutoRead()) {
                // This channel was registered before and autoRead() is set. This means we need to begin read
                // again so that we process inbound data.
                //
                // See https://github.com/netty/netty/issues/4805
                beginRead();
            }
        }
    } catch (Throwable t) {
        // Close the channel directly to avoid FD leak.
        closeForcibly();
        closeFuture.setClosed();
        safeSetFailure(promise, t);
    }
}

AbstractNioChannel.doRegister

執行 doRegister 函數，其被定義在AbstractNioChannel中

protected void doRegister() throws Exception {
  boolean selected = false;
  for (;;) {
    try {
      selectionKey = javaChannel().register(eventLoop().unwrappedSelector(), 0, this);
      return;
    } catch (CancelledKeyException e) {
      if (!selected) {
        // Force the Selector to select now as the "canceled" SelectionKey may still be
        // cached and not removed because no Select.select(..) operation was called yet.
        eventLoop().selectNow();
        selected = true;
      } else {
        // We forced a select operation on the selector before but the SelectionKey is still cached
        // for whatever reason. JDK bug ?
        throw e;
      }
    }
  }
}

其 javaChannel 由NioServerSocketChannel的構造器設置。

這裏註冊的是 0，表示不監聽任何網絡操作。

這裏註冊爲 0 的原因有兩點

該函數處於AbstractNioChannel，而 clientChannel 和 serverChannel 都是其子類，因此其不能設置特殊的操作
通過 SelectionKey 的 interestOps 可以修改其監聽的網絡操作

other

pipeline 實際上是一個AbstractChannelHandlerContext的雙向鏈表，並且其默認的 head 和 tail 分別爲HeadContext和TailContext。

headContext的 register 的實現爲invokeHandlerAddedIfNeeded，即觸發第一次註冊事件，然而此時其實是不觸發的，因爲剛剛在之前已經觸發過了，隨後將在鏈表中傳遞註冊事件

tailContext 是一個空操作。

初始化 server

除此之外，在初始化 server 時，還添加了一個 ChannelInitializer（在 init 函數中）

查看其實現

@Override
@SuppressWarnings("unchecked")
public final void channelRegistered(ChannelHandlerContext ctx) throws Exception {
    // Normally this method will never be called as handlerAdded(...) should call initChannel(...) and remove
    // the handler.
    if (initChannel(ctx)) {
        // we called initChannel(...) so we need to call now pipeline.fireChannelRegistered() to ensure we not
        // miss an event.
        ctx.pipeline().fireChannelRegistered();

        // We are done with init the Channel, removing all the state for the Channel now.
        removeState(ctx);
    } else {
        // Called initChannel(...) before which is the expected behavior, so just forward the event.
        ctx.fireChannelRegistered();
    }
}
private boolean initChannel(ChannelHandlerContext ctx) throws Exception {
  if (initMap.add(ctx)) { // Guard against re-entrance.
    try {
      initChannel((C) ctx.channel());
    } catch (Throwable cause) {
      // Explicitly call exceptionCaught(...) as we removed the handler before calling initChannel(...).
      // We do so to prevent multiple calls to initChannel(...).
      exceptionCaught(ctx, cause);
    } finally {
      ChannelPipeline pipeline = ctx.pipeline();
      if (pipeline.context(this) != null) {
        pipeline.remove(this);
      }
    }
    return true;
  }
  return false;
}

可以看到其在觸發 register 事件觸發時的默認操作爲執行initChannel操作，隨後結束移除 init 的上下文這個操作是自定義的，對於 server 來說這個操作爲

final ChannelPipeline pipeline = ch.pipeline();
ChannelHandler handler = config.handler();
if (handler != null) {
    pipeline.addLast(handler);
}

ch.eventLoop().execute(new Runnable() {
    @Override
    public void run() {
        pipeline.addLast(new ServerBootstrapAcceptor(
                ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
    }
});

在這裏又提交了一個 task 到 channel 綁定的 eventLoop 中，實際上當前正在進行的 register 也是一個 task，因此這裏提交了下一個運行的任務。

需要注意的是這個提交的任務並不一定在下次執行，因爲當前是 netty 線程執行，而用戶線程可能是並行的。

隨後調用 isActive，對於 Server 來說，其實現於NioServerSocketChannel

public boolean isActive() {
  // As java.nio.ServerSocketChannel.isBound() will continue to return true even after the channel was closed
  // we will also need to check if it is open.
  return isOpen() && javaChannel().socket().isBound();
}

隨後觸發 pipeline 的 active 操作，對於 head 來說在觸發了鏈表的所有 active 操作後，若配置爲 autoRead，則進行 channel的 read 操作

private void readIfIsAutoRead() {
    if (channel.config().isAutoRead()) {
        channel.read();
    }
}

隨後若當前 channel 註冊過，則觸發beginRead 函數

此時將調用到AbstractNioChannel 的doBeginRead 操作，其將設置對 channel 添加一個 read 的interestOps。

@Override
protected void doBeginRead() throws Exception {
    // Channel.read() or ChannelHandlerContext.read() was called
    final SelectionKey selectionKey = this.selectionKey;
    if (!selectionKey.isValid()) {
        return;
    }

    readPending = true;

    final int interestOps = selectionKey.interestOps();
    if ((interestOps & readInterestOp) == 0) {
        selectionKey.interestOps(interestOps | readInterestOp);
    }
}

然而對於NioServerSocketChannel來說，readInterestOps 爲OP_ACCEPT

public NioServerSocketChannel(ServerSocketChannel channel) {
    super(null, channel, SelectionKey.OP_ACCEPT);
    config = new NioServerSocketChannelConfig(this, javaChannel().socket());
}

綁定

當觸發並行時用戶線程回到 bind 函數中

private ChannelFuture doBind(final SocketAddress localAddress) {
  final ChannelFuture regFuture = initAndRegister();
  final Channel channel = regFuture.channel();
  if (regFuture.cause() != null) {
    return regFuture;
  }

  if (regFuture.isDone()) {
    // At this point we know that the registration was complete and successful.
    ChannelPromise promise = channel.newPromise();
    doBind0(regFuture, channel, localAddress, promise);
    return promise;
  } else {
    // Registration future is almost always fulfilled already, but just in case it's not.
    final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
    regFuture.addListener(new ChannelFutureListener() {
      @Override
      public void operationComplete(ChannelFuture future) throws Exception {
        Throwable cause = future.cause();
        if (cause != null) {
          // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
          // IllegalStateException once we try to access the EventLoop of the Channel.
          promise.setFailure(cause);
        } else {
          // Registration was successful, so set the correct executor to use.
          // See https://github.com/netty/netty/issues/2586
          promise.registered();

          doBind0(regFuture, channel, localAddress, promise);
        }
      }
    });
    return promise;
  }
}

確定 initAndRegister 沒有發生異常
判斷 register 是否完成

註冊完成時

若當前 register 已經完成了，則直接進行 bind0

private static void doBind0(
  final ChannelFuture regFuture, final Channel channel,
  final SocketAddress localAddress, final ChannelPromise promise) {

  // This method is invoked before channelRegistered() is triggered.  Give user handlers a chance to set up
  // the pipeline in its channelRegistered() implementation.
  channel.eventLoop().execute(new Runnable() {
    @Override
    public void run() {
      if (regFuture.isSuccess()) {
        channel.bind(localAddress, promise).addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
      } else {
        promise.setFailure(regFuture.cause());
      }
    }
  });
}

這個函數的目的是爲了執行 bind，並且對 bind 事件添加失敗關閉事件，要注意的是這裏是給 channel 綁定的線程提交了一個任務，而非直接執行，bind 函數具體的操作暫且不提。

ChannelFutureListener CLOSE_ON_FAILURE = new ChannelFutureListener() {
  @Override
  public void operationComplete(ChannelFuture future) {
    if (!future.isSuccess()) {
      future.channel().close();
    }
  }
};

註冊未完成時

給註冊事件添加一個完成時事件

final PendingRegistrationPromise promise = new PendingRegistrationPromise(channel);
regFuture.addListener(new ChannelFutureListener() {
  @Override
  public void operationComplete(ChannelFuture future) throws Exception {
    Throwable cause = future.cause();
    if (cause != null) {
      // Registration on the EventLoop failed so fail the ChannelPromise directly to not cause an
      // IllegalStateException once we try to access the EventLoop of the Channel.
      promise.setFailure(cause);
    } else {
      // Registration was successful, so set the correct executor to use.
      // See https://github.com/netty/netty/issues/2586
      promise.registered();

      doBind0(regFuture, channel, localAddress, promise);
    }
  }
});

最後仍然是執行 doBind0.

doBind0

bind 函數調用到AbstractChannel

@Override
public ChannelFuture bind(SocketAddress localAddress) {
    return pipeline.bind(localAddress);
}

DefaultChannelPipeline 的 bind

@Override
public final ChannelFuture bind(SocketAddress localAddress) {
    return tail.bind(localAddress);
}

TailContext 的 bind由AbstractChannelHandlerContext 實現

public ChannelFuture bind(final SocketAddress localAddress, final ChannelPromise promise) {
    if (localAddress == null) {
        throw new NullPointerException("localAddress");
    }
    if (isNotValidPromise(promise, false)) {
        // cancelled
        return promise;
    }

    final AbstractChannelHandlerContext next = findContextOutbound(MASK_BIND);
    EventExecutor executor = next.executor();
    if (executor.inEventLoop()) {
        next.invokeBind(localAddress, promise);
    } else {
        safeExecute(executor, new Runnable() {
            @Override
            public void run() {
                next.invokeBind(localAddress, promise);
            }
        }, promise, null);
    }
    return promise;
}

當前情況這裏運行的線程爲 eventLoop

private void invokeBind(SocketAddress localAddress, ChannelPromise promise) {
    if (invokeHandler()) {//期望值爲已完成 handler add 事件
        try {
            ((ChannelOutboundHandler) handler()).bind(this, localAddress, promise);
        } catch (Throwable t) {
            notifyOutboundHandlerException(t, promise);
        }
    } else {//遞歸回bind函數
        bind(localAddress, promise);
    }
}

實際上由於 eventloop 是單線程的，因此執行到這裏時，必定已經註冊完成了，也就是

調用回 HeadContext

@Override
public void bind(
        ChannelHandlerContext ctx, SocketAddress localAddress, ChannelPromise promise) {
    unsafe.bind(localAddress, promise);
}

由 AbstractUnsafe 實現

@Override
public final void bind(final SocketAddress localAddress, final ChannelPromise promise) {
    assertEventLoop();

    if (!promise.setUncancellable() || !ensureOpen(promise)) {
        return;
    }

    // See: https://github.com/netty/netty/issues/576
    if (Boolean.TRUE.equals(config().getOption(ChannelOption.SO_BROADCAST)) &&
        localAddress instanceof InetSocketAddress &&
        !((InetSocketAddress) localAddress).getAddress().isAnyLocalAddress() &&
        !PlatformDependent.isWindows() && !PlatformDependent.maybeSuperUser()) {
        // Warn a user about the fact that a non-root user can't receive a
        // broadcast packet on *nix if the socket is bound on non-wildcard address.
        logger.warn(
                "A non-root user can't receive a broadcast packet if the socket " +
                "is not bound to a wildcard address; binding to a non-wildcard " +
                "address (" + localAddress + ") anyway as requested.");
    }

    boolean wasActive = isActive();
    try {
        doBind(localAddress);
    } catch (Throwable t) {
        safeSetFailure(promise, t);
        closeIfClosed();
        return;
    }

    if (!wasActive && isActive()) {
        invokeLater(new Runnable() {
            @Override
            public void run() {
                pipeline.fireChannelActive();
            }
        });
    }

    safeSetSuccess(promise);
}

doBind函數由 NioServerChannel 實現，這裏是由 eventLoop 執行的

@Override
protected void doBind(SocketAddress localAddress) throws Exception {
    if (PlatformDependent.javaVersion() >= 7) {
        javaChannel().bind(localAddress, config.getBacklog());
    } else {
        javaChannel().socket().bind(localAddress, config.getBacklog());
    }
}

隨後若開始doBind前未 active，並且 bind 後active，則提交一個activefire任務到 eventloop 中。

至此服務端啓動完成

netty服務端啓動過程分析

netty 服務端啓動

源碼分析

初始化

註冊

AbstractNioChannel.doRegister

other

初始化 server

綁定

註冊完成時

註冊未完成時

doBind0

RestTemplate和web的一些坑

netty服務端啓動過程分析

mybatis執行原理

使用JDK動態代理實現裝飾器

通用的單進程執行

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結