作爲Netty的核心組件之一，NioEventLoop負責處理netty中的各種事件，每個NioEventLoop上有一個selector負責輪詢IO事件，有一個定時任務隊列及普通任務隊列負責處理註冊到NioEventLoop上的非IO任務。而1組NioEventLoop右由一個NioEventLoopGroup負責管理，默認一個NioEventLoopGroup管理2倍CPU核數的NioEventLoop。
參照官網的服務端寫法：
public void run() { EventLoopGroup bossGroup = new NioEventLoopGroup(); EventLoopGroup workGroup = new NioEventLoopGroup(); ServerBootstrap b = new ServerBootstrap(); b.group(bossGroup, workGroup) .channel(NioServerSocketChannel.class) .option(ChannelOption.SO_BACKLOG, 128) .childOption(ChannelOption.SO_KEEPALIVE, true) .handler(new ChannelHandler()) .childHandler(new ChannelInitializer<SocketChannel>() { @Override protected void initChannel(SocketChannel socketChannel) throws Exception { socketChannel.pipeline().addLast(new ChannelInboundA(), new ChannelInboundB(), new ChannelInboundC()); } }); try { ChannelFuture future = b.bind(port).sync(); log.info("server start running"); future.channel().closeFuture().sync(); } catch (InterruptedException e) { e.printStackTrace(); } finally { bossGroup.shutdownGracefully(); workGroup.shutdownGracefully(); } }
- NioEventLoop實在初始化NioEventLoopGroup的過程中被初始化；
- 當NioEventLoopGroup的構造參數不傳值時，netty默認會爲每個NioEventLoopGroup配置2倍的CPU核數個NioEventLoop：
static { DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt( "io.netty.eventLoopThreads", Runtime.getRuntime().availableProcessors() * 2)); if (logger.isDebugEnabled()) { logger.debug("-Dio.netty.eventLoopThreads: {}", DEFAULT_EVENT_LOOP_THREADS); } } - 最終調用到NioEventLoopGroup的父類MultithreadEventExecutorGroup的構造方法：
protected MultithreadEventExecutorGroup(int nThreads, Executor executor, EventExecutorChooserFactory chooserFactory, Object... args) { if (nThreads <= 0) { throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads)); } if (executor == null) { executor = new ThreadPerTaskExecutor(newDefaultThreadFactory()); } children = new EventExecutor[nThreads]; for (int i = 0; i < nThreads; i ++) { boolean success = false; try { children[i] = newChild(executor, args); success = true; } catch (Exception e) { // TODO: Think about if this is a good exception type throw new IllegalStateException("failed to create a child event loop", e); } finally { if (!success) { for (int j = 0; j < i; j ++) { children[j].shutdownGracefully(); } for (int j = 0; j < i; j ++) { EventExecutor e = children[j]; try { while (!e.isTerminated()) { e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS); } } catch (InterruptedException interrupted) { // Let the caller handle the interruption. Thread.currentThread().interrupt(); break; } } } } } chooser = chooserFactory.newChooser(children); final FutureListener<Object> terminationListener = new FutureListener<Object>() { @Override public void operationComplete(Future<Object> future) throws Exception { if (terminatedChildren.incrementAndGet() == children.length) { terminationFuture.setSuccess(null); } } }; for (EventExecutor e: children) { e.terminationFuture().addListener(terminationListener); } Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length); Collections.addAll(childrenSet, children); readonlyChildren = Collections.unmodifiableSet(childrenSet); }
- 在該構造方法中主要做了3件事：
1).創建一個線程執行器ThreadPerTaskExecutor，以便於後續NioEventLoop在處理task任務時，可直接調用io.netty.util.concurrent.ThreadPerTaskExecutor#execute該方法啓動線程，異步執行任務。
2).爲NioEventLoopGroup創建EventExecutor數組，及NioEventLoop數組。
3).爲NioEventLoopGroup創建一個NioEventLoop旋轉器，負責有事件事從NioEventLoop數組中選擇NioEventLoop來執行任務。 - 創建ThreadPerTaskExecutor：
1).executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
2).調用newDefaultThreadFactory()設置NioEventLoop線程標識前綴：public DefaultThreadFactory(String poolName, boolean daemon, int priority, ThreadGroup threadGroup) { if (poolName == null) { throw new NullPointerException("poolName"); } if (priority < Thread.MIN_PRIORITY || priority > Thread.MAX_PRIORITY) { throw new IllegalArgumentException( "priority: " + priority + " (expected: Thread.MIN_PRIORITY <= priority <= Thread.MAX_PRIORITY)"); } prefix = poolName + '-' + poolId.incrementAndGet() + '-'; this.daemon = daemon; this.priority = priority; this.threadGroup = threadGroup; }
其中poolName爲nioEventLoopGroup，可以看到線程的前綴爲nioEventLoopGroup-自增id-； - NioEventLoopGroup輪詢創建NioEventLoop線程組：
1).調用NioEventLoopGroup的newChild方法：protected EventLoop newChild(Executor executor, Object... args) throws Exception { return new NioEventLoop(this, executor, (SelectorProvider) args[0], ((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]); }
2).進入NioEventLoop的構造：NioEventLoop(NioEventLoopGroup parent, Executor executor, SelectorProvider selectorProvider, SelectStrategy strategy, RejectedExecutionHandler rejectedExecutionHandler) { super(parent, executor, false, DEFAULT_MAX_PENDING_TASKS, rejectedExecutionHandler); if (selectorProvider == null) { throw new NullPointerException("selectorProvider"); } if (strategy == null) { throw new NullPointerException("selectStrategy"); } provider = selectorProvider; selector = openSelector(); selectStrategy = strategy; }
a.首先調用父類構造保存線程執行器，創建taskQueue有界阻塞隊列：protected SingleThreadEventExecutor(EventExecutorGroup parent, Executor executor, boolean addTaskWakesUp, int maxPendingTasks, RejectedExecutionHandler rejectedHandler) { super(parent); this.addTaskWakesUp = addTaskWakesUp; this.maxPendingTasks = Math.max(16, maxPendingTasks); this.executor = ObjectUtil.checkNotNull(executor, "executor"); taskQueue = newTaskQueue(this.maxPendingTasks); rejectedExecutionHandler = ObjectUtil.checkNotNull(rejectedHandler, "rejectedHandler"); }
b.父類構造執行結束後，再保存selectorProvider，和selectStrategy。
c.爲新創建的NioEventLoop創建一個selector：private Selector openSelector() { final Selector selector; try { selector = provider.openSelector(); } catch (IOException e) { throw new ChannelException("failed to open a new selector", e); } if (DISABLE_KEYSET_OPTIMIZATION) { return selector; } final SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet(); Object maybeSelectorImplClass = AccessController.doPrivileged(new PrivilegedAction<Object>() { @Override public Object run() { try { return Class.forName( "sun.nio.ch.SelectorImpl", false, PlatformDependent.getSystemClassLoader()); } catch (ClassNotFoundException e) { return e; } catch (SecurityException e) { return e; } } }); if (!(maybeSelectorImplClass instanceof Class) || // ensure the current selector implementation is what we can instrument. !((Class<?>) maybeSelectorImplClass).isAssignableFrom(selector.getClass())) { if (maybeSelectorImplClass instanceof Exception) { Exception e = (Exception) maybeSelectorImplClass; logger.trace("failed to instrument a special java.util.Set into: {}", selector, e); } return selector; } final Class<?> selectorImplClass = (Class<?>) maybeSelectorImplClass; Object maybeException = AccessController.doPrivileged(new PrivilegedAction<Object>() { @Override public Object run() { try { Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys"); Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys"); selectedKeysField.setAccessible(true); publicSelectedKeysField.setAccessible(true); selectedKeysField.set(selector, selectedKeySet); publicSelectedKeysField.set(selector, selectedKeySet); return null; } catch (NoSuchFieldException e) { return e; } catch (IllegalAccessException e) { return e; } catch (RuntimeException e) { // JDK 9 can throw an inaccessible object exception here; since Netty compiles // against JDK 7 and this exception was only added in JDK 9, we have to weakly // check the type if ("java.lang.reflect.InaccessibleObjectException".equals(e.getClass().getName())) { return e; } else { throw e; } } } }); if (maybeException instanceof Exception) { selectedKeys = null; Exception e = (Exception) maybeException; logger.trace("failed to instrument a special java.util.Set into: {}", selector, e); } else { selectedKeys = selectedKeySet; logger.trace("instrumented a special java.util.Set into: {}", selector); } return selector; }
再創建NioEventLoop的selector時，Netty除了調用JDK創建Selector外，還會將sun.nio.ch.SelectorImpl中保存SelectionKey的set集合利用反射用數組 SelectedSelectionKeySet替換掉該set集。Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys"); Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys"); selectedKeysField.setAccessible(true); publicSelectedKeysField.setAccessible(true); selectedKeysField.set(selector, selectedKeySet); publicSelectedKeysField.set(selector, selectedKeySet);
SelectedSelectionKeySet添加SelectionKey時是直接在數組的最後有值位後添加，相比於set集合的add時間複雜度爲O(1)，SelectedSelectionKeySet的效率更高。@Override public boolean add(SelectionKey o) { if (o == null) { return false; } if (isA) { int size = keysASize; keysA[size ++] = o; keysASize = size; if (size == keysA.length) { doubleCapacityA(); } } else { int size = keysBSize; keysB[size ++] = o; keysBSize = size; if (size == keysB.length) { doubleCapacityB(); } } return true; }
- 創建NioEventLoop旋轉器：
public EventExecutorChooser newChooser(EventExecutor[] executors) { if (isPowerOfTwo(executors.length)) { return new PowerOfTowEventExecutorChooser(executors); } else { return new GenericEventExecutorChooser(executors); }
1).首選調用isPowerOfTwo(int val)判斷NioEventLoopGroup的NioEventLoop線程組是否是2的冪:private static boolean isPowerOfTwo(int val) { return (val & -val) == val; }
2).如果是2的冪執行PowerOfTowEventExecutorChooser的構造：private static final class PowerOfTowEventExecutorChooser implements EventExecutorChooser { private final AtomicInteger idx = new AtomicInteger(); private final EventExecutor[] executors; PowerOfTowEventExecutorChooser(EventExecutor[] executors) { this.executors = executors; } @Override public EventExecutor next() { return executors[idx.getAndIncrement() & executors.length - 1]; } }
PowerOfTowEventExecutorChooser的構造保存NioEventLoop線程組，該類中還有一個next()方法，即是在有事件分配NioEventLoop時調用，下標是每次自增前&上線程數組的長度減1，利用機器碼快速選擇NioEventLoop；
3).如果不是2的冪：private static final class GenericEventExecutorChooser implements EventExecutorChooser { private final AtomicInteger idx = new AtomicInteger(); private final EventExecutor[] executors; GenericEventExecutorChooser(EventExecutor[] executors) { this.executors = executors; } @Override public EventExecutor next() { return executors[Math.abs(idx.getAndIncrement() % executors.length)]; } }
netty在不是2的冪時，分配NioEventLoop，調用next()獲取NioEventLoop是自增前的值與線程組長度取餘。
顯然與運算的效率要高於求餘運算，可見Netty爲了效率簡直喪心病狂了。。。 - 整個NioEventLoop初始化流程就是這些了。Netty既是採用這種主從多線程模型實現高效的異步處理事件。服務端啓動兩個NioEventLoopGroup，一個boss，一個work，boss負責接受請求分發任務，work負責異步處理任務。boss和work下又有多個 NioEventLoop具體來做事。