mina源碼閱讀，nio相關知識介紹。

我們所有的網絡操作都離不開套接字，網絡的傳輸常見的就是tcp/IP協議，http協議等。當然網絡操作屬於是系統空間上的，非用戶空間能直接操作的。所以存在內核態和用戶態的數據傳輸與拷貝，這個是有性能損耗的，相關知識需自行了解，關於Zero copy，可自行了解。

1、先從底層上來說明幾種網絡IO的模型，如下：

阻塞io模型：

在缺省情形下，所有文件操作都是阻塞的，在進程空間中調用recvfrom，其系統調用直到數據報到達且被拷貝到應用進程的緩衝區中或者發生錯誤才返回，期間一直在等待。我們就說進程在從調用recvfrom開始到它返回的整段時間內是被阻塞的。

 

 

當用戶進程調用了recvfrom這個系統調用，kernel就開始了IO的第一個階段，準備數據，對於network IO, 很多時候數據在一開始還沒有到達，比如還沒有收到一個完整的UDP包，這個時候kernel就要等待足夠的數據到來。而用戶進程這邊，整個進程會被阻塞，當kernel一直等到數據準備好了，它就會將數據從kernel中拷貝到用戶內存，然後kernel返回結果，用戶進程才解除block狀態，重新運行起來。

非阻塞IO模型

進程把一個套接口設置爲非阻塞是在通知內核：當所請求的IO操作不能滿足要求時，不把本進程投入睡眠，而是返回一個錯誤。也就是說當數據沒有到達時並不等待，而是以一個錯誤返回。

 

 

 

從圖中可以看出，當用戶進程發出read操作時，如果kernel中的數據還沒有準備好，它並不會block用戶進程，而是立刻返回一個error。從用戶進程角度講，它發起一個read操作後，並不需要等待，而是馬上就得到了一個結果。用戶進程判斷結果是一個error時，它就知道數據還沒有準備好，於是它可以再次發送read操作，一旦kernel中的數據準備好了，並且又再次收到了用戶進程的system call，那麼它馬上就將數據拷貝到了用戶內存，然後返回。所以，用戶進程其實是需要不斷的主動詢問kernel數據好了沒有。

IO複用模型

linux提供select/poll，進程通過將一個或多個fd傳遞給select或poll系統調用，select/poll會不斷輪詢所負責的所有socket，可以偵測許多fd是否就緒，但select和poll是順序掃描fd是否就緒，並且支持的fd數量有限。linux還提供了epoll系統調用，它是基於事件驅動的方式，而不是順序掃描，當某個socket有數據到達了，可以直接通知用戶進程，而不需要順序輪詢掃描，提高了效率。

 

 

當進程調用了select，整個進程會被block，同時，kernel會監視所有select負責的socket，當任何一個socket的數據準備好了，select就會返回，這個圖和阻塞IO的圖其實並沒有多大區別，事實上，還更差一點，因爲這裏需要使用兩個System call，select和recvFrom，而blocking io只調用了一個system call（recvfrom），但是select的好處在與它可以同時處理多個connection，（如果處理的連接數不是很高的話，使用select/epoll的web server不一定比使用multi-threading + blocking IO的web server性能更好，可能延遲還更大。select/epoll的優勢並不是對於單個連接能處理得更快，而是在於能處理更多的連接。）

信號驅動異步IO模型

首先開啓套接口信號驅動I/O功能, 並通過系統調用sigaction安裝一個信號處理函數（此係統調用立即返回，進程繼續工作，它是非阻塞的）。當數據報準備好被讀時，就爲該進程生成一個SIGIO信號。隨即可以在信號處理程序中調用recvfrom來讀數據報，井通知主循環數據已準備好被處理中。也可以通知主循環，讓它來讀數據報。

 

異步I/O模型

告知內核啓動某個操作，並讓內核在整個操作完成後（包括將數據從內核拷貝到用戶自己的緩衝區）通知用戶進程，這種模型和信號驅動模型的主要區別是：信號驅動IO：由內核通知我們何時可以啓動一個IO操作，異步IO模型：由內核通知我們IO操作何時完成

 

 

用戶進程發起read操作之後，立刻就可以開始去做其他的事了，從kernel的角度，當它受到一個asynchronous read之後，首先它會立刻返回，不會對用戶進程產生任何block，然後，kernel會等待數據準備完成，然後再將數據拷貝到用戶進程內存，當着一切都完成之後，kernel會給用戶進程發送一個signal，告訴它read操作已經完成。

小結 前面幾種都是同步IO，在內核數據copy到用戶空間都是阻塞的。最後一種是異步IO，通過API把IO操作交給操作系統處理，當前進程不關心具體IO的實現，通過回調函數或者信號量通知當前進程直接對IO返回結果進行處理。一個IO操作其實分成了兩個步驟，發起IO請求和實際的IO操作，同步IO和異步IO的區別就在於第二步是否阻塞，如果實際的IO讀寫阻塞請求進程，那就是同步IO,因此前四種都是同步IO,如果不阻塞，而是操作系統幫你做完IO操作再將結果返回給你，那就是異步IO。阻塞IO和非阻塞IO的區別在於第一步，發起IO請求是否會被阻塞，如果阻塞直到完成那麼就是傳統的阻塞IO,如果不阻塞，那就是非阻塞IO.

以上來自網絡，以及JAVA中的NIO說明，放在此處做一個記錄，點此進入。

2、接下來說明mina這種框架是怎麼構造的。

(1)、首先看下mina框架的整體流程，Mina 的執行流程如下所示：

主要採用責任鏈設計模式。

Mina的底層依賴的主要是Java NIO庫，上層提供的是基於事件的異步接口。其整體的結構如下：

 

(2)、首先看下mina中的一個example例子：

此處主要初始化了一個NioSocketAcceptor類，這個類的初始化方法new NioSocketAcceptor()的時序圖如下：

 

 

這個圖中主要是涉及到AbstractPollingIoAcceptor和AbstractPollingIoProcessor，主要初始化兩大線程池和session的默認配置。第一個線程池是acceptor的線程池，用於接收accept事件，第二個是processor的線程池，用於處理讀寫事件。如下圖：

/**
 * Constructor for {@link AbstractPollingIoAcceptor}. You need to provide a default
 * session configuration, a class of {@link IoProcessor} which will be instantiated in a
 * {@link SimpleIoProcessorPool} for using multiple thread for better scaling in multiprocessor
 * systems.
 * 
 * @see SimpleIoProcessorPool
 * 
 * @param sessionConfig
 *            the default configuration for the managed {@link IoSession}
 * @param processorClass a {@link Class} of {@link IoProcessor} for the associated {@link IoSession}
 *            type.
 * @param processorCount the amount of processor to instantiate for the pool
 */
protected AbstractPollingIoAcceptor(IoSessionConfig sessionConfig, Class<? extends IoProcessor<S>> processorClass,
        int processorCount) {
    this(sessionConfig, null, new SimpleIoProcessorPool<S>(processorClass, processorCount)（此處是processor線程池）, true, null);
}

下面是acceptor的線程池：

protected AbstractIoService(IoSessionConfig sessionConfig, Executor executor) {
    if (sessionConfig == null) {
        throw new IllegalArgumentException("sessionConfig");
    }

    if (getTransportMetadata() == null) {
        throw new IllegalArgumentException("TransportMetadata");
    }

    if (!getTransportMetadata().getSessionConfigType().isAssignableFrom(sessionConfig.getClass())) {
        throw new IllegalArgumentException("sessionConfig type: " + sessionConfig.getClass() + " (expected: "
                + getTransportMetadata().getSessionConfigType() + ")");
    }

    // Create the listeners, and add a first listener : a activation listener
    // for this service, which will give information on the service state.
    listeners = new IoServiceListenerSupport(this);
    listeners.add(serviceActivationListener);

    // Stores the given session configuration
    this.sessionConfig = sessionConfig;

    // Make JVM load the exception monitor before some transports
    // change the thread context class loader.
    ExceptionMonitor.getInstance();

    if (executor == null) {
        //此處創建了一個Acceptor的線程池
        this.executor = Executors.newCachedThreadPool();
        createdExecutor = true;
    } else {
        this.executor = executor;
        createdExecutor = false;
    }

    threadName = getClass().getSimpleName() + '-' + id.incrementAndGet();
}

在startupAcceptor()會創建一個acceptor任務，放入線程池執行：

private void startupAcceptor() throws InterruptedException {
    // If the acceptor is not ready, clear the queues
    // TODO : they should already be clean : do we have to do that ?
    if (!selectable) {
        registerQueue.clear();
        cancelQueue.clear();
    }

    // start the acceptor if not already started
    Acceptor acceptor = acceptorRef.get();

    if (acceptor == null) {
        lock.acquire();
        acceptor = new Acceptor();

        if (acceptorRef.compareAndSet(null, acceptor)) {
            executeWorker(acceptor);
        } else {
            lock.release();
        }
    }
}

這個startupAcceptor()方法是在綁定網絡地址acceptor.bind( new InetSocketAddress(PORT) )的時候創建的。也就會去監聽相應的端口。

首先看下Acceptor的run方法，如下：

public void run() {
    assert acceptorRef.get() == this;

    int nHandles = 0;

    // Release the lock
    lock.release();

    while (selectable) {
        try {
            System.out.println("test:"+System.currentTimeMillis());
            // Process the bound sockets to this acceptor.
            // this actually sets the selector to OP_ACCEPT,
            // and binds to the port on which this class will
            // listen on. We do that before the select because 
            // the registerQueue containing the new handler is
            // already updated at this point.
            //此處是進行註冊操作,在ACCEPT事件的選擇器中註冊通道
            nHandles += registerHandles();

            // Detect if we have some keys ready to be processed
            // The select() will be woke up if some new connection
            // have occurred, or if the selector has been explicitly
            // woke up
            int selected = select();

            // Now, if the number of registered handles is 0, we can
            // quit the loop: we don't have any socket listening
            // for incoming connection.
            if (nHandles == 0) {
                acceptorRef.set(null);

                if (registerQueue.isEmpty() && cancelQueue.isEmpty()) {
                    assert acceptorRef.get() != this;
                    break;
                }

                if (!acceptorRef.compareAndSet(null, this)) {
                    assert acceptorRef.get() != this;
                    break;
                }

                assert acceptorRef.get() == this;
            }

            if (selected > 0) {
                // We have some connection request, let's process
                // them here.
                //此處是處理accept事件
                processHandles(selectedHandles());
            }

            // check to see if any cancellation request has been made.
            nHandles -= unregisterHandles();
        } catch (ClosedSelectorException cse) {
            // If the selector has been closed, we can exit the loop
            ExceptionMonitor.getInstance().exceptionCaught(cse);
            break;
        } catch (Exception e) {
            ExceptionMonitor.getInstance().exceptionCaught(e);

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e1) {
                ExceptionMonitor.getInstance().exceptionCaught(e1);
            }
        }
    }

    // Cleanup all the processors, and shutdown the acceptor.
    if (selectable && isDisposing()) {
        selectable = false;
        try {
            if (createdProcessor) {
                processor.dispose();
            }
        } finally {
            try {
                synchronized (disposalLock) {
                    if (isDisposing()) {
                        destroy();
                    }
                }
            } catch (Exception e) {
                ExceptionMonitor.getInstance().exceptionCaught(e);
            } finally {
                disposalFuture.setDone();
            }
        }
    }
}

接下來查看registerHandles()方法，如下：

private int registerHandles() {
    for (;;) {
        // The register queue contains the list of services to manage
        // in this acceptor.
        AcceptorOperationFuture future = registerQueue.poll();

        if (future == null) {
            return 0;
        }

        // We create a temporary map to store the bound handles,
        // as we may have to remove them all if there is an exception
        // during the sockets opening.
        Map<SocketAddress, H> newHandles = new ConcurrentHashMap<>();
        List<SocketAddress> localAddresses = future.getLocalAddresses();

        try {
            // Process all the addresses
            for (SocketAddress a : localAddresses) {
                //此處就是打開監聽，這是一個抽象方法，在NioSocketAcceptor中實現
                H handle = open(a);
                newHandles.put(localAddress(handle), handle);
            }

            // Everything went ok, we can now update the map storing
            // all the bound sockets.
            boundHandles.putAll(newHandles);

            // and notify.
            future.setDone();
            
            return newHandles.size();
        } catch (Exception e) {
            // We store the exception in the future
            future.setException(e);
        } finally {
            // Roll back if failed to bind all addresses.
            if (future.getException() != null) {
                for (H handle : newHandles.values()) {
                    try {
                        close(handle);
                    } catch (Exception e) {
                        ExceptionMonitor.getInstance().exceptionCaught(e);
                    }
                }

                // Wake up the selector to be sure we will process the newly bound handle
                // and not block forever in the select()
                wakeup();
            }
        }
    }
}

在NioSocketAcceptor類中的open方法如下：

protected ServerSocketChannel open(SocketAddress localAddress) throws Exception {
    // Creates the listening ServerSocket

    ServerSocketChannel channel = null;

    if (selectorProvider != null) {
        channel = selectorProvider.openServerSocketChannel();
    } else {
        channel = ServerSocketChannel.open();
    }

    boolean success = false;

    try {
        // This is a non blocking socket channel
        channel.configureBlocking(false);

        // Configure the server socket,
        ServerSocket socket = channel.socket();

        // Set the reuseAddress flag accordingly with the setting
        socket.setReuseAddress(isReuseAddress());

        // and bind.
        try {
            socket.bind(localAddress, getBacklog());
        } catch (IOException ioe) {
            // Add some info regarding the address we try to bind to the
            // message
            String newMessage = "Error while binding on " + localAddress + "\n" + "original message : "
                    + ioe.getMessage();
            Exception e = new IOException(newMessage);
            e.initCause(ioe.getCause());

            // And close the channel
            channel.close();

            throw e;
        }

        // Register the channel within the selector for ACCEPT event
        channel.register(selector, SelectionKey.OP_ACCEPT);
        success = true;
    } finally {
        if (!success) {
            close(channel);
        }
    }
    return channel;
}

至此Acceptor介紹完畢，接下介紹processor，即上文中介紹的Acceptor的run方法中的 processHandles(selectedHandles())，進入該方法，如下：

private void processHandles(Iterator<H> handles) throws Exception {
    while (handles.hasNext()) {
        H handle = handles.next();
        handles.remove();

        // Associates a new created connection to a processor,
        // and get back a session
        //將processor線程池綁定到session上
        S session = accept(processor, handle);

        if (session == null) {
            continue;
        }

        initSession(session, null, null);
        System.out.println("add a session");
        // add the session to the SocketIoProcessor
        //此處就是從session上獲取processor的線程池，然後將session綁定到該線程池的某個processor上
        session.getProcessor().add(session);
    }
}

進入session.getProcessor().add(session)的add方法，先進入SimpleIoProcessorPool.add方法，如下：

/**
 * {@inheritDoc}
 */
@Override
public final void add(S session) {
    getProcessor(session).add(session);
}

先獲取和該session綁定的線程，如果不存在，則從線程池中取一個線程出來處理這個session，並綁定。getProcessor(session)如下：

private IoProcessor<S> getProcessor(S session) {
    IoProcessor<S> processor = (IoProcessor<S>) session.getAttribute(PROCESSOR);

    if (processor == null) {
        if (disposed || disposing) {
            throw new IllegalStateException("A disposed processor cannot be accessed.");
        }

        processor = pool[Math.abs((int) session.getId()) % pool.length];

        if (processor == null) {
            throw new IllegalStateException("A disposed processor cannot be accessed.");
        }

        session.setAttributeIfAbsent(PROCESSOR, processor);
    }

    return processor;
}

繼續查看getProcessor(session).add(session)的add方法，該方法需要進入AbstractPollingIoProcessor.add方法，如下：

public final void add(S session) {
    if (disposed || disposing) {
        throw new IllegalStateException("Already disposed.");
    }

    // Adds the session to the newSession queue and starts the worker
   //這是一個異步操作，先放入隊列，再啓動processor線程去取出並綁定。
    newSessions.add(session);
    //開啓一個處理任務
    startupProcessor();
}

startupProcessor()方法如下：

private void startupProcessor() {
    Processor processor = processorRef.get();

    if (processor == null) {
        processor = new Processor();

        if (processorRef.compareAndSet(null, processor)) {
            executor.execute(new NamePreservingRunnable(processor, threadName));
        }
    }

    // Just stop the select() and start it again, so that the processor
    // can be activated immediately.
   //喚醒selector
    wakeup();
}

接下來，看下processor的run方法，如下：

public void run() {
    assert processorRef.get() == this;

    lastIdleCheckTime = System.currentTimeMillis();
    int nbTries = 10;

    for (;;) {
        try {
            // This select has a timeout so that we can manage
            // idle session when we get out of the select every
            // second. (note : this is a hack to avoid creating
            // a dedicated thread).
            long t0 = System.currentTimeMillis();
            int selected = select(SELECT_TIMEOUT);
            long t1 = System.currentTimeMillis();
            long delta = t1 - t0;

            if (!wakeupCalled.getAndSet(false) && (selected == 0) && (delta < 100)) {
                // Last chance : the select() may have been
                // interrupted because we have had an closed channel.
                if (isBrokenConnection()) {
                    LOG.warn("Broken connection");
                } else {
                    // Ok, we are hit by the nasty epoll
                    // spinning.
                    // Basically, there is a race condition
                    // which causes a closing file descriptor not to be
                    // considered as available as a selected channel,
                    // but
                    // it stopped the select. The next time we will
                    // call select(), it will exit immediately for the
                    // same
                    // reason, and do so forever, consuming 100%
                    // CPU.
                    // We have to destroy the selector, and
                    // register all the socket on a new one.
                    if (nbTries == 0) {
                        LOG.warn("Create a new selector. Selected is 0, delta = " + delta);
                        registerNewSelector();
                        nbTries = 10;
                    } else {
                        nbTries--;
                    }
                }
            } else {
                nbTries = 10;
            }
            
            // Manage newly created session first
            //此處主要是在讀事件的選擇器上註冊通道，另外是創建這個session的IoFilterChain，並將會話創建事件傳播到責任鏈IoFilterChain上
            if(handleNewSessions() == 0) {
                // Get a chance to exit the infinite loop if there are no
                // more sessions on this Processor
                if (allSessionsCount() == 0) {
                    processorRef.set(null);

                    if (newSessions.isEmpty() && isSelectorEmpty()) {
                        // newSessions.add() precedes startupProcessor
                        assert processorRef.get() != this;
                        break;
                    }

                    assert processorRef.get() != this;

                    if (!processorRef.compareAndSet(null, this)) {
                        // startupProcessor won race, so must exit processor
                        assert processorRef.get() != this;
                        break;
                    }

                    assert processorRef.get() == this;
                }
            }

            updateTrafficMask();

            // Now, if we have had some incoming or outgoing events,
            // deal with them
            if (selected > 0) {
                // LOG.debug("Processing ..."); // This log hurts one of
                // the MDCFilter test...
                //處理讀寫事件
                process();
            }
            
            // Write the pending requests
            long currentTime = System.currentTimeMillis();
            flush(currentTime);
            
            // Last, not least, send Idle events to the idle sessions
            notifyIdleSessions(currentTime);
            
            // And manage removed sessions
            removeSessions();
            
            // Disconnect all sessions immediately if disposal has been
            // requested so that we exit this loop eventually.
            if (isDisposing()) {
                boolean hasKeys = false;

                for (Iterator<S> i = allSessions(); i.hasNext();) {
                    IoSession session = i.next();

                    scheduleRemove((S) session);

                    if (session.isActive()) {
                        hasKeys = true;
                    }
                }

                wakeup();
            }
        } catch (ClosedSelectorException cse) {
            // If the selector has been closed, we can exit the loop
            // But first, dump a stack trace
            ExceptionMonitor.getInstance().exceptionCaught(cse);
            break;
        } catch (Exception e) {
            ExceptionMonitor.getInstance().exceptionCaught(e);

            try {
                Thread.sleep(1000);
            } catch (InterruptedException e1) {
                ExceptionMonitor.getInstance().exceptionCaught(e1);
            }
        }
    }

    try {
        synchronized (disposalLock) {
            if (disposing) {
                doDispose();
            }
        }
    } catch (Exception e) {
        ExceptionMonitor.getInstance().exceptionCaught(e);
    } finally {
        disposalFuture.setValue(true);
    }
}

NioProcessor的init方法，註冊通道，如下：

@Override
protected void init(NioSession session) throws Exception {
    SelectableChannel ch = (SelectableChannel) session.getChannel();
    ch.configureBlocking(false);
    selectorLock.readLock().lock();
    
    try {
        session.setSelectionKey(ch.register(selector, SelectionKey.OP_READ, session));
    } finally {
        selectorLock.readLock().unlock();
    }
}

在AbstractPollingIoProcessor中的process()方法，如下：

private void process() throws Exception {
    for (Iterator<S> i = selectedSessions(); i.hasNext();) {
        S session = i.next();
        //處理對應的session
        process(session);
        i.remove();
    }
}

/**
 * Deal with session ready for the read or write operations, or both.
 */
private void process(S session) {
    // Process Reads
    if (isReadable(session) && !session.isReadSuspended()) {
        read(session);
    }

    // Process writes
    if (isWritable(session) && !session.isWriteSuspended() && session.setScheduledForFlush(true)) {
        // add the session to the queue, if it's not already there
        flushingSessions.add(session);
    }
}

至此，整個執行流程已經通過源碼方式過完。

上述的時序圖是在idea中裝sequence diagram插件，這個可以自行安裝。非常好用，幫助閱讀代碼，理解流程。