網絡與RPC

標籤： Java基礎

Java爲網絡編程提供的java.net包封裝了底層通信細節, 包含了大量的基礎組件以及TCP/UDP協議的編程接口, 使得開發者可以專注於解決問題, 而不用關注通信細節.

基礎組件

java.net包下包含如下網絡基礎組件:

InetAddress
代表一個IP地址,提供了獲取IP地址、主機名、域名,以及測試IP地址是否可達等方法; Inet4Address/Inet6Address是其兩個子類,分別代表IPv4與IPv6.
URLDecoder、URLEncoder
完成普通字符串和application/x-www-form-urlencoded字符串的解碼/編碼.

當字符串包含非西歐字符時,需要將其編碼成application/x-www-form-urlencoded MIME字符. 編碼規則是每個中文字符佔兩個字節, 每個字節轉換成兩個十六進制的數字,因此每個中文字符將轉換成%XX%XX形式.當然,採用不同的字符編碼會使每個中文字符轉換的字節數並不完全相同,所以URLDecoder/URLEncoder使用時需要指定字符集.
URL、URI
URL(Uniform Resource Locator)對象代表統一資源定位器,他是一根指向互聯網”資源”的指針,它包含了一條可打開的能夠到達資源的輸入流, 因此他提供了很多方法訪問來互聯網內容openStream().
URI(Uniform Resource Identifiers)對象代表統一資源標識符,Java中的URI不能用訪問任何資源, 其作用僅限於定位/解析.
URLConnection
URL的openConnection()方法返回一個URLConnection,代表一條與URL所指向的遠程資源的連接, 因此可以通過URLConnection實例向URL發送GET/POST請求,訪問資源.
- 發送GET請求只需將請求參數放在URL字符串之後,以?/&隔開,直接調用URLConnection的getInputStream()方法即可;
- 發送POST請求則需要先設置doIn/doOut 兩個請求頭(對應setDoInput(boolean doinput)/setDoOutput(boolean dooutput)), 再使用對應的OutputStream來發送請求數據.

TCP協議

TCP是一種可靠的傳輸層協議,它在通信兩端各建立一個Socket,從而在C/S之間形成一條虛擬的通信鏈路,然後兩端基於這條虛擬鏈路進行通信(關於TCP協議的詳細介紹可參考的博客TCP/IP入門(3) –傳輸層).Java對TCP提供了良好的封裝,使用Socket對象來代表兩端通信接口,並通過Socket產生的IO流來進行網絡通信.

ServerSocket

Java使用ServerSocket接收來自Client的Socket連接(ServerSocket向開發者隱藏了很多底層通信細節,如bind()/listen()等, 詳見博客 Socket實踐(2) -導引). 如果沒有連接的話他將一直處於等待狀態.因此ServerSocket提供如下常用方法:

方法	描述
`Socket accept()`	Listens for a connection to be made to this socket and accepts it.
`void bind(SocketAddress endpoint, int backlog)`	Binds the ServerSocket to a specific address (IP address and port number).
`static void setSocketFactory(SocketImplFactory fac)`	Sets the server socket implementation factory for the application.
`void close()`	Closes this socket.

ServerSocket還提供了一些設置連接標誌的方法如: SO_RCVBUF/ SO_REUSEADDR/SO_TIMEOUT.

Socket

當客戶端執行到如下代碼時,該Client會連接到指定服務器:

Socket socket = new Socket("127.0.0.1", 8081);

讓服務端ServerSocket的accept()方法繼續執行,於是Server與Client就產生一對互相連接的Socket.此時程序就無須再分服務端/客戶端,而是通過各自的Socket進行通信.
Socket提供瞭如下常用方法:

方法	描述
`void bind(SocketAddress bindpoint)`	Binds the socket to a local address.
`void connect(SocketAddress endpoint, int timeout)`	Connects this socket to the server with a specified timeout value.
`InputStream getInputStream()`	Returns an input stream for this socket.
`OutputStream getOutputStream()`	Returns an output stream for this socket.
`void close()`	Closes this socket.

Socket同樣也提供了設置連接標誌位的方法:SO_KEEPALIVE/OOBINLINE/SO_RCVBUF/SO_REUSEADDR/SO_SNDBUF/SO_LINGER/SO_TIMEOUT/TCP_NODELAY

當使用getInputStream()/getOutputStream()拿到InputStream/OutputStream之後, 我們就可以使用IO相關知識來順暢的通信了(詳見博客 Java I/O).

Server

/**
 * Echo 回聲服務器
 *
 * @author jifang
 * @since 16/8/4 下午4:07.
 */
public class Server {

    private static final ExecutorService executer = Executors.newFixedThreadPool(10);


    public static void main(String[] args) throws IOException {

        ServerSocket server = new ServerSocket(8088);
        while (true) {
            Socket client = server.accept();
            executer.submit(new ClientTask(client));
        }
    }

    private static final class ClientTask implements Runnable {

        private Socket client;

        public ClientTask(Socket client) {
            this.client = client;
        }

        @Override
        public void run() {
            try (InputStream in = client.getInputStream();
                 OutputStream out = client.getOutputStream()) {

                String line;
                while (!(line = read(in)).isEmpty()) {
                    System.out.println(line);
                    out.write(String.format("echo : %s", line).getBytes());
                    out.flush();
                }
                client.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    private static final int BUFFER_SIZE = 1024;

    private static String read(InputStream in) throws IOException {
        byte[] buffer = new byte[BUFFER_SIZE];
        int count = in.read(buffer);
        return new String(buffer, 0, count);
    }
}

Client

public class Client {

    public static void main(String[] args) throws IOException {

        Socket socket = new Socket("127.0.0.1", 8088);

        try (BufferedReader consoleReader = new BufferedReader(new InputStreamReader(System.in));
             InputStream in = socket.getInputStream();
             OutputStream out = socket.getOutputStream()) {

            String line;
            while ((line = consoleReader.readLine()) != null) {
                out.write(line.getBytes());
                out.flush();

                String data = read(in);
                System.out.println(data);
            }
        }
        socket.close();
    }

    private static final int BUFFER_SIZE = 1024;

    private static String read(InputStream in) throws IOException {
        byte[] buffer = new byte[BUFFER_SIZE];
        int count = in.read(buffer);
        return new String(buffer, 0, count);
    }
}

半關閉Socket

Socket提供了兩個半關閉的方法(只關閉輸入流 or 輸出流):

方法	描述
`void shutdownInput()`	Places the input stream for this socket at “end of stream”.
`void shutdownOutput()`	Disables the output stream for this socket.
`boolean isInputShutdown()`	Returns whether the read-half of the socket connection is closed.
`boolean isOutputShutdown()`	Returns whether the write-half of the socket connection is closed.

當調用shutdownXxx()方法關閉對應流之後,Socket處於“半關閉”狀態,用以表示輸入/輸出已經完成,此時該Socket再也無法打開輸入流/輸出流,因此這種做法不適合保持持久通信的交互式應用, 只適用於像HTTP之類的一站式的通信協議.

注意: 即使同一個Socket兩個shutdownXxx()方法都調用了,該Socket也沒有關閉, 只是不能讀寫數據而已.

UDP協議

UDP是一種面向非連接的傳輸層協議,在正式通信前不必與對方建立連接,因此它的通信效率極高,但可靠性不如TCP. 因此UDP適用於一次只傳送少量數據、對可靠性要求不高 的環境.(關於UDP協議的詳細介紹也可參考的博客TCP/IP入門(3) –傳輸層).

DatagramSocket

Java使用DatagramSocket代表UDP的Socket, 可以將DatagramSocket比作碼頭,他不維護狀態,不產生IO流, 唯一作用就是發送/接收貨物(數據)DatagramPacket:

`DatagramSocket`	描述
`DatagramSocket(int port)`	Constructs a datagram socket and binds it to the specified port on the local host machine.
`DatagramSocket(SocketAddress bindaddr)`	Creates a datagram socket, bound to the specified local socket address.
`void receive(DatagramPacket p)`	Receives a datagram packet from this socket.
`void send(DatagramPacket p)`	Sends a datagram packet from this socket.
`void bind(SocketAddress addr)`	Binds this DatagramSocket to a specific address & port.
`void connect(InetAddress address, int port)`	Connects the socket to a remote address for this socket.
`void connect(SocketAddress addr)`	Connects this socket to a remote socket address (IP address + port number).
`void close()`	Closes this datagram socket.

DatagramSocket同樣也提供了一些設置標誌的方法如:SO_SNDBUF/SO_REUSEADDR/SO_RCVBUF/SO_BROADCAST

DatagramPacket

Java中使用DatagramPacket來代表數據報,DatagramSocket接收/發送數據都通過DatagramPacket完成:

`DatagramPacket`	描述
`DatagramSocket()`	Constructs a datagram socket and binds it to any available port on the local host machine.
`DatagramPacket(byte[] buf, int length)`	Constructs a DatagramPacket for receiving packets of length length.
`DatagramPacket(byte[] buf, int length, InetAddress address, int port)`	Constructs a datagram packet for sending packets of length length to the specified port number on the specified host.
`byte[] getData()`	Returns the data buffer.
`void setData(byte[] buf)`	Set the data buffer for this packet.
`void setSocketAddress(SocketAddress address)`	Sets the SocketAddress (usually IP address + port number) of the remote host to which this datagram is being sent.
`SocketAddress getSocketAddress()`	Gets the SocketAddress (usually IP address + port number) of the remote host that this packet is being sent to or is coming from.

DatagramSocket造出實例之後就可通過receive/send方法來接收/發送數據,從方法細節可以看出, 使用DatagramSocket發送數據但並不知道數據的目的地在哪兒,而是由DatagramPacket自身決定.

Server

/**
 * @author jifang
 * @since 16/8/6 下午4:52.
 */
public class UDPServer {

    private static final byte[] RECEIVE_BUFFER = new byte[1024];

    public static void main(String[] args) throws IOException {
        DatagramSocket socket = new DatagramSocket(8088);

        DatagramPacket packet = new DatagramPacket(RECEIVE_BUFFER, RECEIVE_BUFFER.length);
        while (true) {
            socket.receive(packet);
            String content = new String(packet.getData(), 0, packet.getLength());
            System.out.println(String.format("Client IP: %s, Port: %s, Receive: %s", packet.getAddress(), packet.getPort(), content));

            String resp = String.format("Hello %s", content);
            socket.send(new DatagramPacket(resp.getBytes(), resp.length(), packet.getSocketAddress()));
        }
    }
}

Client

public class UDPClient {

    private static final InetSocketAddress ADDRESS = new InetSocketAddress("127.0.0.1", 8088);

    private static final byte[] RECEIVE_BUFFER = new byte[1024];

    public static void main(String[] args) throws IOException {
        DatagramSocket socket = new DatagramSocket();

        DatagramPacket recePacket = new DatagramPacket(RECEIVE_BUFFER, RECEIVE_BUFFER.length);
        BufferedReader reader = new BufferedReader(new InputStreamReader(System.in));
        String line;
        while ((line = reader.readLine()) != null) {
            socket.send(new DatagramPacket(line.getBytes(), line.length(), ADDRESS));

            socket.receive(recePacket);
            System.out.println(new String(recePacket.getData(), 0, recePacket.getLength()));
        }
    }
}

Proxy

從1.5版本開始, Java提供了Proxy和ProxySelector:

Proxy代表一個代理服務器: 在打開URLConnection連接或創建Socket連接時指定Proxy;
ProxySelector代表一個代理選擇器: 他提供對代理服務器更加靈活的配置,如可以對HTTP、 HTTPS、FTP、SOCKS等進行分別配置,還可設置針對不同主機和地址配置不使用代理.

關於代理詳細介紹可參考博客: Nginx - 代理、緩存.

Proxy

Proxy有一個構造器:Proxy(Proxy.Type type, SocketAddress sa)用於創建表示代理服務器的Proxy對象, 其中type參數表示代理服務器類型:

type	描述
`Proxy.Type.DIRECT`	Represents a direct connection, or the absence of a proxy.
`Proxy.Type.HTTP`	Represents proxy for high level protocols such as HTTP or FTP.
`Proxy.Type.SOCKS`	Represents a SOCKS (V4 or V5) proxy.

一旦創建Proxy之後, 就可打開連接時傳入, 以作爲本次連接所使用的代理服務器:

public class ProxyClient {

    private static final InetSocketAddress proxyAddr = new InetSocketAddress("139.129.9.166", 8001);

    private static final String url = "http://www.baidu.com/";

    @Test
    public void client() throws Exception {
        Proxy proxy = new Proxy(Proxy.Type.HTTP, proxyAddr);
        URLConnection connection = new URL(url).openConnection(proxy);
        Reader reader = new InputStreamReader(connection.getInputStream());
        String content = CharStreams.toString(reader);
        System.out.println(content);
    }
}

ProxySelector

直接使用Proxy每次連接都需顯示指定Proxy,而ProxySelector提供了每次打開連接都具有默認代理的方法:

public class ProxyClient {

    protected static final Logger LOGGER = LoggerFactory.getLogger(ProxyClient.class);

    private static final String url = "http://www.jd.com/";

    private static List<Proxy> proxies = new ArrayList<>();

    static {
        proxies.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("127.0.0.1", 8001)));
        proxies.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("127.0.0.1", 8002)));
    }

    @Test
    public void client() throws Exception {
        ProxySelector.setDefault(new ProxySelector() {
            @Override
            public List<Proxy> select(URI uri) {
                String scheme = uri.getScheme();
                if (scheme.equalsIgnoreCase("HTTP")) {
                    return Collections.singletonList(proxies.get(0));
                } else {
                    return Collections.singletonList(proxies.get(1));
                }
            }

            @Override
            public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
                LOGGER.error("connection failed , URI: {}, SocketAddress: {} ", uri, sa, ioe);
            }
        });

        URLConnection connection = new URL(url).openConnection();
        Reader reader = new InputStreamReader(connection.getInputStream());
        String content = CharStreams.toString(reader);
        System.out.println(content);
    }
}

Java還爲ProxySelector提供了一個默認實現類sun.net.spi.DefaultProxySelector並將其註冊爲默認的代理選擇器.其實現策略如下:

select(): 根據系統配置屬性(如http.proxyHost、http.proxyPort、http.nonProxyHosts等)來決定使用哪個代理服務器,詳見默認代理選擇器、DefaultProxySelector.java.

connectFailed(): 連接失敗則會嘗試不使用代理服務器, 直接連接遠程主機.

NIO

從1.4版本開始, Java提供了NIO/非阻塞IO來開發高性能網絡服務器.NIO可以讓服務器使用一個/有限幾個線程同時處理所有客戶端連接.

JDK1.5_update10版本使用epoll替代了傳統的select與poll,極大的提升了NIO性能, 詳細介紹可參考: select的限制與poll的使用、epoll原理與封裝.

Java爲NIO爲提供瞭如下常用類:

Selector
非阻塞IO核心, 是SelectableChanel的多路複用器, 所有希望採用非阻塞方式通信的Channel都需要註冊到Selector, 然後Selector可以同時阻塞監聽多個Channel:

`Selector`	描述
`static Selector open()`	Opens a selector.
`Set<SelectionKey> selectedKeys()`	Returns this selector’s selected-key set.
`int select()`	Selects a set of keys whose corresponding channels are ready for I/O operations.
`int select(long timeout)`	Selects a set of keys whose corresponding channels are ready for I/O operations.
`Set<SelectionKey> keys()`	Returns this selector’s key set.
`int selectNow()`	Selects a set of keys whose corresponding channels are ready for I/O operations.
`Selector wakeup()`	Causes the first selection operation that has not yet returned to return immediately.
`SelectorProvider provider()`	Returns the provider that created this channel.
`void close()`	Closes this selector.

SelectableChannel
代表可以支持非阻塞的Channel對象,默認阻塞,必須開啓非阻塞模式纔可利用非阻塞特性,被註冊到Selector上,並由SelectionKey代表這種註冊關係:

`SelectableChannel`	描述
`SelectionKey register(Selector sel, int ops)`	Registers this channel with the given selector, returning a selection key.
`SelectionKey register(Selector sel, int ops, Object att)`	Registers this channel with the given selector, returning a selection key.
`SelectableChannel configureBlocking(boolean block)`	Adjusts this channel’s blocking mode.
`boolean isBlocking()`	Tells whether or not every I/O operation on this channel will block until it completes.
`int validOps()`	Returns an operation set identifying this channel’s supported operations.
`boolean isRegistered()`	Tells whether or not this channel is currently registered with any selectors.
`SelectionKey keyFor(Selector sel)`	Retrieves the key representing the channel’s registration with the given selector.

validOps()方法返回一個整數值, 表示該Channel支持的IO操作, 在SelectionKey定義了四種IO操作常量:

常量	描述
`OP_ACCEPT`	Operation-set bit for socket-accept operations.
`OP_CONNECT`	Operation-set bit for socket-connect operations.
`OP_READ`	Operation-set bit for read operations.
`OP_WRITE`	Operation-set bit for write operations.

所以可以根據validOps()返回值確定該SelectableChannel支持的IO操作(如ServerSocketChannel只支持OP_ACCEPT、SocketChannel支持OP_CONNECT/OP_READ/OP_WRITE等), 但SelectionKey提供了更加方便的方法來查看發生了什麼IO事件:

SelectionKey
代表SelectableChannel和Selector之間的註冊關係:

方法	描述
`SelectableChannel channel()`	Returns the channel for which this key was created.
`Selector selector()`	Returns the selector for which this key was created.
`boolean isAcceptable()`	Tests whether this key’s channel is ready to accept a new socket connection.
`boolean isConnectable()`	Tests whether this key’s channel has either finished, or failed to finish, its socket-connection operation.
`boolean isReadable()`	Tests whether this key’s channel is ready for reading.
`boolean isWritable()`	Tests whether this key’s channel is ready for writing.
`int interestOps()`	Retrieves this key’s interest set.
`SelectionKey interestOps(int ops)`	Sets this key’s interest set to the given value.
`int readyOps()`	Retrieves this key’s ready-operation set.
`Object attach(Object ob)`	Attaches the given object to this key.
`Object attachment()`	Retrieves the current attachment.
`void cancel()`	Requests that the registration of this key’s channel with its selector be cancelled.
`boolean isValid()`	Tells whether or not this key is valid.

NIO示例

public class Server {

    private static ByteBuffer BUFFER = ByteBuffer.allocate(1024);

    public static void main(String[] args) throws IOException {

        Selector selector = Selector.open();

        ServerSocketChannel server = ServerSocketChannel.open();
        server.bind(new InetSocketAddress(8088));
        server.configureBlocking(false);
        server.register(selector, SelectionKey.OP_ACCEPT);

        while (selector.select() > 0) {
            Set<SelectionKey> keys = selector.selectedKeys();
            Iterator<SelectionKey> iterator = keys.iterator();
            while (iterator.hasNext()) {
                SelectionKey key = iterator.next();

                // ServerSocket可以接受連接
                if (key.isAcceptable()) {
                    ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
                    SocketChannel client = serverChannel.accept();
                    client.configureBlocking(false);
                    client.register(selector, SelectionKey.OP_READ);
                }
                // Socket可讀
                else if (key.isReadable()) {
                    SocketChannel clientChannel = (SocketChannel) key.channel();
                    String content = readFully(clientChannel);

                    if (!content.isEmpty()) {
                        System.out.println(content);
                        ByteBuffer buffer = ((ByteBuffer) BUFFER.clear()).put(String.format("Hello %s", content).getBytes());
                        for (SelectionKey aKey : selector.keys()) {
                            SelectableChannel channel = aKey.channel();
                            if (channel instanceof SocketChannel) {
                                ((SocketChannel) channel).write((ByteBuffer) buffer.flip());
                            }
                        }
                    }
                    // 客戶端斷開, 取消註冊
                    else {
                        key.cancel();
                    }
                }

                iterator.remove();
            }
        }
    }

    private static String readFully(SocketChannel client) throws IOException {
        StringBuilder sb = new StringBuilder();
        BUFFER.clear();
        while (client.read(BUFFER) > 0) {
            sb.append(StandardCharsets.UTF_8.decode((ByteBuffer) BUFFER.flip()));
            BUFFER.clear();
        }

        return sb.toString();
    }
}

注意每次迭代末尾的iterator.remove()調用.Selector自己不會移除已觸發過的SelectionKey實例,必須在處理完SelectableChanel時自己移除, 等下次該Channel就緒時, Selector會再次將其放入已觸發的Key中.

小結

使用SelectorNO來處理多Channel的好處是: 只需要更少的線程來處理多個IO(事實上可以只用一個線程處理所有通道); 因爲對於操作系統而言,多個線程間上下文切換的開銷很大,而且每個線程都要佔用系統的部分資源(如線程私有內存等),因此使用的線程越少越好. 但現代操作系統和CPU在多任務方面的表現越來越好,而且CPU多核已成事實, 所以線程開銷隨着時間的推移變得越來越小,因此不使用多任務可能就是在浪費CPU,而且NIO開發相對繁瑣, 因此在使用or不使用Selector還需要做進一步的權衡.

AIO

Java7的NIO.2提供了異步Channel支持, 以提供更高效的IO. 這種基於異步Channel的IO機制被稱爲AIO.Java提供了一系列以Asynchronous開頭的Channel接口和類:

可以看出,AIO包含2個接口、3個實現類: 其中AsynchronousServerSocketChannel/AsynchronousSocketChannel是支持TCP通信的異步Channel:

AsynchronousServerSocketChannel
負責監聽, 與ServerSocketChannel相似:

方法	描述
`static AsynchronousServerSocketChannel open()`	Opens an asynchronous server-socket channel.
`static AsynchronousServerSocketChannel open(AsynchronousChannelGroup group)`	Opens an asynchronous server-socket channel.
`AsynchronousServerSocketChannel bind(SocketAddress local)`	Binds the channel’s socket to a local address and configures the socket to listen for connections.
`Future<AsynchronousSocketChannel> accept()`	Accepts a connection.
`<A> void accept(A attachment, CompletionHandler<AsynchronousSocketChannel,? super A> handler)`	Accepts a connection.
`abstract <T> AsynchronousServerSocketChannel setOption(SocketOption<T> name, T value)`	Sets the value of a socket option.

AsynchronousChannelGroup是異步Channel分組管理器, 創建時傳入ExecutorService, 使其綁定一個線程池,以實現資源共享. 他主要負責兩個任務: 處理IO事件和觸發CompletionHandler:

public interface CompletionHandler<V,A> {

    /**
     * Invoked when an operation has completed.
     *
     * @param   result
     *          The result of the I/O operation.
     * @param   attachment
     *          The object attached to the I/O operation when it was initiated.
     */
    void completed(V result, A attachment);

    /**
     * Invoked when an operation fails.
     *
     * @param   exc
     *          The exception to indicate why the I/O operation failed
     * @param   attachment
     *          The object attached to the I/O operation when it was initiated.
     */
    void failed(Throwable exc, A attachment);
}

AsynchronousSocketChannel
客戶端的異步Channel, 瞭解了AsynchronousServerSocketChannel的使用之後, AsynchronousSocketChannel就非常簡單了:

方法	描述
`static AsynchronousSocketChannel open()`	Opens an asynchronous socket channel.
`static AsynchronousSocketChannel open(AsynchronousChannelGroup group)`	Opens an asynchronous socket channel.
`Future<Void> connect(SocketAddress remote)`	Connects this channel.
`<A> void connect(SocketAddress remote, A attachment, CompletionHandler<Void,? super A> handler)`	Connects this channel.
`Future<Integer> read(ByteBuffer dst)`	Reads a sequence of bytes from this channel into the given buffer.
`<A> void read(ByteBuffer dst, A attachment, CompletionHandler<Integer,? super A> handler)`	Reads a sequence of bytes from this channel into the given buffer.
`Future<Integer> write(ByteBuffer src)`	Writes a sequence of bytes to this channel from the given buffer.
`<A> void write(ByteBuffer src, A attachment, CompletionHandler<Integer,? super A> handler)`	Writes a sequence of bytes to this channel from the given buffer.

示例

public class Server {

    private static final Logger LOGGER = LoggerFactory.getLogger(Server.class);

    private static AsynchronousChannelGroup initChannelGroup() throws IOException {
        return AsynchronousChannelGroup.withThreadPool(Executors.newFixedThreadPool(10));
    }

    public static void main(String[] args) throws IOException, InterruptedException {
        AsynchronousServerSocketChannel server = AsynchronousServerSocketChannel
                .open(initChannelGroup())
                .bind(new InetSocketAddress(8088));

        server.accept(null, new AcceptHandler(server));

        Thread.sleep(10 * 1000 * 1000);
        server.close();
    }

    private static class AcceptHandler implements CompletionHandler<AsynchronousSocketChannel, Object> {

        private List<AsynchronousSocketChannel> clients;

        private AsynchronousServerSocketChannel server;

        public AcceptHandler(AsynchronousServerSocketChannel server) {
            this.server = server;
            this.clients = new ArrayList<>();
        }

        @Override
        public void completed(final AsynchronousSocketChannel client, Object attachment) {
            // Server繼續接收下一次請求
            server.accept(null, this);
            clients.add(client);

            // 讀取數據
            final ByteBuffer buffer = ByteBuffer.allocate(1024);
            client.read(buffer, null, new CompletionHandler<Integer, Objects>() {

                @Override
                public void completed(Integer result, Objects attachment) {
                    buffer.flip();
                    String content = StandardCharsets.UTF_8.decode(buffer).toString();

                    if (!content.isEmpty()) {
                        System.out.println(String.format("receive: %s", content));

                        for (AsynchronousSocketChannel client : clients) {
                            try {
                                buffer.flip();
                                client.write(buffer).get();
                            } catch (InterruptedException | ExecutionException ignored) {
                            }
                        }

                        // Client繼續接收數據
                        buffer.clear();
                        client.read(buffer, null, this);
                    }
                    // 客戶端斷開連接
                    else {
                        clients.remove(client);
                    }
                }

                @Override
                public void failed(Throwable exc, Objects attachment) {
                    LOGGER.error("read error: ", exc);
                }
            });
        }

        @Override
        public void failed(Throwable exc, Object attachment) {
            LOGGER.error("accept error: ", exc);
        }
    }
}

其他關於AIO的介紹可參考博客 Select-I/O複用、Java nio 2.0 AIO.

RPC

RPC(Remote Procedure Call: 遠程過程調用)是實現SOA的基礎,其主要目的是使構建分佈式計算/應用更容易,在提供強大遠程調用能力的同時又不損失本地調用的語義簡潔性(被調代碼並不在調用者本地執行,但其調用方式幾乎與本地一模一樣).
類似Dubbo/Hessian/Thrift的RPC框架對項目有着非常重要的現實意義:可以將龐大的系統拆分成多個模塊,每個模塊又可根據不同的壓力啓動不同數量的實例,模塊間通過RPC透明通信,從而將集中式系統改造成分佈式以提高其擴展能力,優化硬件資源利用率.

現在我們簡單實現一個RPC框架的原型, 技術選型如下:

通信: TCP
配置: ZooKeeper
序列化: Hession2
模式: 動態代理/線程池

服務

無論是否爲RPC, 都需要首先實現可調用服務Service,只是RPC方式會將Service實現部署在服務端, 並提供Server接口給客戶端:

/**
 * @author jifang
 * @since 16/8/4 上午10:56.
 */
public class HelloServiceImpl implements IHelloService {

    @Override
    public String sayHello(String name) {
        return String.format("Hello %s.", name);
    }

    @Override
    public String sayGoodBye(String name) {
        return String.format("%s Good Bye.", name);
    }
}

public class CalcServiceImpl implements ICalcService {

    @Override
    public Long add(Long a, Long b) {
        return a + b;
    }

    @Override
    public Long minus(Long a, Long b) {
        return a - b;
    }
}

C/S通信協議

定義InterfaceWrapper: Client/Server間通信協議數據包裝對象:

public class InterfaceWrapper implements Serializable {

    private Class<?> inter;

    private String methodName;

    private Class<?>[] paramTypes;

    private Object[] arguments;

    public Class<?> getInter() {
        return inter;
    }

    public void setInter(Class<?> inter) {
        this.inter = inter;
    }

    public String getMethodName() {
        return methodName;
    }

    public void setMethodName(String methodName) {
        this.methodName = methodName;
    }

    public Class<?>[] getParamTypes() {
        return paramTypes;
    }

    public void setParamTypes(Class<?>[] paramTypes) {
        this.paramTypes = paramTypes;
    }

    public Object[] getArguments() {
        return arguments;
    }

    public void setArguments(Object[] arguments) {
        this.arguments = arguments;
    }
}

服務端: Producer

Producer: 服務生產者

public class Producer {

    private static final int _1H = 1000 * 60 * 60;

    @Test
    public void producer() throws IOException, InterruptedException {
        IHelloService helloService = new HelloServiceImpl();
        ICalcService calcService = new CalcServiceImpl();
        ProducerClient.getInstance().publish(IHelloService.class, helloService);
        ProducerClient.getInstance().publish(ICalcService.class, calcService);

        Thread.sleep(_1H);
    }
}

ProducerClient

public class ProducerClient {

    static {
        new Thread(new ServerBootstrap()).start();
    }

    private static ProducerClient instance = new ProducerClient();

    public static ProducerClient getInstance() {
        return instance;
    }

    public void publish(Class<?> inter, Object impl) {
        ServiceImplMap.push(inter, impl);
    }
}

ProducerClient是面向生產者的Client接口,系統啓動時即啓動一個新線程ServerBootstrap建立ServerSocket等待在accept()上:

public class ServerBootstrap implements Runnable {

    private static final Logger LOGGER = LoggerFactory.getLogger(ServerBootstrap.class);

    @Override
    public void run() {
        ServerSocket server = ServerSocketInstance.getServerSocket();

        try {
            while (true) {
                Socket client = server.accept();
                ThreadPoolInstance.getExecutor().submit(new ProducerInvokeTask(client));
            }
        } catch (IOException e) {
            LOGGER.error("accept error: ", e);
        }
    }
}

每當發佈一個服務, ProducerClient向ConcurrentMap中插入一條記錄(便於連接建立時從Map中獲得interface對應Service實例), 並將服務端ip:port發佈到ZooKeeper, 便於客戶端發現並連接:

public class ServiceImplMap {

    private static final ConcurrentMap<Class<?>, Object> map = new ConcurrentHashMap<>();

    private static final ZkClient zk;

    private static final String FIRST_LEVEL_PATH = "/mi.rpc";


    static {
        zk = new ZkClient(ConfigMap.getServer("zk.servers"));
    }


    private static void createNode(String first, String second, String dest) {

        if (!zk.exists(first)) {
            zk.createPersistent(first);
        }

        if (!zk.exists(first + "/" + second)) {
            zk.createPersistent(first + "/" + second);
        }

        zk.createEphemeral(first + "/" + second + "/" + dest);
    }

    public static void push(Class<?> inter, Object impl) {
        map.put(inter, impl);

        // 發佈到ZK
        String ip;
        try {
            ip = InetAddress.getLocalHost().getHostAddress();
        } catch (UnknownHostException e) {
            ip = null;
            e.printStackTrace();
        }
        int port = ServerSocketInstance.getServerSocket().getLocalPort();
        createNode(FIRST_LEVEL_PATH, inter.getName(), String.format("%s:%s", ip, port));
    }

    public static Object get(Class<?> inter) {
        return map.get(inter);
    }
}

一旦有客戶端建立連接(服務消費者嘗試調用服務端方法), ServerBootstrap爲該Socket分配一個線程:
1. 首先從Socket讀取數據並反序列化爲InterfaceWrapper對象;
2. 根據InterfaceWrapper提供的數據調用目標服務方法;
3. 得到計算結果, 序列化後寫入Socket.

public class ProducerInvokeTask implements Runnable {

    private static final Logger LOGGER = LoggerFactory.getLogger(ProducerInvokeTask.class);

    private Socket client;

    public ProducerInvokeTask(Socket client) {
        this.client = client;
    }

    @Override
    public void run() {
        try (InputStream in = client.getInputStream();
             OutputStream out = client.getOutputStream()) {

            // 讀出數據 反序列化
            byte[] bytes = ReaderUtil.toByteArray(in);
            InterfaceWrapper wrapper = Hessian2Serializer.deserialize(bytes);

            // 執行方法調用
            Object service = ServiceImplMap.get(wrapper.getInter());
            Method method = service.getClass().getMethod(wrapper.getMethodName(), wrapper.getParamTypes());
            Object result = method.invoke(service, wrapper.getArguments());

            // 序列化 寫入數據
            bytes = Hessian2Serializer.serialize(result);
            out.write(bytes);
        } catch (IOException | NoSuchMethodException | InvocationTargetException | IllegalAccessException e) {
            LOGGER.error("Producer Invoke Task Error: ", e);
        }
    }
}

客戶端: Consumer

public class Consumer {

    @Test
    public void consumer() {
        IHelloService helloService = ConsumerClient.getInstance().subscribe(IHelloService.class);
        ICalcService calcService = ConsumerClient.getInstance().subscribe(ICalcService.class);

        String hello = helloService.sayHello("翡青");
        System.out.println(hello);
        String goodBye = helloService.sayGoodBye("翡青");
        System.out.println(goodBye);
        long add = calcService.add(1L, 1L);
        System.out.println(add);
        long minus = calcService.minus(1L, 1L);
        System.out.println(minus);
    }
}

ConsumerClient

客戶端需要訂閱服務: 從ZooKeeper中拉取該interface對應服務端ip:port, 返回一個動態代理對象:

public class ConsumerClient {

    private static ConsumerClient instance = new ConsumerClient();

    public static ConsumerClient getInstance() {
        return instance;
    }

    private static final String PATH = "/mi.rpc/%s/";

    private static ZkClient zk;

    static {
        zk = new ZkClient(ConfigMap.getClient("zk.servers"));
    }

    // 從ZooKeeper拉取數據, 簡易負載均衡
    private String getAddress(String name) {
        List<String> children = zk.getChildren(String.format(PATH, name));
        int index = new Random().nextInt(children.size());
        return children.get(index);
    }

    @SuppressWarnings("all")
    public <T> T subscribe(Class<T> inter) {
        checkInterface(inter);
        String[] address = getAddress(inter.getName()).split(":");
        String ip = address[0];
        int port = Integer.valueOf(address[1]);

        return (T) Proxy.newProxyInstance(inter.getClassLoader(), new Class[]{inter}, new ConsumerIInvokeTask(inter, ip, port));
    }

    private void checkInterface(Class<?> inter) {
        if (inter == null || !inter.isInterface()) {
            throw new IllegalArgumentException("inter Mast a interface class");
        }
    }
}

待到客戶端實際調用interface內方法, 纔會執行ConsumerIInvokeTask的invoke():
1. 創建與ServerSocket連接,將需要的接口信息(接口Class、方法名、參數類型、參數值)包裝成InterfaceWrapper使用Hession2序列化後傳遞給服務端;
2. 等待服務端接收數據執行方法然後將結果數據序列化返回;
3. 客戶端反序列化爲目標對象, 然後invoke()返回執行結果:

public class ConsumerIInvokeTask implements InvocationHandler {

    private Class<?> inter;

    private String ip;

    private int port;

    public ConsumerIInvokeTask(Class<?> inter, String ip, int port) {
        this.inter = inter;
        this.ip = ip;
        this.port = port;
    }

    @Override
    public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {

        Socket client = new Socket(ip, port);

        try (
                OutputStream out = client.getOutputStream();
                InputStream in = client.getInputStream()
        ) {
            // 序列化 寫入數據
            InterfaceWrapper wrapper = new InterfaceWrapper();
            wrapper.setInter(this.inter);
            wrapper.setMethodName(method.getName());
            wrapper.setParamTypes(method.getParameterTypes());
            wrapper.setArguments(args);

            byte[] bytes = Hessian2Serializer.serialize(wrapper);
            out.write(bytes);

            // 讀出數據 反序列化
            bytes = ReaderUtil.toByteArray(in);
            return Hessian2Serializer.deserialize(bytes);
        }
    }
}

限於篇幅此只列出最核心代碼, 詳細參考Git地址: https://git.oschina.net/feiqing/MiRPC.git

參考 & 擴展: 你應該知道的 RPC 原理; Step by step玩轉RPC; 深入淺出 RPC - 淺出篇; 深入淺出 RPC - 深入篇; Effective java 中文版（第2版）; Java併發編程實戰; 瘋狂Java講義; 分佈式服務框架：原理與實踐

網絡與RPC

網絡與RPC

基礎組件

TCP協議

ServerSocket

Socket

半關閉Socket

UDP協議

DatagramSocket

DatagramPacket

Proxy

Proxy

ProxySelector

NIO

NIO示例

小結

AIO

RPC

服務

C/S通信協議

服務端: Producer

客戶端: Consumer

Python 潮流週刊#50：我最喜歡的 Python 3.13 新特性！

Spring MVC 實踐 - Component

網絡與RPC

Redis與Java - 數據結構

Spring MVC 實踐 - Base

Servlet - Upload、Download、Async、動態註冊

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結