性能與Netty並列的Apache NIO HttpServer庫使用詳解

apache java庫家族有一個用NIO實現的http server，性能跟netty並列，而且更加容易使用。

這個庫依賴以下幾個jar包，其中有幾個是必須的，有幾個則在特定功能下才用的到
httpcore-4.4.3.jar
httpcore-nio-4.4.3.jar
這兩個庫是必須的，是http server運行的基礎

httpclient-4.5.1.jar
這個庫不是必須的，但是其中有一些工具類封裝着一些常用解析http請求數據的功能，能提高生產力

commons-fileupload-1.4.jar
javax.servlet-api-3.1.0.jar
這兩個庫在處理上傳文件的時候要用到，如果服務器沒有處理上傳文件請求，可以不導入。

以上jar文件帶的版本號可以忽略，可以下載最新版本的使用

下面講解具體的實現方法

HttpProcessor httpproc = HttpProcessorBuilder.create()
        .add(new ResponseDate())
        .add(new ResponseServer("apache nio http server"))
        .add(new ResponseContent())
        .add(new ResponseConnControl())
        .build();

UriHttpAsyncRequestHandlerMapper reqistry = new UriHttpAsyncRequestHandlerMapper();

reqistry.register("/test_get", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest data, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        httpExchange.getResponse().setEntity(new NStringEntity("hello world"));
        httpExchange.submitResponse();
    }
});


HttpAsyncService protocolHandler = new HttpAsyncService(httpproc, reqistry);
NHttpConnectionFactory<DefaultNHttpServerConnection> connFactory = new DefaultNHttpServerConnectionFactory(
        ConnectionConfig.DEFAULT);

IOEventDispatch ioEventDispatch = new DefaultHttpServerIODispatch(protocolHandler, connFactory);
IOReactorConfig config = IOReactorConfig.custom()
        .setIoThreadCount(2)
        .setSoTimeout(5000)
        .setConnectTimeout(5000)
        .build();
try {
    ListeningIOReactor ioReactor = new DefaultListeningIOReactor(config);
    ioReactor.listen(new InetSocketAddress("127.0.0.1", 8088));
    ioReactor.execute(ioEventDispatch);
} catch ( IOException e ) {
    e.printStackTrace();
}

上面的代碼即啓動的了http server，在瀏覽器中輸入
http://localhost:8088/test_get
就能輸出hello world

上面的代碼有幾部分需要用戶手動配置

HttpProcessor

HttpProcessor httpproc = HttpProcessorBuilder.create()
        .add(new ResponseDate())
        .add(new ResponseServer("apache nio http server"))
        .add(new ResponseContent())
        .add(new ResponseConnControl())
        .build();

這部分用來配置每個請求的響應信息

Connection: keep-alive
Content-Length:1024
Date: Thu, 24 Sep 2020 09:37:34 GMT
Server: http-core-nio

你也可以根據自己的需求自定義實現，繼承HttpResponseInterceptor類即可

UriHttpAsyncRequestHandlerMapper

UriHttpAsyncRequestHandlerMapper reqistry = new UriHttpAsyncRequestHandlerMapper();

reqistry.register("/test_get", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest data, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        httpExchange.getResponse().setEntity(new NStringEntity("hello world"));
        httpExchange.submitResponse();
    }
});

這部分是最重要的，用於映射url和對應的處理程序，它並不難理解，按照這個模板套用即可。

IOReactorConfig

IOReactorConfig config = IOReactorConfig.custom()
        .setIoThreadCount(2)
        .setSoTimeout(5000)
        .setConnectTimeout(5000)
        .build();

這一部分用於設置服務器的核心參數，它可以設置的參數相當的多，其中從應用的角度出發setIoThreadCount是比較重要的，用於設置http server處理請求的線程數量。實際上，這個值設置成1或者2就夠了，也就是用1到2條線程處理網絡請求，因爲使用非阻塞的NIO機制，所以即使單線程也能處理成千上萬的請求，但是這裏有一個前提條件，在請求對應的處理程序中，不能直接處理業務邏輯，而應該將業務邏輯提交給另外的線程池，否則一旦某個業務邏輯阻塞，將影響到整個服務器的運行。

比如我們可以這樣做

ExecutorService executorService = Executors.newFixedThreadPool(10);

reqistry.register("/test_get", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest data, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        executorService.execute(()->{
            try {
                httpExchange.getResponse().setEntity(new NStringEntity("hello world"));
                httpExchange.submitResponse();
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }
        });
    }
});

注意，在線程池的任務中不能直接使用HttpRequest對象，否則會用併發問題，如果要解析HttpRequest中的參數，請在線程池外完成。

此外，當請求處理完畢，必須調用
httpExchange.submitResponse()
否則請求將一直處於等待狀態無法完成。

以上是服務器的基礎用法，也就是
httpcore-4.4.3.jar
httpcore-nio-4.4.3.jar
兩個庫中的功能。

下面如何解析http請求的參數以及處理上傳文件

處理查詢字符串

http://localhost:8088/test_get?a=1&b=2

如果我們通過查詢字符串傳遞參數給服務器，服務器必須要解析這兩個參數

reqistry.register("/test_get", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest request, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        String strUrl = request.getRequestLine().getUri();
        String[] urlItems = strUrl.split("\\?");
        String queryString = "";
        if( urlItems.length >= 2) {
            queryString = urlItems[1];
        }
        //url後面的查詢字符串鍵值對
        List<NameValuePair> queryStringInfo = URLEncodedUtils.parse(queryString,Charset.forName("utf8"));
        System.out.println(queryStringInfo);
        httpExchange.submitResponse();
    }
});

因爲這個庫並沒有對http消息進行深度封裝，我們只能獲得請求的url，然後自己解析字符串，所幸，httpclient-4.5.1.jar 庫提供了工具方法幫助我們實現解析

List<NameValuePair> queryStringInfo = 
    URLEncodedUtils.parse(queryString,Charset.forName("utf8"));

這句代碼就是將
a=1&b=2

這樣的查詢字符串轉換成鍵值對列表，方便我們通過程序訪問。我們也可以將NameValuePair列表抓轉換成Map

Map<String,String> queryStringMap = queryStringInfo.stream()
        .collect(Collectors.toMap(NameValuePair::getName,NameValuePair::getValue));

處理post請求

reqistry.register("/test_post", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest request, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        if( request instanceof BasicHttpEntityEnclosingRequest) {
            BasicHttpEntityEnclosingRequest entityEnclosingRequest = (BasicHttpEntityEnclosingRequest)request;
            HttpEntity httpEntity = entityEnclosingRequest.getEntity();
            String postData = EntityUtils.toString(httpEntity);
            System.out.println(postData);
        }
        httpExchange.submitResponse();
    }
});

處理post請求的方法和處理get的稍有不同

 String postData = EntityUtils.toString(httpEntity)

直到這裏獲得了post提交上來的數據，如果數據是json字符串，則可以通過json庫直接使用。如果是x-www-form-urlencoded之類的鍵值對字符串，則可以跟處理get請求參數一樣處理，轉換成NameValuePair列表

List<NameValuePair> postInfo = 
    URLEncodedUtils.parse(postData,Charset.forName("utf8"));

處理上傳文件

處理上傳文件需要用到這兩個庫
commons-fileupload-1.4.jar
javax.servlet-api-3.1.0.jar

首先需要實現一個繼承自RequestContext的類型

public class FileUploadRequestContext implements RequestContext {
    HttpEntity httpEntity;

    public FileUploadRequestContext(HttpEntity httpEntity) {
        this.httpEntity = httpEntity;
    }

    @Override
    public String getCharacterEncoding() {text
        return "utf8";
    }

    @Override
    public String getContentType() {
        return httpEntity.getContentType().getValue();
    }

    @Override
    public int getContentLength() {
        return (int)httpEntity.getContentLength();
    }

    @Override
    public InputStream getInputStream() throws IOException {
        return httpEntity.getContent();
    }
}

然後以如下方式使用

reqistry.register("/test_upload_file", new HttpAsyncRequestHandler<HttpRequest>(){
    @Override
    public HttpAsyncRequestConsumer<HttpRequest> processRequest(HttpRequest request, HttpContext context) throws HttpException, IOException {
        return new BasicAsyncRequestConsumer();
    }

    @Override
    public void handle(HttpRequest request, HttpAsyncExchange httpExchange, HttpContext context) throws HttpException, IOException {
        if( request instanceof BasicHttpEntityEnclosingRequest) {
            BasicHttpEntityEnclosingRequest entityEnclosingRequest = (BasicHttpEntityEnclosingRequest)request;
            HttpEntity httpEntity = entityEnclosingRequest.getEntity();
            DiskFileItemFactory factory = new DiskFileItemFactory();
            ServletFileUpload upload = new ServletFileUpload(factory);
            upload.setFileSizeMax(1024 * 1024 * 1024);
            try {
                List<FileItem>  fileItems = upload.parseRequest(new FileUploadRequestContext(httpEntity));
                for(FileItem fileItem : fileItems) {
                    //普通數據字段
                    if( fileItem.isFormField()) {
                        String key = fileItem.getFieldName();
                        String value = fileItem.getString();
                    } else {
                        //文件字段
                        try(  FileOutputStream file = new FileOutputStream("pic.jpg") ) {
                            file.write(fileItem.get());
                            file.flush();
                        }
                    }
                }
            } catch (FileUploadException e) {
                e.printStackTrace();
            }
        }
        httpExchange.submitResponse();
    }
});

其中

List<FileItem>  fileItems = 
    upload.parseRequest(new FileUploadRequestContext(httpEntity));

這句代碼將 httpEntity 轉換成 FileItem 列表，FileItem有可能是普通的post數據字段，也可能是文件字段，我們可以通過
fileItem.isFormField()
來判別，如果值爲true則表示普通數據字段，否則是文件。

性能與Netty並列的Apache NIO HttpServer庫使用詳解

HttpProcessor

UriHttpAsyncRequestHandlerMapper

IOReactorConfig

處理查詢字符串

處理post請求

處理上傳文件

這個網絡爬蟲代碼，拿到數據之後如何存到csv文件中去？

【面試準備】【SQL】數據庫有哪些約束？

.NET開源強大、易於使用的緩存框架 - FusionCache

面試，有時候是個運氣活

netty處理http文件上傳

性能與Netty並列的Apache NIO HttpServer庫使用詳解

logback自定義appender讀取xml中的配置 logback自定義appender讀取xml中的配置

沈向洋從微軟離職，挺好的一二三

Java使用Jetty實現靜態網站一二

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結