NGINX + TOMCAT出現大量的TIME-WAIT狀態的TCP連接解決

昨天把兩個tomcat服務停止後，更新項目，更新完啓動，兩個tomcat都報了異常，其中一個tomcat報的異常如下：

[com.alibaba.druid.proxy.DruidDriver]，但在Web應用程序停止時無法註銷它。 爲防止內存泄漏，JDBC驅動程序已被強制取消註冊。
14-May-2020 14:38:24.188 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web應用程序 [ROOT] 註冊了JDBC驅動程序 [com.mysql.jdbc.Driver]，但在Web應用程序停止時無法註銷它。 爲防止內存泄漏，JDBC驅動程序已被強制取消註冊。
14-May-2020 14:38:24.189 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [ROOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 14:38:24.231 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web應用程序 [WebLogin] 註冊了JDBC驅動程序 [com.alibaba.druid.proxy.DruidDriver]，但在Web應用程序停止時無法註銷它。 爲防止內存泄漏，JDBC驅動程序已被強制取消註冊。
14-May-2020 14:38:24.231 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc Web應用程序 [WebLogin] 註冊了JDBC驅動程序 [com.mysql.jdbc.Driver]，但在Web應用程序停止時無法註銷它。 爲防止內存泄漏，JDBC驅動程序已被強制取消註冊。
14-May-2020 14:38:24.232 警告 [Thread-6] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [WebLogin] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 14:38:24.245 信息 [Thread-6] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["http-nio-9026"]
14-May-2020 14:38:24.247 信息 [Thread-6] org.apache.coyote.AbstractProtocol.stop Stopping ProtocolHandler ["ajp-nio-9029"]
14-May-2020 14:38:24.250 信息 [Thread-6] org.apache.coyote.AbstractProtocol.destroy 正在摧毀協議處理器 ["http-nio-9026"]
14-May-2020 14:38:24.251 信息 [Thread-6] org.apache.coyote.AbstractProtocol.destroy 正在摧毀協議處理器 ["ajp-nio-9029"]

另一個tomcat報的異常如下：

14-May-2020 19:50:17.160 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [myProject] registered the JDBC driver [com.alibaba.druid.proxy.DruidDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
14-May-2020 19:50:17.162 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesJdbc The web application [myProject] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
14-May-2020 19:50:17.230 警告 [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [myProject] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(Unknown Source)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
14-May-2020 19:50:17.310 信息 [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [D:\tomcatServer\tomcat-9.0.10\webapps\myProject.war] has finished in [7,240] ms
14-May-2020 19:50:17.314 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\docs]
14-May-2020 19:50:17.361 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\docs] has finished in [47] ms
14-May-2020 19:50:17.362 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\manager]
14-May-2020 19:50:17.391 信息 [main] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [D:\tomcatServer\tomcat-9.0.10\webapps\manager] has finished in [28] ms
14-May-2020 19:50:17.395 信息 [main] org.apache.catalina.startup.Catalina.start Server startup in 18478 ms
14-May-2020 19:50:17.397 嚴重 [main] org.apache.catalina.core.StandardServer.await StandardServer.await: create[localhost:9015]: 
 java.net.BindException: Address already in use: JVM_Bind
	at java.net.DualStackPlainSocketImpl.bind0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketBind(Unknown Source)
	at java.net.AbstractPlainSocketImpl.bind(Unknown Source)
	at java.net.PlainSocketImpl.bind(Unknown Source)
	at java.net.ServerSocket.bind(Unknown Source)
	at java.net.ServerSocket.<init>(Unknown Source)
	at org.apache.catalina.core.StandardServer.await(StandardServer.java:422)
	at org.apache.catalina.startup.Catalina.await(Catalina.java:770)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:716)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)

當時是百思不得其解，我是正常關閉tomcat服務以後才更新的文件，怎麼會報內存溢出呢？當時也是一時手足無措，把想到的關聯因素redis和nginx關閉後重新啓動還是老樣子。今天有點頭緒，會不會是關閉tomcat服務時還有殘餘線程存在呢？於是今天再現昨天的情景，先把兩個tomcat關閉，然後通過cmd命令查看端口號佔用，

 可以看到，儘管兩個tomcat服務已關閉，但依然有TIME_WAIT進程在佔用。所以重新啓動服務時報了內存溢出的異常。由於我通過nginx反向代理了tomcat，受到TCP協議的重傳機制的限制。那麼問題就出在tomcat和nginx的配置上了，查了下網上資料，主要原因是：

1.nginx開啓了keepalive ，而且默認用的http1.1

2.tomcat開啓了keepalive，默認也使用http1.1

3.但是nginx代理卻使用的http1.0，默認不開啓keepalive，所以就造成了大量的TIME-WAID狀態的TCP連接

同時也給出兩種解決方法，第二種我沒試，感興趣可以前往一試https://my.oschina.net/haitaohu/blog/3043113，不知效果如何，我是參照第一種方法即nginx官網的方法，修改我的nginx配置，

nginx官網提供

For HTTP, the proxy_http_version directive should be set to “1.1” and the “Connection” header field should be cleared:
參考地址:http://nginx.org/en/docs/http/ngx_http_upstream_module.html

upstream http_backend {
    server 127.0.0.1:8080;

    keepalive 16;
}

server {
    ...

    location /http/ {
        proxy_pass http://http_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        ...
    }
}

修改好後，重新加載nginx,再次使用cmd命令查看端口號佔用情況，結果如下：

可以看到，原先TIME_WAIT那些進程消失了，然後去啓動tomcat,，去看日誌，發現異常已經消失，也可以訪問項目了。