Java踩坑記系列之線程池

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"線程池大家都很熟悉，無論是平時的業務開發還是框架中間件都會用到，大部分都是基於JDK線程池","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"ThreadPoolExecutor","attrs":{}}],"attrs":{}},{"type":"text","text":"做的封裝，比如tomcat的線程池，當然也有單獨開發的，但都會牽涉到這幾個核心參數的設置：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"核心線程數","attrs":{}},{"type":"text","text":"，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"等待隊列","attrs":{}},{"type":"text","text":"，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最大線程數","attrs":{}},{"type":"text","text":"，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拒絕策略","attrs":{}},{"type":"text","text":"等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"先說下我們項目組在使用線程池時踩到的坑：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"線程池的參數設置一定要結合具體的業務場景，區分I/O密集和CPU密集，如果是I/O密集型業務，核心線程數，","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"workQueue","attrs":{}}],"attrs":{}},{"type":"text","text":"等待隊列，最大線程數等參數設置不合理不僅不能發揮線程池的作用，反而會影響現有業務","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"等待隊列","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"workQueue","attrs":{}}],"attrs":{}},{"type":"text","text":"填滿後，新創建的線程會優先處理新請求進來的任務，而不是去處理隊列裏的任務，隊列裏的任務只能等核心線程數忙完了才能被執行。有可能造成隊列裏的任務長時間等待，導致隊列積壓，尤其是I/O密集場景","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"如果需要得到線程池裏的線程執行結果，使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future","attrs":{}}],"attrs":{}},{"type":"text","text":"的方式，拒絕策略不能使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"DiscardPolicy","attrs":{}}],"attrs":{}},{"type":"text","text":"，這種丟棄策略雖然不執行子線程的任務，但是還是會返回","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future","attrs":{}}],"attrs":{}},{"type":"text","text":"對象(其實在這種情況下我們已經不需要線程池返回的結果了)，然後後續代碼即使判斷了","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future!=null","attrs":{}}],"attrs":{}},{"type":"text","text":"也沒用，這樣的話還是會走到","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future.get()","attrs":{}}],"attrs":{}},{"type":"text","text":"方法，如果get方法沒有設置超時時間會導致一直阻塞下去!","attrs":{}}]}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"僞代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"// 如果線程池已滿，新的請求會直接執行拒絕策略\nFuture future = executor.submit(() -> {\n // 業務邏輯,比如調用第三方接口等耗時操作放在線程池裏執行\n return result;\n});\n\n// 主流程調用邏輯\nif(future != null) // 如果拒絕策略設置不合理還是會走到下面代碼\n future.get(超時時間); // 調用方阻塞等待結果返回，直到超時","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面就結合實際業務情況逐一進行分析。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然這些問題一部分是對線程池理解不夠導致的，還有一部分是線程池本身的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一. 背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"公司有個接口部分功能使用了線程池，這個功能不依賴核心接口，但有一定的耗時，所以放在線程池裏和主線程並行執行，等線程池裏的任務執行完通過","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future.get","attrs":{}}],"attrs":{}},{"type":"text","text":"的方式獲取線程池裏的線程執行結果，然後合併到主流程的結果裏返回給前端，業務場景很簡單，大致流程如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/de/debc640489b0cea1eddac695b75156ca.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"初衷也是爲了不影響主流程的性能，不增加整體響應時間。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是之前使用的線程池jdk的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"newCachedThreadPool","attrs":{}}],"attrs":{}},{"type":"text","text":"，因爲sonar掃描提示說有內存溢出的風險(最大線程數是","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Integer.MAX_VALUE","attrs":{}}],"attrs":{}},{"type":"text","text":")所以當時改成使用原生的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"ThreadPoolExecutor","attrs":{}}],"attrs":{}},{"type":"text","text":"，通過指定核心線程數和最大線程數，來解決sonar問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是改過的線程池並不適合我們這種I/O密集型的業務場景(大部分業務都是通過調用接口實現的)，當時設置的核心線程數是cpu核數(線上機器是4核)，等待隊列是2048，最大線程數是cpu核數*2，從而引發了一系列問題。。。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二. 排查過程","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上線後的現象是使用線程池的接口整體響應時間變長，有的甚至到10秒才返回數據，通過線程dump分析發現有大量的線程都阻塞在","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future.get","attrs":{}}],"attrs":{}},{"type":"text","text":"方法上，如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9e/9e5c7ec561e1473ce99a59e8c585608e.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"future.get","attrs":{}}],"attrs":{}},{"type":"text","text":"方法會阻塞當前主流程，在超時時間內等待子線程返回結果，如果超時還沒結果則結束等待繼續執行後續的代碼，超時時間設置的是默認接口超時時間10秒(後面已改爲200ms)，至此可以確定接口總耗時是因爲流程都卡在了","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future.get","attrs":{}}],"attrs":{}},{"type":"text","text":"這一步了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但這不是根本原因，","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future","attrs":{}}],"attrs":{}},{"type":"text","text":"是線程池返回的，僞代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"Future future = executor.submit(() -> {\n // 業務邏輯,比如調用第三方接口等耗時操作放在線程池裏執行\n return result;\n});","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過上面的代碼可知","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future","attrs":{}}],"attrs":{}},{"type":"text","text":"沒有結果的原因是提交到線程池裏的任務遲遲沒有被執行。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那爲什麼沒有執行呢？繼續分析線程池的dump文件發現，線程池裏的線程數已達到最大數量，滿負荷運行，如圖：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f7/f74edd36d449fd6f75cbcecb906c92d1.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"SubThread","attrs":{}}],"attrs":{}},{"type":"text","text":"是我們自己定義的線程池裏線程的名字，8個線程都是runnable狀態，說明等待隊列裏已經塞滿任務了，之前設置的隊列長度是2048，也就是說還有2048個任務等待執行，這無疑加劇了整個接口的耗時。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"線程池的執行順序是：","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"核心線程數","attrs":{}},{"type":"text","text":" -> ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"等待隊列","attrs":{}},{"type":"text","text":" -> ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最大線程數","attrs":{}},{"type":"text","text":" -> ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拒絕策略","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果對線程dump分析不太瞭解的可以看下之前的一篇文章：","attrs":{}},{"type":"link","attrs":{"href":"http://javakk.com/176.html","title":null},"content":[{"type":"text","text":"Windows環境下如何進行線程dump分析","attrs":{}}]},{"type":"text","text":"，雖然環境不一樣但原理類似。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏基本確定接口耗時變長的主要原因是線程池設置不合理導致的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外還有一些偶發問題，就是線上日誌顯示雖然線程池執行了，但是線程池裏的任務卻沒有記錄運行日誌，線程池裏的任務是調用另外一個服務的接口，和對方接口負責人確認也確實調用了他們的接口，可我們自己的日誌裏卻沒有記錄下調用報文，經過進一步查看代碼發現當時的線程池拒絕策略也被修改過，並不是默認的拋出異常不執行策略","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"AbortPolicy","attrs":{}}],"attrs":{}},{"type":"text","text":"，而是設置的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"CallerRunsPolicy","attrs":{}}],"attrs":{}},{"type":"text","text":"策略，即交給調用方執行！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fc/fc2d291f30a7d53bf6b752b03ed6553f.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/765ba86f1796bf2d1fd415650089ef9d.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"也就是說當線程池達到最大負荷時執行的拒絕策略是","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"讓主流程去執行提交到線程池裏的任務，這樣除了進一步加劇整個接口的耗時外，還會導致主流程被hang死，最關鍵的是無法確定是在哪一步執行提交到線程池的任務","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分析日誌埋點可以推斷出調用的時間點應該是已經調用完了記錄日誌的方法，要返回給前端結果的時才執行線程池裏任務，此時記錄日誌的方法已調用過，不會再去打印日誌了，而且子任務返回的結果也無法合併到主流程結果裏，因爲合併主流程結果和線程池任務返回結果的方法也在之前調用過，不會回過頭來再調用了，大致流程如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1c/1cdb12c126f1318513a8aa6d1f9cd5ce.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實這種拒絕策略並不適合我們現在的業務場景，因爲線程池裏的任務不是核心任務，不應該影響主流程的執行。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三. 改進","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"調整線程池參數，核心線程數基於線上接口的QPS計算，最大線程數參考線上tomcat的最大線程數配置，能夠cover住高峯流量，隊列設置的儘量小，避免造成任務擠壓。關於線程數如何設置會在後續文章中單獨講解。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"擴展線程池，封裝原生JDK線程池","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"ThreadPoolExecutor","attrs":{}}],"attrs":{}},{"type":"text","text":"，增加對線程池各項指標的監控，包括線程池運行狀態、核心線程數、最大線程數、任務等待數、已完成任務數、線程池異常關閉等信息，便於實時監控和定位問題。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"重寫線程池拒絕策略，主要也是記錄超出線程池負載情況下的各項指標情況，以及調用線程的堆棧信息，便於排查分析，通過拋出異常方式中斷執行，避免引用的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future","attrs":{}}],"attrs":{}},{"type":"text","text":"不爲null的問題。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"合理調整","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"future.get","attrs":{}}],"attrs":{}},{"type":"text","text":"超時時間，防止阻塞主線程時間過長。","attrs":{}}]}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"線程池內部流程：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d4/d4456c06c156d97baa74f3064f8e2ebc.png","alt":"image","title":"image","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"線程池監控和自定義拒絕策略的代碼如下，大家可以結合自己的業務場景拿去使用：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"package com.javakk;\n\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\nimport java.text.MessageFormat;\nimport java.util.List;\nimport java.util.concurrent.*;\n\n/**\n * 自定義線程池

\n * 1.監控線程池狀態及異常關閉等情況

\n * 2.監控線程池運行時的各項指標, 比如:任務等待數、已完成任務數、任務異常信息、核心線程數、最大線程數等

\n * author: 老K\n */\npublic class ThreadPoolExt extends ThreadPoolExecutor{\n\n private static final Logger log = LoggerFactory.getLogger(ThreadPoolExt.class);\n\n private TimeUnit timeUnit;\n\n public ThreadPoolExt(int corePoolSize,\n int maximumPoolSize,\n long keepAliveTime,\n TimeUnit unit,\n BlockingQueue workQueue,\n ThreadFactory threadFactory,\n RejectedExecutionHandler handler) {\n super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);\n this.timeUnit = unit;\n }\n\n @Override\n public void shutdown() {\n // 線程池將要關閉事件,此方法會等待線程池中正在執行的任務和隊列中等待的任務執行完畢再關閉\n monitor(\"ThreadPool will be shutdown:\");\n super.shutdown();\n }\n\n @Override\n public List shutdownNow() {\n // 線程池立即關閉事件,此方法會立即關閉線程池,但是會返回隊列中等待的任務\n monitor(\"ThreadPool going to immediately be shutdown:\");\n // 記錄被丟棄的任務, 暫時只記錄日誌, 後續可根據業務場景做進一步處理\n List dropTasks = null;\n try {\n dropTasks = super.shutdownNow();\n log.error(MessageFormat.format(\"ThreadPool discard task count:{0}\", dropTasks.size()));\n } catch (Exception e) {\n log.error(\"ThreadPool shutdownNow error\", e);\n }\n return dropTasks;\n }\n\n @Override\n protected void beforeExecute(Thread t, Runnable r) {\n // 監控線程池運行時的各項指標\n monitor(\"ThreadPool monitor data:\");\n }\n\n @Override\n protected void afterExecute(Runnable r, Throwable ex) {\n if (ex != null) { // 監控線程池中的線程執行是否異常\n log.error(\"unknown exception caught in ThreadPool afterExecute:\", ex);\n }\n }\n\n /**\n * 監控線程池運行時的各項指標, 比如:任務等待數、任務異常信息、已完成任務數、核心線程數、最大線程數等

\n */\n private void monitor(String title){\n try {\n // 線程池監控信息記錄, 這裏需要注意寫ES的時機,尤其是多個子線程的日誌合併到主流程的記錄方式\n String threadPoolMonitor = MessageFormat.format(\n \"{0}{1}core pool size:{2}, current pool size:{3}, queue wait size:{4}, active count:{5}, completed task count:{6}, \" +\n \"task count:{7}, largest pool size:{8}, max pool size:{9}, keep alive time:{10}, is shutdown:{11}, is terminated:{12}, \" +\n \"thread name:{13}{14}\",\n System.lineSeparator(), title, this.getCorePoolSize(), this.getPoolSize(),\n this.getQueue().size(), this.getActiveCount(), this.getCompletedTaskCount(), this.getTaskCount(), this.getLargestPoolSize(),\n this.getMaximumPoolSize(), this.getKeepAliveTime(timeUnit != null ? timeUnit : TimeUnit.SECONDS), this.isShutdown(),\n this.isTerminated(), Thread.currentThread().getName(), System.lineSeparator());\n log.info(threadPoolMonitor);\n } catch (Exception e) {\n log.error(\"ThreadPool monitor error\", e);\n }\n }\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自定義拒絕策略代碼：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"package com.javakk;\n\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\nimport java.lang.management.*;\nimport java.text.MessageFormat;\nimport java.util.concurrent.RejectedExecutionException;\nimport java.util.concurrent.RejectedExecutionHandler;\nimport java.util.concurrent.ThreadPoolExecutor;\nimport java.util.concurrent.TimeUnit;\n\n/**\n * 自定義線程池拒絕策略:

\n * 1.記錄線程池的核心線程數,活躍數,已完成數等信息,以及調用線程的堆棧信息,便於排查

\n * 2.拋出異常中斷執行

\n * author: 老K\n */\npublic class RejectedPolicyWithReport implements RejectedExecutionHandler {\n\n private static final Logger log = LoggerFactory.getLogger(RejectedPolicyWithReport.class);\n\n private static volatile long lastPrintTime = 0;\n\n private static final long TEN_MINUTES_MILLS = 10 * 60 * 1000;\n\n private static Semaphore guard = new Semaphore(1);\n @Override\n public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {\n try {\n String title = \"thread pool execute reject policy!!\";\n String msg = MessageFormat.format(\n \"{0}{1}core pool size:{2}, current pool size:{3}, queue wait size:{4}, active count:{5}, completed task count:{6}, \" +\n \"task count:{7}, largest pool size:{8}, max pool size:{9}, keep alive time:{10}, is shutdown:{11}, is terminated:{12}, \" +\n \"thread name:{13}{14}\",\n System.lineSeparator(), title, e.getCorePoolSize(), e.getPoolSize(), e.getQueue().size(), e.getActiveCount(),\n e.getCompletedTaskCount(), e.getTaskCount(), e.getLargestPoolSize(), e.getMaximumPoolSize(), e.getKeepAliveTime(TimeUnit.SECONDS),\n e.isShutdown(), e.isTerminated(), Thread.currentThread().getName(), System.lineSeparator());\n log.info(msg);\n threadDump(); // 記錄線程堆棧信息包括鎖爭用信息\n } catch (Exception ex) {\n log.error(\"RejectedPolicyWithReport rejectedExecution error\", ex);\n }\n throw new RejectedExecutionException(\"thread pool execute reject policy!!\");\n }\n\n /**\n * 獲取線程dump信息

\n * 注意: 該方法默認會記錄所有線程和鎖信息雖然方便debug, 使用時最好加開關和間隔調用, 否則可能會增加latency

\n * 1.當前線程的基本信息:id,name,state

\n * 2.堆棧信息

\n * 3.鎖相關信息(可以設置不記錄)

\n * 默認在log記錄

\n * @return\n */\n private void threadDump() {\n long now = System.currentTimeMillis();\n // 每隔10分鐘dump一次\n if (now - lastPrintTime < TEN_MINUTES_MILLS) { \n return; \n } \n if (!guard.tryAcquire()) { \n return; \n } \n // 異步dump線程池信息 \n ExecutorService pool = Executors.newSingleThreadExecutor(); \n pool.execute(() -> {\n try {\n ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();\n StringBuilder sb = new StringBuilder();\n for (ThreadInfo threadInfo : threadMxBean.dumpAllThreads(true, true)) {\n sb.append(getThreadDumpString(threadInfo));\n }\n log.error(\"thread dump info:\", sb.toString());\n } catch (Exception e) {\n log.error(\"thread dump error\", e);\n } finally {\n guard.release();\n }\n lastPrintTime = System.currentTimeMillis();\n });\n pool.shutdown();\n }\n\n @SuppressWarnings(\"all\")\n private String getThreadDumpString(ThreadInfo threadInfo) {\n StringBuilder sb = new StringBuilder(\"\"\" + threadInfo.getThreadName() + \"\"\" +\n \" Id=\" + threadInfo.getThreadId() + \" \" +\n threadInfo.getThreadState());\n if (threadInfo.getLockName() != null) {\n sb.append(\" on \" + threadInfo.getLockName());\n }\n if (threadInfo.getLockOwnerName() != null) {\n sb.append(\" owned by \"\" + threadInfo.getLockOwnerName() +\n \"\" Id=\" + threadInfo.getLockOwnerId());\n }\n if (threadInfo.isSuspended()) {\n sb.append(\" (suspended)\");\n }\n if (threadInfo.isInNative()) {\n sb.append(\" (in native)\");\n }\n sb.append('n');\n int i = 0;\n\n StackTraceElement[] stackTrace = threadInfo.getStackTrace();\n MonitorInfo[] lockedMonitors = threadInfo.getLockedMonitors();\n for (; i < stackTrace.length && i < 32; i++) {\n StackTraceElement ste = stackTrace[i];\n sb.append(\"tat \" + ste.toString());\n sb.append('n');\n if (i == 0 && threadInfo.getLockInfo() != null) {\n Thread.State ts = threadInfo.getThreadState();\n switch (ts) {\n case BLOCKED:\n sb.append(\"t- blocked on \" + threadInfo.getLockInfo());\n sb.append('n');\n break;\n case WAITING:\n sb.append(\"t- waiting on \" + threadInfo.getLockInfo());\n sb.append('n');\n break;\n case TIMED_WAITING:\n sb.append(\"t- waiting on \" + threadInfo.getLockInfo());\n sb.append('n');\n break;\n default:\n }\n }\n\n for (MonitorInfo mi : lockedMonitors) {\n if (mi.getLockedStackDepth() == i) {\n sb.append(\"t- locked \" + mi);\n sb.append('n');\n }\n }\n }\n if (i < stackTrace.length) {\n sb.append(\"t...\");\n sb.append('n');\n }\n\n LockInfo[] locks = threadInfo.getLockedSynchronizers();\n if (locks.length > 0) {\n sb.append(\"ntNumber of locked synchronizers = \" + locks.length);\n sb.append('n');\n for (LockInfo li : locks) {\n sb.append(\"t- \" + li);\n sb.append('n');\n }\n }\n sb.append('n');\n return sb.toString();\n }\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章來源：","attrs":{}},{"type":"link","attrs":{"href":"http://javakk.com/188.html","title":null},"content":[{"type":"text","text":"http://javakk.com/188.html","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}