微服務架構服務容錯設計分析

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"color","attrs":{"color":"#40A9FF","name":"blue"}},{"type":"strong","attrs":{}}],"text":"真正的大師永遠懷着一顆學徒的心","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"引言","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在微服務體系架構中，由於拆解的服務數變多了，服務發生故障的地方也會相應的增加，因此如何保證服務架構健壯是一個值得深思的問題。微服務容錯機制正是這樣一種穩定性解決方案，可以理解微微服務架構的保險絲，通過它可以對業務平臺形成一種有效的保護機制。在發生平臺異常時候，容錯機制是平臺穩定運行的最後一道屏障。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"微服務架構爲什麼需要容錯機制","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"說起來可能有一些年頭了，以前小時候家裏面經常出現電壓不穩電燈忽明忽暗，有時候甚至出現短路停電的情況。每次一片漆黑的時候，大人們最常說的一句話就是：哎，估計又是保險絲斷了哦。這裏的保險絲就是保護家中電路電器的一種手段，當發生短路情況時，通過熔斷保險絲來保護家中電器不受電流過載的影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在微服務架構體系中的熔斷降級正是起到保險絲作用的基礎組件。我們在進行架構設計時，不僅需要滿足業務要求，同時也需要面向失敗進行設計，意思就是說當外部條件發生變化或者內部出現異常時，平臺的架構能夠將這種異常的影響降到最低，強大的容錯能力是優秀架構的關鍵指標。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"回到今天的主題容錯機制，我們可以反過來想，如果沒有如果沒有熔斷降級系統容錯機制，整個系統平臺在異常情況下，會發生怎樣的系統反應，我們先看下以下幾種場景。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"場景一：服務節點異常影響上游服務調用方","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設我們有客戶端服務，需要分別調用Service1集羣的接口、Service2集羣的接口以及Service3集羣的接口來完成一項業務流程，如果Service3集羣發生異常情況，服務都還在，但是可能由於出現fullGC、慢查詢、業務異常等情況，客戶端在調用Service3集羣時出現timeout，不能在規定時間內進行服務響應。當調用請求不斷髮出時，此時Client中的工作線程將會被這些調用的time out阻塞住，當業務不斷進行請求時，Client對應的工作線程會越來越多的被阻塞住，進而導致客戶端不可用。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/90/9026dff843af76b2a9306e5d91afd129.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當此客戶端不可用時，可能上層調用方也會由","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Client","attrs":{}}],"attrs":{}},{"type":"text","text":"的不可用向上影響，導致上層調用方也出現異常，就像病毒一樣一層一層的傳導，異常影響被不斷向上放大，最終導致整個平臺不可用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ef/ef7ef8574fef1cd104e7ead0171c8e10.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"場景二：流量激增導致服務不可用，影響其他依賴該服務的服務異常","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們假設Service1集羣可以承載6000QPS的流量，正常情況下上游三個服務的流量總和都小於這個閾值。但是當流量發生激增時，其中一個服務的QPS就超過了10000QPS，超出了Service1集羣的服務能力，因此造成了集羣的異常出現響應異常的情況，此時Service2集羣以及Service3集羣由於依賴Service1集羣的服務，同樣會出現線程阻塞的情況，最終導致整個平臺的異常。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6c/6c235c292c9ab8e1b8b99aefddf60455.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此基於以上分析，微服務架構中引入熔斷降級組件是爲了提升微服務架構整體的容錯能力。主要避免以下三種場景對平臺穩定性的影響。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、單個服務集羣節點出現異常故障，其影響範圍可能被無限向上遊服務放大；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、由於使用了共同基礎服務，基礎服務出現異常時，多租戶相互影響；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3、某個服務的瞬時流量突增，某個服務集羣扛不住，影響整個平臺穩定性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"如何破局","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"資源隔離","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們以現實生活中的船艙爲例，實際的船艙格局大致如下所示，船艙底部並不是完全的一整個空心結構，而是通過格子進行區域隔離。爲什麼這麼設計呢？主要還是爲了一旦發生船艙漏水，隻影響其中一個隔離區域，而不是整個船艙都灌滿水，從而達到一定程度保護船體不會因爲一處漏洞而沉沒的目的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼藉助於船艙隔離的思想，在我們的程序世界中，我們是不是也可以採用資源隔離的方式來保護我們的微服務架構呢？答案是肯定的，也的確是這麼做的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/08/0873eb31636c2fff91781665b0c35c26.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"1、線程池隔離","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們可以通過線程池隔離的方式來實現資源的隔離，不同的請求使用相應的線程池來處理，即便出現請求資源time out的情況，最多影響當前線程池的資源，而不會影響整個服務的線程資源。類似船艙中的隔離區域。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/84/84b3651ece52df7ac623b12fe55a2a79.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"2、信號量隔離","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"信號量主要是用來控制線程數的，規定好一些調用最大的併發量，超過指定的信號量後，可以將請求丟棄或者延時處理，防止線程的不斷增長導致的服務異常。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6a/6a992da3a09d2a29c0ba233f83d025ff.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"熔斷","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所謂熔斷，正如上文所說，它的作用就是想保險絲一樣，在流量過大或者請求錯誤率過大的情況下，此時保險絲就熔斷了，對應的業務鏈路斷開，不再提供服務。當流量恢復正常或者錯誤降低後，熔斷開關再進行閉合，之前的業務鏈路重新恢復。這是一種很好的保護後端微服務的一種方式。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在大促期間，平臺需要用足夠的機器去保證核心商業鏈路正常運轉，對於退款、退貨這種服務，則可以通過暫時熔斷的方式不對外提供服務，當大促過了之後再進行恢復。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b9/b9a90084ca9fded4c54c306258fb2c44.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在熔斷機制中，核心的內容就是斷路器的設計，斷路器設計主要有兩方面一個是狀態轉換的設計，一個是如何根據閾值以統計數據來執行核心的斷路功能。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"降級","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和熔斷的場景有點蕾絲，當系統流量過多時，系統資源有限，平臺處理不過來那麼多的請求。此時就可以可將不是那麼重要的功能模塊進行降級處理，停止對外服務，這樣可以釋放出更多的資源供給核心功能的去用。如下所示，商品詳情頁中，商品服務中方商品列表纔是最重要的服務，至於用戶的積分以及用戶的頭像此時並不是核心業務，所以在系統能力有限的情況下，優先讓商品服務對外提供服務，其他服務進行降級處理。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/35/35b528fe69f5396b87526b079381f8c9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文主要對微服務架構中的容錯機制進行了分析，從爲什麼要有容錯機制到如何通過資源隔離、熔斷以及降級等方式實現微服務容錯保護進行了闡述。下一篇文章將帶大家看看熔斷降級組件","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Hystrix","attrs":{}}],"attrs":{}},{"type":"text","text":"的工作原理，敬請期待哦。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我是慕楓，感謝各位小夥伴點贊、收藏和評論，文章持續跟新，我們下期再見！","attrs":{}}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"微信搜索：慕楓技術筆記，優質文章持續更新，我們有學習打卡的羣可以拉你進，一起努力衝擊大廠，另外有很多學習以及面試的材料提供給大家。","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7f/7ff6f4cf7dc92ed189149727ace4dc43.gif","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}