分佈式系統設計理念這麼難學?

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"link","attrs":{"href":"#分佈式系統","title":null}},{"type":"text","text":"分佈式系統"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"身爲二十一世紀的一名程序員,沒聽說過分佈式系統就顯得自己好像沒有女票一樣尷尬。無論是出去面試跟面試官吹水,還是在工作中和同事吹水,分佈式系統永遠是你顯得高人一等的籌碼。分佈式系統已經誕生了好幾十年,說起來比我們八零後程序員好要老成,隨着現代互聯網的崛起,對於系統在性能,可靠性上的要求大大提高。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統的定義其實很簡單,也很抽象:任何由處於不同物理位置的多個進程提供相同服務的系統都可以稱之爲分佈式系統,退一萬步講,同一臺服務器上的不同進程也可以組成分佈式系統"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統的首要目標是提高系統的整體性能,但不僅限於吞吐量,可靠性,響應時間,數據一致性等,其中提高性能指標是最重要的。如果最終設計出來的分佈式系統佔用了更多的資源卻還比不上單機的性能,那這個分佈式系統是失敗的,理論上沒有存在的價值"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個分佈式系統的整體性能提高並不是單單依靠擴展來實現,提高單機的處理性能仍然很重要,一個把單機性能發揮到極致的分佈式系統,在同等性能的需求下,採用的資源要遠遠小於其他系統。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"link","attrs":{"href":"#分佈式系統痛點","title":null}},{"type":"text","text":"分佈式系統痛點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個好的分佈式系統在性能方面要遠超單機系統,但是在數據行爲方面要表現的和單機系統一樣優秀,其中包括數據的一致性,硬件的故障發生率,網絡的不穩定性等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無論是單機系統還是分佈式系統都存在無法迴避並且無法徹底去除的風險,比如:硬盤發生故障,網絡發生癱瘓,光纖被挖.....分佈式系統隨着節點的增加,把這些故障的發生率也隨之增大,所以分佈式系統其中一個目標是要儘量降低這些風險,也就是所謂的容錯性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"既要快還要不出錯,這在“倫理”上是衝突的。就像我們平時說的分佈式鎖,如果要保證對一個資源的修改不會發生線程安全問題,就要付出降低性能的代價。至於性能和容錯性怎樣選擇,還需要具體到每個業務場景中,比如支付場景中,數據的正確性可能要比性能指標更重要,而那些日誌型數據,比如用戶的登錄日誌,這些數據的最大特點就是允許小部分丟失,在這樣的日誌系統設計中,可能性能指標要大於容錯性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前爛大街的CAP原則的講解,是針對分佈式系統的一個抽象理論,包括之後BASE理論,也是針對分佈式系統的一種指導方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"link","attrs":{"href":"#設計分佈式系統","title":null}},{"type":"text","text":"設計分佈式系統"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統的特性就決定了它自出生之日起,就有多個節點如何協同工作的難題。就像一個團隊,如果讓這個團隊有條不紊的工作本來就是個難題。一堆節點爲了完成同樣的任務,註定需要一個規範方圓的規則。就目前已知的方案中,主要有中心化和去中心化兩種解決方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#中心化","title":null}},{"type":"text","text":"中心化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"中心化的分佈式設計理念是目前主流的方案,在中心化的設計方案中,節點是有角色區分的:Leader節點和Work節點,即:領導和幹活的。就和現實中類似,leader只負責分發任務和監督,Work節點只負責領取任務幹活,多說一句,這裏Work節點領取任務,當然從通信的角度來說,又可以分爲push和pull(推和拉)方式。推方式是指,leader節點主動將任務分發給Work節點,拉方式是指:Work節點主動去申請任務。至於push和pull的優缺點,不作爲今天的主題展開討論。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3f/3fcfca15c00f7ee70e0f9061f8f8af69.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在任何系統中,都無可避免的需要考慮節點down掉的問題,分佈式系統也一樣。在中心化分佈式設計中,leader和Work也一樣都存在down掉的可能性。如果leader掉的話,整個系統都陷入癱瘓,按照最簡單的設計思路,leader節點需要一個從節點或者備用節點,在主節點down掉之後,從主節點或者備用主節點可以手動或者自動實現leader節點服務。至於一個Work節點down掉,一般不會使整個系統陷入癱瘓,除非全部Work節點同時down掉。一個Work節點down掉,但是會影響這個節點當前正在執行的任務,所以在必要的條件下可以設計成任務需要Ack纔好,即:一個任務的完成需要確認,如果長時間沒有確認,leader會發起重新分配任務的操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"說到leader的問題,現在目前大多數分佈式系統都具備了自動選舉leader的功能,這還要感謝paxos,raft等選舉算法。在leader不可用的時候,這些系統會自動根據節點情況選舉出新的leader節點來繼續提供服務,這大大提高了系統可用性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在所有的中心化設計中,數據的寫操作都發生在leader節點,這在某種程度上類似於單機系統,所以這種中心化設計並不適合那些大量寫的操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#去中心化","title":null}},{"type":"text","text":"去中心化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在去中心化分佈式系統設計中,節點類型並不區分Leader和Work,所有節點都是相等的。所以任何一個節點down掉都不會導致整個系統癱瘓,這是它的優勢。但是獲取系統中每個節點的信息卻比中心化設計要難很多,在中心化設計中,leader節點存儲着系統中所有的節點信息,並可以實時把這些信息同步到其他節點,同時可以利用相應算法來達到一致性的要求。去中心化的設計中,每個節點只能依靠和其他節點不斷通信來獲取整個系統的節點信息,這在技術難度上要比中心化高出很多。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在網絡中,網絡是不可靠的。恰恰是這個原因,又加大了每個節點互相通信的難度。在極限情況下,去中心化的設計方案會出現多個小範圍的“團伙”,這就是所說的腦裂。比如:現在一個由10個節點組成的分佈式系統,有可能由於網絡原因會劃分爲兩個5節點互相通信的兩個“團伙”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b4/b4278c5f17a0418555faf2838a4815c8.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果出現腦裂的情況,目前主流的解決方案和數據庫死鎖的處理情況類似,自爆一個對系統影響比較小的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"完全中心化和去中心化的系統並不常見,反而現在慢慢出現的是二者的搭配者,表面看似去中心化,設計理念卻是中心化的思路,在這種架構下,leader是程序根據某種算法選舉出來的,而且在系統leader發生故障的時候,系統會自動重新選舉leader節點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"link","attrs":{"href":"#寫在最後","title":null}},{"type":"text","text":"寫在最後"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於每個系統來說,可靠性是它要實現的主要目標之一,尤其是分佈式系統。在網絡通信,硬件設備等條件都非100%可靠的情況下,如何提高分佈式系統的可用性是一個很深的話題。就算是國內頂尖的BAT等大廠,也沒有一個系統能達到100%的可用性,4個9的可用性已經是很巔峯了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統本質上是多個節點通過網絡IO組成的,其中夾雜着一些不可抗拒的元素,所以請記住一句話:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統是不可靠的,我們只能儘量減小故障發生率,卻根除不了,如果你的老闆要你設計一套100%可用性的系統,要麼他是二貨,要麼他是二B"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章