分佈式系統架構設計 - 從CAP到PACELC

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式系統是一門講究實踐的軟件工程，只有PK過設計方案，從微觀處手把手的敲過一行行的代碼，才能知道細節在哪裏，難點在哪裏，痛點、挑戰點在哪裏。同時，分佈式系統也是一門講究理論的軟件工程，從宏觀處着眼深刻理解系統設計的理論，將理論與實踐相結合，想好、做好、說好纔是真的好。因此，宏觀處着眼，微觀處着手，才能真正掌握分佈式系統。自此，本文開始講述分佈式系統設計裏的CAP與PACELC基礎理論。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"CAP理論與PACELC理論"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"CAP理論"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CAP理論是分佈式系統最爲基本的指導理論之一，是分佈式系統設計時最爲基本的取捨依據，CAP理論認爲一致性、可用性、分區容忍性不能同時滿足，即："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#ffffff","name":"user"}}],"text":"一致性(Consistency): 所有的節點在同一時刻看到同樣的數據；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#ffffff","name":"user"}}],"text":"可用性(Availability): 節點失效不會影響系統的讀寫；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#ffffff","name":"user"}}],"text":"分區容忍性(Partition Tolerance): 系統能支持網絡分區，即使分區之間的消息丟失系統也正常工作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，CAP理論也有其自身的侷限性。在工程實踐中CAP理論的應用可以一分爲二："},{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#ffffff","name":"user"}}],"text":"系統整體以及系統內部。比如，系統整體可以選擇CA或者CP，但是系統內部微觀處有些特性卻可以同時滿足CAP三要素，因爲分區是件極少發生的事，爲了追求卓越的設計理念可以儘量同時滿足CAP三要素。根據業務場景的不同，不同的分佈式系統會根據自身業務的需求在CAP三者中進行取捨， CAP理論的意義是一種在分佈式系統設計時取捨的參考因素，而非絕對的三者必舍其一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，在CAP理論中是沒有提到系統的時延（Latency）的，而訪問時延（Latency）卻是很重要的可用性(Availability)因"},{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#ffffff","name":"user"}}],"text":"素，因此又延申出了PACELC理論。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"PACELC理論"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PACELC理論是CAP理論的擴展，PACELC理論在wiki上的定義是:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"It states that in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C) (as per the CAP theorem), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C)."}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"意思即："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果有分區partition (P)，系統就必須在availability 和consistency (A and C)之間取得平衡; 否則else (E) 當系統運行在無分區情況下,系統需要在 latency (L) 和 consistency (C)之間取得平衡"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖，在PACELC裏添加了Latency要素："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f7/f7fd52874176a88fdd78085a69fa197b.png","alt":"cap-pacelc","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前分佈式系統設計指導理論應當採用PACELC理論替代CAP理論，理由如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PACELC更能滿足實際操作中分佈式系統的工作場景是更好的工程實現策略；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當partition (P)存在的場景下，需要在availability 和consistency (A and C)之間取捨，但是實際上分佈式系統中絕大多數時間裏partition (P)是不存在的，那麼就需要在latency (L) 和 consistency (C)之間作取捨；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Availability在不存在partition (P)的場景下跟 latency關聯,在partition (P)時跟”可靠性“指標相關聯；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PACELC 可以在 latency 與 consistency之間獲得平衡；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CAP 理論忽略了 一致性和時延之間的取捨；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PACELC理論是建立在CAP理論之上的，二者都描述了一致性(Consistency)、可用性(Availability)和分區容忍性(Partition Tolerance)之間的約束與取捨。而PACELC理論則更進一步描述了即使在沒有Partition的場景下，也存在Latency和Consistency之間的取捨，從而爲分佈式系統的Consistency模型提供了一個更爲完整的理論依據。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"理論應用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要保證系統數據的高可用（high availability）那麼有個技術方案是採用數據冗餘備份的方式，那麼就涉及到複製數據，而進行分佈式系統的數據複製，就會出現在Consistency和Latency之間做個取捨的要求。舉個例子，如下圖所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7f/7f0a963fef0d1c16d289308c0dd57144.png","alt":"consistency-latency","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在強一致性複製場景下，需要三副本都下盤才能返回OK確認信息給client端，假設Master節點向 Slave 節點複製數據，時延的限制是 20ms，有時候，slave 2 硬盤或網絡出現故障，Master 往 Slave 複製數據的時延超過 了20ms，這個時候如果還一直等待 slave 2 返回結果再通知給client就會出現性能和時延抖動，而且這種抖動是經常會發生的長尾效應。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"依據PACELC理論，我們可以在 consistency和Latency之間做個取捨，比如 slave 2 節點的時延超過 20ms了，就不等待slave 2 返回，master 和 slave 1 返回結果給client即可，如果 slave 2 出現 超時的 次數超過 5次那麼就認爲 這個節點可能出現故障，打個故障標籤，進行後續的處理。採用這種方式可以消除寫時的長尾抖動，獲得更優雅的寫時性能曲線。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文遵循理論與實踐相結合的指導思想講述了CAP理論與PACELC理論。日拱一卒，功不唐捐，分享是最好的學習，與其跟隨不如創新，希望這個知識點對大家有用。另作者能力與認知都有限，”我講的，可能都是錯的“，歡迎大家拍磚留念。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"作者簡介"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常平，中科大碩，某AI芯片獨角獸深度學習首席軟件主管工程師，前EMC 大數據資深首席工程師，主要工作背景在深度學習、流式大數據、分佈式中間件以及Linux內核。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"參考文獻"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[1] "},{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/PACELC_theorem","title":null},"content":[{"type":"text","text":"https://en.wikipedia.org/wiki/PACELC_theorem"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[2] CAP理論與分佈式系統設計，S先生"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}