設計數據庫集羣讀寫分離並非易事

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"靈魂拷問："}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"解決數據庫讀寫瓶頸有哪些解決方案呢？"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這些方案解決了什麼問題呢？"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這些方案有那些優勢和劣勢呢？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個可以抵抗高併發流量系統的背後必定有一個高性能的數據庫集羣，就像每一個成功的男人背後總有一個強勢的女人一樣。數據庫集羣在部署模式上屬於分佈式，但是CAP原則卻不適用於分佈式數據庫，具體原因可見之前文章：、"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/t1nbzIBL7JDLQMbKTczDjw","title":null},"content":[{"type":"text","text":"晦澀難懂的CAP，是否完全正確？"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要想實現數據庫讀寫的高性能，目前針對寫操作的優化方案主要有分庫分表以及採用IO更優的設備來輔助，具體可見之前的文章："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/e68ID5y7_GZ7JqsbqKPVjg","title":null},"content":[{"type":"text","text":"做好分庫分表其實很難之一"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/OiwiHnSSQ8Z_fxI-yTDicQ","title":null},"content":[{"type":"text","text":"做好分庫分表其實很難之二"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/_NTTQENqt2Cf60cV7QNAFA","title":null},"content":[{"type":"text","text":"用NOSql給高併發系統加速"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分庫分表作爲一種普遍的解決方案，幾乎已經成爲面試者吹水的利劍，卻很少有人在意它所帶來的副作用。其實分庫分表是利用了分治的思路來解決數據庫的瓶頸問題，這種方案同時解決了併發讀和併發寫的瓶頸，利用數據分片的方式，以堆積硬件的方式來抵抗了高流量的衝擊，當然帶來了某些業務需要跨庫查詢，跨表join等問題，不過這些問題總能以別的解決方案來應對。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫讀寫分離是解決數據庫性能瓶頸的另外一個方案，和分庫分表方案相比較，他們有着本質的區別。分庫分表會把數據分散在多個庫表中，然後利用數據分片的規則來讀取和寫入數據，而讀寫分離是利用“冗餘”的方式來應對大流量的衝擊。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#讀寫分離原理","title":null}},{"type":"text","text":"讀寫分離原理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀寫分離的基本原理是將數據讀寫分散到不同的數據庫節點上，寫操作一般只發生在主節點，可以接受少量延遲的讀操作發生在從節點上"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ea/ea92ec25e829df28cef6ec76291a9199.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"至於讀寫分離的實現方式："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多臺數據庫服務器組件成集羣，並配置主從關係"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"主節點負責讀寫操作，從節點只負責讀操作"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"主節點通過數據複製機制，把數據從主節點同步到所有的從節點"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"業務方利用程序或者中間件把寫操作發送給主節點，將讀操作發送給從節點"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#讀寫分離優勢","title":null}},{"type":"text","text":"讀寫分離優勢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一般的系統都會滿足28原則，既：80%的操作是讀操作，20%的操作是寫操作。系統的讀操作佔比越大，讀寫分離的優勢就越發明顯，因爲讀操作可以通過簡單的增加數據庫從節點來解決，當然從節點的增加並不是毫無限制，當從節點到達一定數量的時候，必然會影響主從同步的效率，會降低主節點的性能，這個時候需要考慮一致性和可用性的平衡問題了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外一點，在很多業務中都會有一定的數據統計需求，單機數據庫的時候，這些統計需求執行的sql和業務sql混合在一起，在一定程度上會影響正常業務的運行，尤其是那些數據量比較大的業務場景。在做了讀寫分離的策略之後，統計業務完全可以獨佔一個從庫來進行統計，就算是比較耗時的操作，也不會影響正常的業務運行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫的讀寫分離方案在所有讀操作場景中，發揮了最大優勢"}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#讀寫分離劣勢","title":null}},{"type":"text","text":"讀寫分離劣勢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫讀寫分離有一個很多系統都會遇到的問題，那就是有些業務在寫操作成功之後需要實時的讀取到數據，可是數據從主節點同步到從節點是有一定時間延遲的，所以很多情況下業務方在從節點並不能實時的讀取到正確的數據，這種業務場景其實就是主節點也需要提供讀操作的典型場景，當然如果系統架設的有緩存模塊，在主節點寫操作成功之後可以同步更新緩存，以達到業務需要實時數據的要求。"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#路由機制","title":null}},{"type":"text","text":"路由機制"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀寫分離在寫操作上有着嚴格的要求，寫操作必鬚髮生在主節點上，因爲讀寫分離是基於中心化的思想來建立的集羣，中心化的思想要求主節點上的數據必須是最新且最全的。這就要求調用方必須要區分出主節點纔可以。"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"代碼封裝"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用程序代碼封裝讀寫分離邏輯需要在代碼中抽象出一個數據訪問層，在這一層中實現操作分離以及數據庫的連接管理等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9c/9c402947dfc3874c6fb39dac505c4cda.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用代碼封裝讀寫分離邏輯在落地上並非易事，需要經過很長時間的測試纔可以上生產環境。如果公司內部存在多個語言的開發團隊，每個語言可能都需要實現一次，開發量還是比較大的。但是在針對不同的業務中，可以做到定製化的需求，在落地過程中還需要考慮如果主從發生切換，代碼中必須要有類似選舉的過程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫中間件"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫中間件是指基於數據庫提供的SQL協議來開發的一套和具體業務無關的系統，它的作用也是實現操作分離和數據庫的連接管理等，它同樣也是對讀寫分離的一個抽象層，但是這個抽象層是基於數據庫協議的，對於業務的使用方來說，就像訪問單個數據庫一樣方便。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f9/f914f8fa25f84093f8660afccd78c7f5.jpeg","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#同步延遲","title":null}},{"type":"text","text":"同步延遲"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"任何分佈式的系統都逃不過一致性的問題。數據庫的主從架構也是一樣，發生在主節點的操作需要同步給每個從庫。像MySQL的主從複製是依賴於binlog的，主從複製就是將binlog中的數據從主庫複製到從庫上，一般這個過程都會採用異步的方式，因爲在網絡延遲的情況下，如果採用同步方式會大大降低主庫的可用性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在binlog的複製過程中，極低的概率會發生binlog還沒有來得及刷新到磁盤就出現磁盤壞掉或者down機的情況，最終的效果就是主從數據的不一致，但是這種不可抗拒的因素，一般是可以容忍的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還有一種現象，一般數據從主節點複製到從節點會開啓單線程模式，如果主庫產生新數據的速度大於同步的速度，那有可能會進一步加大主從同步的延遲時間，這個是否可以考慮開啓多線程或者利用緩存模塊來屏蔽同步延遲的問題呢？"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#主備方案","title":null}},{"type":"text","text":"主備方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"說到數據庫主從的架構部署方式，還有一種類似的方案：主備。主備是利用冗餘一個節點來做備用節點，但是這個節點在主節點正常運行的情況下，不會對外提供服務，做了一個真正的“備胎”。當主節點掛掉，備用節點會代替主節點的位置，併成爲主節點開始對外提供服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"主備方式可以利用簡單的類似keepalive機制來實現自動化，理論上不需要進行選舉操作。利用主備方式來實現數據庫高可用有哪些特點呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可用性是利用keepalive機制來保證的，這個切換過程對業務是透明的，業務方無需修改任何代碼"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀寫都在主庫上進行，很容易產生單點的瓶頸問題，由於沒有其他節點的數據同步過程，所以數據可以保證一致性"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"主備架構中，備庫只是單純的備份，整體的資源利用率50%，因爲備庫一直在被閒置"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"擴展性比較差，無法做到橫向擴展，但是可以利用分庫分表來解決擴展性問題"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一主一備或者一主多備方案在資源的利用率上很低，所以後來出現了多主的架構，多主架構是指，會存在多個主庫，每個主庫都提供讀寫功能，這就涉及到多個主庫之間數據同步的方式，雖然性能上要比一主要高，但是數據一致性上很難搞。所以很多互聯網公司並不推薦使用這種方案。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#寫在最後","title":null}},{"type":"text","text":"寫在最後"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫的擴展由於其屬於有狀態的範疇，所以比無狀態的網站或者服務要困難很多。現在主流的落地方案也都是基於“分”的策略，分庫分表方案和主從讀寫分離方案是兩種最常用的擴展方式，在很多情況下，二者是結合起來使用的，即：在分庫分表的情況下，每個節點採用主從讀寫分離的方式，這也是目前比較主流的方式了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"更多精彩文章"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342955119549267969&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"分佈式大併發系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342959003139227648&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"架構設計系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342962375443529728&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"趣學算法和數據結構系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342964237798391808&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"設計模式系列"}]}]}]}]}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f8/f8af5984765a267892bf1a1272272625.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}