Mysql學習筆記：分庫分表(sharding)

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當數據庫性能出現瓶頸時就需要通過擴展來提升性能，對於擴展性來說要麼加強機器本身的性能，要麼把任務分發到不同的機器上。對於數據庫來說通過強悍的機器解決成本是很大的，如Oracle。通過多個廉價的機器實現水平擴展是現代的主流解決方案，如Mysql。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫水平擴展的核心是把數據拆分成不同的單元並放在不同的獨立的實例上，這樣就做到了負載均衡。拆分分爲邏輯和物理拆分，邏輯拆分是對物理上不可分割的實例進行邏輯上的分割，物理拆分是拆分成多個獨立的實例："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"邏輯拆分"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分區（Partition）"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分表"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"物理拆分"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀寫分離"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"垂直拆分（分庫）"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"水平拆分（分表）"}]}]}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"1.邏輯拆分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1.1 分區"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我理解的邏輯分區：舉個例子，操作系統中的分區，是將硬盤根據大小進行邏輯分區，就是我們看到的C、D、E、F盤，邏輯分區還是在同一個操作系統中。數據庫產品的Partition分區也是一樣的道理，將數據進行邏輯分區，對數據劃分界限。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySql 支持Range，List，Hash，Key。最常用的是Range。"},{"type":"text","marks":[{"type":"italic"},{"type":"underline"}],"text":"注意不同的版本對分區類型的支持有些不同！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Range：範圍"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"CREATE TABLE employees (\n id INT NOT NULL,\n fname VARCHAR(30),\n lname VARCHAR(30),\n hired DATE NOT NULL DEFAULT '1970-01-01',\n separated DATE NOT NULL DEFAULT '9999-12-31',\n job_code INT NOT NULL,\n store_id INT NOT NULL\n)\nPARTITION BY RANGE (store_id) (\n PARTITION p0 VALUES LESS THAN (6),\n PARTITION p1 VALUES LESS THAN (11),\n PARTITION p2 VALUES LESS THAN (16),\n PARTITION p3 VALUES LESS THAN (21)\n);"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"LIST：列表"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"CREATE TABLE employees (\n id INT NOT NULL,\n fname VARCHAR(30),\n lname VARCHAR(30),\n hired DATE NOT NULL DEFAULT '1970-01-01',\n separated DATE NOT NULL DEFAULT '9999-12-31',\n job_code INT,\n store_id INT\n)\nPARTITION BY LIST(store_id) (\n PARTITION pNorth VALUES IN (3,5,6,9,17),\n PARTITION pEast VALUES IN (1,2,10,11,19,20),\n PARTITION pWest VALUES IN (4,12,13,14,18),\n PARTITION pCentral VALUES IN (7,8,15,16)\n);"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Key：鍵"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"CREATE TABLE k1 (\n id INT NOT NULL,\n name VARCHAR(20),\n UNIQUE KEY (id)\n)\nPARTITION BY KEY()\nPARTITIONS 2;"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HASH：哈希"}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"CREATE TABLE employees (\n id INT NOT NULL,\n fname VARCHAR(30),\n lname VARCHAR(30),\n hired DATE NOT NULL DEFAULT '1970-01-01',\n separated DATE NOT NULL DEFAULT '9999-12-31',\n job_code INT,\n store_id INT\n)\nPARTITION BY HASH( YEAR(hired) )\nPARTITIONS 4;"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#454545","name":"user"}},{"type":"strong"}],"text":"例子："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據：新聞表，2010開始記錄，假設10年到15年每年的數據爲200W，總數1000W；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"條件：查詢15年7月所有的新聞數據；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"未分區：需要把表遍歷，1000W條數據，查詢性能就不用說了；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分區：按照年份分區，當要查詢15年數據，只會遍歷15年的數據200W條，加快了查詢；"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1.2 分表"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當單表數據行數超過一定量級時，讀/寫 會變慢，查詢需要檢索更多數據，DML操作需要更多時間創建/更新索引；我們可以通過把這些數據分散到多個表中來提高效率，這樣只涉及到部分數據而不是所有，最常用的分表算法是"},{"type":"link","attrs":{"href":"https://www.yuque.com/madiao/kb/qmpxxu#TPo5x","title":null},"content":[{"type":"text","text":"哈希算法"}]},{"type":"text","text":"。哈希函數使用除留餘數法，即取餘的方式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"建立所需要的N個表，表名：user_0 ... user_N-1，通過對ID取餘運算直接路由到所在的表"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_0： 5%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_1： 1%5 / 6%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_2： 2%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_3： 3%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_4： 4%5"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"小結：邏輯分區是數據庫提供的功能，不用對應用和業務做任何改變就能實現。哈希分表實現簡單，只需要修改少量代碼就能實現。對單表進行分表後，能夠大大提高我們讀寫的效率。"}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"2.物理拆分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2.1 讀寫分離（主從複製）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"讀寫分離的核心是把讀/寫操作路由的不同實例上，實例之間要的數據要保障一致（通過複製實現），路由可以自己識別 Insert/Update/Delete/Selete 做路由，也可以使用代理(mysql proxy)或中間件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一般站點的讀操作比寫操作更加密集，查詢量暴增的時候單臺服務器無法處理這麼多讀操作，我們需要增加額外的服務器來支撐，使用主從方式，主做寫操作，從做讀操作，通過主從複製達到數據一致性，這樣讀操作壓力會被分散。mysql使用單線程把主機數據複製到從機上實現數據一致性，所以需要對主從進行配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/78/78ce5d236b060e6d397228f0f081c722.png","alt":null,"title":"image.png","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上面的主從架構中，如果從庫有很多個可能會出現複製延遲過大現象，原因是因爲mysql複製需要在slave和master建立長連接，並且master需要開啓binlog dump線程進行數據推送，過多的slave會導致複製延遲過大。可以"},{"type":"text","marks":[{"type":"strong"}],"text":"增加複製源"},{"type":"text","text":"和開啓"},{"type":"text","marks":[{"type":"strong"}],"text":"半同步複製"},{"type":"text","text":"解決"},{"type":"text","marks":[{"type":"strong"}],"text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"1.增加複製源"},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/00/0080c2d21061a7ab11b794e48cca9fc4.png","alt":null,"title":"image.png","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"2.開啓半同步複製："},{"type":"text","text":"主庫提交事務時，將事件寫入它的二進制日誌，而從庫在準備就緒時請求它們。主庫無需等待從庫的ACK回覆，直接提交事務並返回客戶端。異步複製不確保所有事件都能到達從庫，無法保證數據完整性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2.2 垂直拆分（分庫）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀寫分離不能解決寫操作頻繁帶來的性能瓶頸，比如主庫寫操作佔80%，這時需要把寫操作拆分到獨立的實例上，"},{"type":"text","marks":[{"type":"strong"}],"text":"垂直拆分"},{"type":"text","text":"是按照業務相關度把數據拆分到不同的DB上，這樣寫操作自然就被拆分開來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c1/c11457316aacd088a8b8d3bdfe1bd2a9.png","alt":"image.png","title":"image.png","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"拆分了之後還可以繼續做讀寫分離進一步提升性能，但"},{"type":"text","marks":[{"type":"strong"}],"text":"垂直拆分也帶來了問題，原本在一個事務中的數據操作，在拆分之後就無法在同一個事務中完成，這使得我們業務應用需要額外的成本去解決，如通過引入分佈式事務 或 最終一致來解決。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2.3 水平拆分（分表）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對數據庫做了垂直切分和讀寫分離可以解決大部分站點的問題，但是在體量巨大的應用中主數據庫寫操作壓力依然會達到極限，這時需要對錶進行水平拆分並分佈在不同機器上面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/31/3106fc5825c52afa266649762d3f3246.png","alt":null,"title":"image.png","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"水平拆分最簡單的方式就是用哈希算法，一個表只能根據一個字段sharding。下面列舉了一些常用的拆分方法："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"1.簡單hash算法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"建立所需要的N個表，表名：user_0 ... user_N-1，通過對ID取餘運算直接路由到所在的表："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_0： 5%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_1： 1%5 / 6%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_2： 2%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_3： 3%5"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"user_4： 4%5"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"優點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢分片位置的時間複雜度爲O(1)，簡單有效。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"缺點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"動態擴容有侷限"},{"type":"text","text":"：當容量不足需要增加分片數量來擴容，哈希值會發生改變，涉及全量數據遷移。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"熱點數據集中"},{"type":"text","text":"：活躍用戶分到了同一個片上，這個實例壓力非常大可能會過載。"}]}]}]},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"2.一致性hash算法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在擴容時簡單hash算法需要全量數據遷移成本和風險很高，一致性hash算法對該算法進行了優化，通過對固定值2^32-1進行取餘保證hash結果不變，再通過範圍把環拆分成N份，增加節點時隻影響新節點到逆時針第一個節點之間的數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a2/a275105cf39dbf80f58654e7d8945b5b.png","alt":null,"title":"image.png","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"整體擴容"},{"type":"text","text":"：如果分片數量不足需要擴容，因爲要保證數據分佈均勻，所以受影響的節點會"},{"type":"text","marks":[{"type":"strong"}],"text":"佔總量的一半。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d4/d41fcabfe45df727075e004fe5932b89.png","alt":"image.png","title":"image.png","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"局部擴容："},{"type":"text","text":"一致性hash通過在局部增加節點實現靈活擴容，而不必每次都翻倍擴容，可以對"},{"type":"text","marks":[{"type":"strong"}],"text":"熱點數據表進行再拆分，"},{"type":"text","text":"隻影響新節點到逆時針第一個節點之間的數據，但是需要額外再維護映射表保證其他節點還映射到舊錶。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/67/67b74379a7cc64554a1499a3ce523fae.png","alt":"image.png","title":"image.png","style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"優點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以靈活選擇局部還是整體擴容，局部擴容可以對某個熱點數據的節點再拆分而不影響其他節點。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"缺點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在節點過多的情況下查詢效率較低，表映射實現複雜。"}]}]}]},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"3.動態映射"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"熱點數據集中"},{"type":"text","text":"可能是由於某個ID產生的數據過多造成的，通過配置指定到具體的分片上可以過熱問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"優點：可以做局部擴容解決熱點數據問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"缺點：實現比較複雜，每次都需要查詢獲取對應分片性能比簡答hash差，會影響查詢效率。"}]},{"type":"horizontalrule"},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"2.4 拆分帶來的問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"物理拆分帶來好處的同時也帶來的一些問題："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"跨庫事務"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過分佈式事務 或 最終一致解決"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"跨庫Join"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把Join操作拆分成多次查詢並在應用中做聚合"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用搜索引擎做數據聚合和查詢"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用CQRS做數據聚合"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"跨表分頁和排序："}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由中間件去所有分片聚合數據，再做分頁和排序"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來講一下CQRS是怎麼做的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3d/3dbbe2f3cb915e74294df2a2c9f65108.png","alt":null,"title":"image.png","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CQRS是對應用做讀寫職責分離，每次寫操作都會以類似日誌的形式記錄在Event Store中，並不是直接修改字段值到期望值，再由Event Bus把事件同步到讀服務，讀服務對讀庫數據進行修改，所有查詢都會走讀服務。在該架構模式中讀服務可以把想要的業務數據聚合到讀庫中，其實就是通過冗餘數據的方式避免應用去多庫中查詢和聚合數據，以空間換時間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}