騰訊二面：MySQL的半同步是什麼？

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"前言","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"年後在進行騰訊二面的時候，寫完算法的後問的第一個問題就是，MySQL的半同步是什麼？我當時直接懵了，我以爲是問的MySQL的兩階段提交的問題呢？結果確認了一下後不是兩階段提交，然後面試官看我連問的是啥都不知道，就直接跳過這個問題，直接聊下一個問題了。所以這次總結一下這部分的知識內容，文字內容比較多，可能會有些枯燥，但對於這方面感興趣的人來說還是比較有意思的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"MySQL的主從複製","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"我們的一般在大規模的項目上，都是使用MySQL的複製功能來創建MySQL的主從集羣的。主要是可以通過爲服務器配置一個或多個備庫的方式來進行數據同步。複製的功能不僅有利於構建 高性能應用，同時也是高可用、可擴展性、災難恢復、備份以及數據倉庫等工作的基礎 。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"說得通俗一點，通過MySQL的主從複製來實現讀寫分離，相比單點數據庫又讀又寫來說，提升了業務系統性能，優化了用戶體驗。另外通過主從複製實現了數據庫的高可用，當主節點MySQL掛了的時候，可以用從庫來頂上。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"MySQL支持的複製方式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL支持三種複製方式：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"基於語句的複製（也稱爲邏輯複製）主要是指，在主數據庫上執行的SQL語句，在從數據庫上會重複執行一遍 。MySQL默認採用的就是這種複製，效率比較高。但是也是有一定的問題的，如果SQL中使用uuid()、rand()等函數，那麼複製到從庫的數據就會有偏差。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"基於行的複製，指將更新處理後的數據複製到從數據庫，而不是執行一邊語句 。從MySQL5.1的版本才被支持。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"混合複製，默認採用語句複製，當發現語句不能進行精準複製數據時（例如語句中含有uuid()、rand()等函數），採用基於行的複製 。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"主從複製原理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL的複製原理概述上來講大體可以分爲這三步","attrs":{}}]},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"在主庫上把數據更改，記錄到二進制日誌（Binary Log）中。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"從庫將主庫上的日誌複製到自己的中繼日誌（Relay Log）中。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"備庫讀取中繼日誌中的事件，將其重放到備庫數據之上。","attrs":{}}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"主要過程如下圖：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f1/f11b77f9839f5f953e24b0acc4eeab40.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"下面來詳細說一下複製的這三步：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"第一步：是在主庫上記錄二進制日誌，首先主庫要開啓binlog日誌記錄功能，並授權Slave從庫可以訪問的權限。這裏需要注意的一點就是binlog的日誌裏的順序是按照事務提交的順序來記錄的而非每條語句的執行順序。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"第二步：從庫將binLog複製到其本地的RelayLog中。首先從庫會啓動一個工作線程，稱爲I/O線程，I/O線程跟主庫建立一個普通的客戶端連接，然後主庫上啓動一個特殊的二進制轉儲（binlog dump）線程，此轉儲線程會讀取binlog中的事件。當追趕上主庫後，會進行休眠，直到主庫通知有新的更新語句時才繼續被喚醒。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"這樣通過從庫上的I/O線程和主庫上的binlog dump線程，就將binlog數據傳輸到從庫上的relaylog中了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"第三步：從庫中啓動一個SQL線程，從relaylog中讀取事件並在備庫中執行，從而實現備庫數據的更新。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"這種複製架構實現了獲取事件和重放事件的解耦，運行I/O線程能夠獨立於SQL線程之外工作。但是這種架構也限制複製的過程，最重要的一點是在主庫上併發運行的查詢在備庫中只能串行化執行，因爲只有一個SQL線程來重放中繼日誌中的事件。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"說到這個主從複製的串行化執行的問題，我就想到了一個之前在工作中遇到的一個問題，就是有這麼一個業務場景，我們有一個操作是初始化一批數據，數據是從一個外部系統的接口中獲取的，然後我是通過線程池裏的多個線程並行從外部系統的接口中獲取數據，每個線程獲取到數據後，直接插入到數據庫中。然後在數據全部入庫完成後，然後去執行批量查詢，將剛插入到數據庫中的數據查詢出來，放到ElasticSearch中。結果每次放入到ES中的數據總是不完整，後來研究了半天都不行，最終是讓查詢也走的主庫才解決的問題。當時不知道是MySQL主從複製的串行化從而導致的這個問題。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"MySQL主從複製模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL的主從複製其實是支持， ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"異步複製","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 、 ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"半同步複製","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 、 ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"GTID複製","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 等多種複製模式的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"異步模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL的默認複製模式就是異步模式，主要是 指MySQL的主服務器上的I/O線程，將數據寫到binlong中就直接返回給客戶端數據更新成功，不考慮數據是否傳輸到從服務器，以及是否寫入到relaylog中 。在這種模式下，複製數據其實是有風險的，一旦數據只寫到了主庫的binlog中還沒來得及同步到從庫時，就會造成數據的丟失。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"但是這種模式確也是效率最高的，因爲變更數據的功能都只是在主庫中完成就可以了，從庫複製數據不會影響到主庫的寫數據操作。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8e/8ec8cba155110164aa1532d4d9af4d57.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"上面我也說了，這種異步複製模式雖然效率高，但是數據丟失的風險很大，所以就有了後面要介紹的半同步複製模式。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"半同步模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL從 ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"5.5","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 版本開始通過以插件的形式開始支持半同步的主從複製模式。什麼是半同步主從複製模式呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"這裏通過對比的方式來說明一下：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"異步複製模式","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：上面我們已經介紹了，異步複製模式，主庫在執行完客戶端提交的事務後，只要將執行邏輯寫入到binlog後，就立即返回給客戶端，並不關心從庫是否執行成功，這樣就會有一個隱患，就是在主庫執行的binlog還沒同步到從庫時，主庫掛了，這個時候從庫就就會被強行提升爲主庫，這個時候就有可能造成數據丟失。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"同步複製模式","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：當主庫執行完客戶端提交的事務後，需要等到所有從庫也都執行完這一事務後，才返回給客戶端執行成功。因爲要等到所有從庫都執行完，執行過程中會被阻塞，等待返回結果，所以性能上會有很嚴重的影響。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"半同步複製模式","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：半同步複製模式，可以說是介於異步和同步之間的一種複製模式，主庫在執行完客戶端提交的事務後，要等待至少一個從庫接收到binlog並將數據寫入到relay log中才返回給客戶端成功結果。半同步複製模式，比異步模式提高了數據的可用性，但是也產生了一定的性能延遲，最少要一個TCP/IP連接的往返時間。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"半同步複製模式，可以很明確的知道，在一個事務提交成功之後，此事務至少會存在於兩個地方一個是主庫一個是從庫中的某一個。 主要原理是，在master的dump線程去通知從庫時，增加了一個ACK機制，也就是會確認從庫是否收到事務的標誌碼，master的dump線程不但要發送binlog到從庫，還有負責接收slave的ACK。當出現異常時，Slave沒有ACK事務，那麼將自動降級爲異步複製，直到異常修復後再自動變爲半同步複製","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL半同步複製的流程如下：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/de/de7b5f23d1117162df28c007f9dd95ae.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"半同步複製的隱患","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"半同步複製模式也存在一定的數據風險，當事務在主庫提交完後等待從庫ACK的過程中，如果Master宕機了，這個時候就會有兩種情況的問題。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"事務還沒發送到Slave上","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：若事務還沒發送Slave上，客戶端在收到失敗結果後，會重新提交事務，因爲重新提交的事務是在新的Master上執行的，所以會執行成功，後面若是之前的Master恢復後，會以Slave的身份加入到集羣中，這個時候，之前的事務就會被執行兩次，第一次是之前此臺機器作爲Master的時候執行的，第二次是做爲Slave後從主庫中同步過來的。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"事務已經同步到Slave上","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：因爲事務已經同步到Slave了，所以當客戶端收到失敗結果後，再次提交事務，你那麼此事務就會在當前Slave機器上執行兩次。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"爲了解決上面的隱患，MySQL從5.7版本開始，增加了一種新的半同步方式。新的半同步方式的執行過程是將“ ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"Storage Commit","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ”這一步移動到了“ ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"Write Slave dump","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ”後面。這樣保證了 ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"只有Slave的事務ACK後，才提交主庫事務","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 。MySQL 5.7.2版本新增了一個參數來進行配置： rpl_semi_sync_master_wait_point ，此參數有兩個值可配置：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"AFTER_SYNC","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：參數值爲AFTER_SYNC時，代表採用的是新的半同步複製方式。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"AFTER_COMMIT","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ：代表採用的是之前的舊方式的半同步複製模式。","attrs":{}}]}]}],"attrs":{}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/52/52415fb25e93aaeb30f7a097f11dba58.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL從5.7.2版本開始，默認的半同步複製方式就是 AFTER_SYNC 方式了，但是方案不是萬能的，因爲 AFTER_SYNC 方式是在事務同步到Slave後才提交主庫的事務的，若是當主庫等待Slave同步成功的過程中Master掛了，這個Master事務提交就失敗了，客戶端也收到了事務執行失敗的結果了，但是Slave上已經將binLog的內容寫到Relay Log裏了，這個時候，Slave數據就會多了，但是多了數據一般問題不算嚴重，多了總比少了好。MySQL，在沒辦法解決分佈式數據一致性問題的情況下，它能保證的是不丟數據，多了數據總比丟數據要好。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"這裏說幾個的半同步複製模式的參數：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"mysql> show variables like '%Rpl%';\n+-------------------------------------------+------------+\n| Variable_name | Value |\n+-------------------------------------------+------------+\n| rpl_semi_sync_master_enabled | ON |\n| rpl_semi_sync_master_timeout | 10000 |\n| rpl_semi_sync_master_trace_level | 32 |\n| rpl_semi_sync_master_wait_for_slave_count | 1 |\n| rpl_semi_sync_master_wait_no_slave | ON |\n| rpl_semi_sync_master_wait_point | AFTER_SYNC |\n| rpl_stop_slave_timeout | 31536000 |\n+-------------------------------------------+------------+","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"-- 半同步複製模式開關\nrpl_semi_sync_master_enabled\n-- 半同步複製，超時時間，單位毫秒，當超過此時間後，自動切換爲異步複製模式 \nrpl_semi_sync_master_timeout\n-- MySQL 5.7.3引入的，該變量設置主需要等待多少個slave應答，才能返回給客戶端，默認爲1。\nrpl_semi_sync_master_wait_for_slave_count\n-- 此值代表當前集羣中的slave數量是否還能夠滿足當前配置的半同步複製模式，默認爲ON，當不滿足半同步複製模式後，全部Slave切換到異步複製，此值也會變爲OFF\nrpl_semi_sync_master_wait_no_slave\n-- 代表半同步複製提交事務的方式，5.7.2之後，默認爲AFTER_SYNC\nrpl_semi_sync_master_wait_point","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"GTID模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL從5.6版本開始推出了GTID複製模式，GTID即全局事務ID (global transaction identifier)的簡稱，GTID是由UUID+TransactionId組成的，UUID是單個MySQL實例的唯一標識，在第一次啓動MySQL實例時會自動生成一個server_uuid, 並且默認寫入到數據目錄下的auto.cnf(mysql/data/auto.cnf)文件裏。TransactionId是該MySQL上執行事務的數量，隨着事務數量增加而遞增。這樣保證了 ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"GTID在一組複製中，全局唯一","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" 。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"這樣通過GTID可以清晰地看到，當前事務是從哪個實例上提交的，提交的第多少個事務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"來看一個GTID的具體形式：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"mysql> show master status;\n+-----------+----------+--------------+------------------+-------------------------------------------+\n| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |\n+-----------+----------+--------------+------------------+-------------------------------------------+\n| on.000003 | 187 | | | 76147e28-8086-4f8c-9f98-1cf33d92978d:1-322|\n+-----------+----------+--------------+------------------+-------------------------------------------+\n1 row in set (0.00 sec)","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"GTID：76147e28-8086-4f8c-9f98-1cf33d92978d:1-322","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"UUID：76147e28-8086-4f8c-9f98-1cf33d92978d","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"TransactionId:1-322","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"GTID的工作原理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"由於GTID在一組主從複製集羣中的唯一性，從而保證了每個GTID的事務只在一個MySQL上執行一次。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"那麼是怎麼實現這種機制的呢？GTID的原理又是什麼樣的呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"當從服務器連接主服務器時，把自己執行過的GTID（ ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"Executed_Gtid_Set: 即已經執行的事務編碼","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ）以及獲取到GTID（ ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong","attrs":{}}],"text":"Retrieved_Gtid_Set: 即從庫已經接收到主庫的事務編號","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ）都傳給主服務器。主服務器會從服務器缺少的GTID以及對應的transactionID都發送給從服務器，讓從服務器補全數據。當主服務器宕機時，會找出同步數據最成功的那臺conf服務器，直接將它提升爲主服務器。若是強制要求某一臺不是同步最成功的一臺從服務器爲主，會先通過change命令到最成功的那臺服務器，將GTID進行補全，然後再把強制要求的那臺機器提升爲主。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"主要數據同步機制可以分爲這幾步：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"master更新數據時，在事務前生產GTID，一同記錄到binlog中。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"slave端的i/o線程，將變更的binlog寫入到relay log中。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"sql線程從relay log中獲取GTID，然後對比Slave端的binlog是否有記錄。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"如果有記錄，說明該GTID的事務已經執行，slave會忽略該GTID。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"如果沒有記錄，Slave會從relay log中執行該GTID事務，並記錄到binlog。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"在解析過程中，判斷是否有主鍵，如果沒有主鍵就使用二級索引，再沒有二級索引就掃描全表。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"初始結構如下圖","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/61/61c2bdcfe2a87307b65c1a3c037a3b5d.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"當Master出現宕機後，就會演變成下圖。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b2/b23c7d0928a7fb874d101d8b1e9bafa3.jpeg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"通過上圖我們可以看出來，當Master掛掉後，Slave-1執行完了Master的事務，Slave-2延時一些，所以沒有執行完Master的事務，這個時候提升Slave-1爲主，Slave-2連接了新主（Slave-1）後，將最新的GTID傳給新主，然後Slave-1就從這個GTID的下一個GTID開始發送事務給Slave-2。 這種自我尋找複製位置的模式減少事務丟失的可能性以及故障恢復的時間。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"GTID的優劣勢","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"通過上面的分析我們可以得出GTID的優勢是：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"每一個事務對應一個執行ID，一個GTID在一個服務器上只會執行一次;","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"GTID是用來代替傳統複製的方法，GTID複製與普通複製模式的最大不同就是不需要指定二進制文件名和位置;","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"減少手工干預和降低服務故障時間，當主機掛了之後通過軟件從衆多的備機中提升一臺備機爲主機;","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"GTID的缺點也很明顯：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"首先不支持非事務的存儲引擎；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"不支持create table ... select 語句複製(主庫直接報錯);(原理: 會生成兩個sql, 一個是DDL創建表SQL, 一個是insert into 插入數據的sql; 由於DDL會導致自動提交, 所以這個sql至少需要兩個GTID, 但是GTID模式下, 只能給這個sql生成一個GTID)","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"不允許一個SQL同時更新一個事務引擎表和非事務引擎表;","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"在一個MySQL複製羣組中，要求全部開啓GTID或關閉GTID。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"開啓GTID需要重啓 (mysql5.7除外);","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"開啓GTID後，就不再使用原來的傳統複製方式（不像半同步複製，半同步複製失敗後，可以降級到異步複製）;","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"對於create temporary table 和 drop temporary table語句不支持;","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"不支持sql_slave_skip_counter;","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"其實GTID的這部分內容挺多的，如果有想深入研究的可以去看看這篇文章。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"最後說幾個開啓GTID的必備條件：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL 5.6 版本，在my.cnf文件中添加:","attrs":{}}]}]}],"attrs":{}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"gtid_mode=on (必選) #開啓gtid功能\nlog_bin=log-bin=mysql-bin (必選) #開啓binlog二進制日誌功能\nlog-slave-updates=1 (必選) #也可以將1寫爲on\nenforce-gtid-consistency=1 (必選) #也可以將1寫爲on","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"MySQL 5.7或更高版本，在my.cnf文件中添加:","attrs":{}}]}]}],"attrs":{}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"gtid_mode=on (必選)\nenforce-gtid-consistency=1 （必選）\nlog_bin=mysql-bin （可選） #高可用切換，最好開啓該功能\nlog-slave-updates=1 （可選） #高可用切換，最好打開該功能","attrs":{}}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接： http://www.cnblogs.com/jimoer/p/14673646.html","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果覺得本文對你有幫助，可以關注一下我公衆號，回覆關鍵字【面試】即可得到一份Java核心知識點整理與一份面試大禮包！另有更多技術乾貨文章以及相關資料共享，大家一起學習進步！","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/20/20d8e05b028b03ea8419f639ab2bae6f.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}