面試官：Redis 主從複製時網絡開小差了怎麼整？

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這期我們繼續回到之前的 Redis 話題。今天主要講的是主從複製數據一致性相關以及面對網絡中斷如何進行數據同步的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不 BB 了，直接上鍾吧！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/75/7577348325e0b5b027e31d5ce22c28ae.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖注：思維導圖","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"主從模式配置","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於 Redis 主從大家可能並不陌生，但是配置的話日常工作中並不會經常操作。在這裏簡單介紹下主從的相關配置。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1、主從模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/92/92a49c7c6367735dae8dd90eef365986.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis 中設置主從的方式很簡單，通常有兩種：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過在配置文件 redis.conf 中設置 slaveof 方式（永久）；","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"直接在客戶端執行 slaveof ip port 的方式（臨時）；","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2、主-從-從模式","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6b/6bf3c41ace003ad79112d83b5cc3c35e.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於主-從-從的模式來說，配置也與上邊的操作類似，在這裏就不多贅述了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"主從一致性原理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"瞭解了主從配置後，下面就要進入正題了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在主從中，通常的操作是主庫用來寫入數據，從庫用來讀取數據。這樣的好處是避免了所有的請求壓力都打在了主庫上，同時系統的伸縮性也得到了很大的提升。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是問題就來了，讀從庫時的數據要與主庫保持一致，那就需要主庫的數據在寫入後同步到從庫中。如何保持主庫與從庫的數據一致性，當有多個從庫時，又如何做到呢？","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1、全量複製","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這是第一次同步時所發生的傳遞關係。看名字就知道，主庫第一次就毫無保留的把所有數據都傳遞給了從庫。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們先來看下它們是如何發生第一次關係的（就知道你會想歪）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a1066295a25f1edcdcc9089724c6624a.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖中的同步流程已經很清晰了，總共分爲三部分：","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"（1）主從節點建立聯繫","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當從節點與主節點第一次建立聯繫時，從節點會向主節點發送 psync 命令，表示要進行數據同步。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如你看到的 psync 命令後會帶有兩個參數：一個是 runID，一個是偏移量 offset。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"runID：","attrs":{}},{"type":"text","text":"每個Redis實例生成的隨機且唯一的ID，在這裏表示的是主節點的ID。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"offset：","attrs":{}},{"type":"text","text":"複製偏移量。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在圖中第一次複製時因爲不知道主庫ID和偏移量，因此用“？”和“-1”分別來表示runID 和 offset。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當主節點接收到 psync 命令後，會使用 FULLSYNC命令向從節點發送 runID 及offset 兩個參數。從節點將其信息保存下來。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"到這裏關係算是建立了下來。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"（2）主節點同步RDB文件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"RDB文件，這是一個老面孔了，持久化時會用到的二進制文件。在這裏起着主從數據同步的作用，也就是說主從同步是依賴 RDB 文件來實現的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從節點接收到 RDB 文件後，在本地完成數據加載，算是完成了主從同步。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"到這裏你有沒有發現什麼問題？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們回想下 RDB 文件是如何生成的。在持久化那篇文章裏，我們介紹過，父進程 fork 了一個子進程來進行生成 RDB 文件。父進程並不阻塞接收處理客戶端的命令。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5e/5e5e32ecd8c3e1a04faa3b18784fc546.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼問題就產生了，當主節點把 RDB 文件發送給從節點時，主節點同時接收的命令又該如何來處理？ ","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"（3）主節點同步緩衝區命令","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這一步就是來解決 RDB 文件生成後，父進程又接收到寫命令同步的問題的。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了保證主從節點數據的一致性，主節點中會使用緩衝區來記錄 RDB 文件生成後接收到的寫操作命令。在 RDB 文件發送完成後會把緩衝區的命令發送給從節點來執行。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"到這裏，主從節點的數據同步算是完成了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2、級聯操作","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們再來回顧下整個同步流程，從建立關係，生成 RDB 文件，傳輸給從節點到最後緩衝區命令發送給從節點。這是一個從節點與主節點同步的完整流程。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼我們再來思考：當有多個從節點，也就是一主多從時，第一次連接時都要進行全量複製。但是在生成 RDB 文件時，父進程 fork 子進程時可能會出現阻塞，同時在傳輸 RDB 文件時也會佔用帶寬，浪費資源。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種情況我們該如何來解決呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不知道你對文章開頭的 主-從-從模式是否還有印象。通過對從節點再建立從節點。同步數據時從級聯的從節點上進行同步，從而就減輕了主節點的壓力。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"網絡開小差了","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的流程我們已經知道了正常情況下主從節點的複製過程了，但是當網絡中斷導致主從連接失敗等異常情況下，主從同步又是如何來進行的？ ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這裏要提到一個","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"增量複製","attrs":{}},{"type":"text","text":"的名詞，與全量複製不同的是，它是根據主從節點的偏移量來進行數據同步的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"什麼意思呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還記得在全量複製裏我們所提到過的緩衝區嗎？就是用來存儲生成 RDB 文件後的寫命令的，這裏我們稱爲緩衝區A。主從節點斷開連接後，除了會將後續接收到的寫命令寫入緩衝區A的同時，還會寫入到另一個緩衝區B裏。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在緩衝區B裏，主從節點分別會維護一個偏移量 offset。剛開始時，主節點的寫位置與從節點的讀位置在同一起點，隨着主節點的不斷寫入，偏移量也會逐漸增大。同樣地，從節點複製完後偏移量也在不斷增加。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7b/7bdfbc435d6b03049c543ee8b74b5d6a.png","alt":"","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當網絡斷開連接時，從節點不再進行同步，此時主節點由於不斷接收新的寫操作的偏移量會大於從節點的偏移量。當連接恢復時，從節點向主節點發送帶有偏移量的psync 命令，主節點根據偏移量來進行比較，只需將未同步寫命令同步給從節點即可。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"主從一致性原理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從節點第一次進行連接時，主節點會生成 RDB 文件進行全量複製，同時將新寫入的命令存儲進緩衝區，發送給從節點，從而保證數據一致性；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了減少數據同步給主節點帶來的壓力，可以通過從節點級聯的方式進行同步。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"網絡斷連重新連接後，主從節點通過分別維護的偏移量來同步寫命令。 ","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"視頻推薦：https://www.bilibili.com/video/BV1Wv411B7tX","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}