Redis進階篇二——持久化

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1. 簡介"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"持久化即將數據保存到可永久保存的存儲設備中。我們知道 Redis 爲了保證效率而把數據都緩存在內存中，但當我們重啓系統或關閉系統後，緩存在內存中的數據都會消失，所以爲了讓有些數據能保留下，Redis 持久化存儲就應運而生。Redis 提供了兩種方式進行持久化，一種是"},{"type":"text","marks":[{"type":"strong"}],"text":"RDB 持久化"},{"type":"text","text":"，另一種是 "},{"type":"text","marks":[{"type":"strong"}],"text":"AOF(append only file) 持久化"},{"type":"text","text":"，下面我們逐一介紹。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2. RDB 持久化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"RDB 是 Redis 默認的持久化機制，它的工作原理是把當前內存中的數據生成快照 (snapshot) 的方式寫入磁盤中的二進制文件中，默認的文件名爲 dump.rdb。恢復時將快照文件直接讀到內存中。RDB 有兩種觸發方式，分別是自動觸發和手動觸發。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1 自動觸發"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動觸發使用 save 相關配置觸發，比如 “save m n”，表示在 m 秒內數據庫存在 n 次修改時，自動觸發BGSAVE (BGSAVE 命令在手動觸發時會介紹)。Redis 默認配置如下："}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"save 900 1 # 在 900 秒內如果至少有 1 個 key 的值變化，則觸發\nsave 300 10 # 在 300 秒內如果至少有 10 個 key 的值變化，則觸發\nsave 60 10000 # 在 60 秒內如果至少有 10000 個 key 的值變化，則觸發"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當實際操作滿足配置的 save 形式時就會進行 RDB 持久化，將當前的數據快照保存。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2 手動觸發"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"手動觸發進行 RDB 持久化涉及到兩個 Redis 服務器命令："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"SAVE：執行一個同步保存操作，將當前 Redis 實例的所有數據快照以 RDB 文件的形式保存到磁盤中。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"BGSAVE：用於在後臺異步保存當前數據庫的數據到磁盤。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SAVE 命令由於是同步操作，因此會阻塞當前 Redis 服務器知道 RDB 持久化過程完成爲止，對於內存比較大的實例會造成長時間阻塞，不建議在線上環境使用。BGSAVE 命令會執行 fork 操作創建一個子進程，由子進程完成 RDB 持久化過程，完成後自動結束，阻塞只發生在 fork 過程，一般時間很短。基本上 Redis 內部的 RDB 操作都是採用 BGSAVE 命令。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.3 RDB 優缺點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"RDB 的優點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"RDB 文件是一個緊湊壓縮的二進制文件，它保存了 Redis 在某個時間點的數據，比較適合進行備份和災難恢復。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"生成 RDB 文件的過程是由父進程 fork 操作創建的一個子進程完成，父進程仍然可以接受其他命令請求，不用進行任何磁盤 IO 操作。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"由於 RDB 文件是二進制文件，因此在恢復數據集時速度更快。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"RDB 的缺點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"RDB 無法做到實時持久化/秒級持久化。因爲 BGSAVE 命令需要執行 fork 操作創建子進程，如果頻繁操作必然會佔用大量內存，執行成本過高，反而使性能降低。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"RDB 文件使用特定二進制格式保存，Redis 版本演進過程中有多個格式的 RDB 版本，容易出現版本不兼容問題。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"RDB 是隔一段時間進行備份，如果 Redis 在備份時出現意外宕機，那麼就會失去最後一次快照的數據。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3. AOF 持久化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"差別於通過保存數據庫中的鍵值對的 RDB 持久化方式，AOF 持久化是通過保存 Redis 服務器所執行的寫命令來記錄數據庫狀態，重啓時再重新執行 AOF 文件中的命令以完成數據恢復。AOF 的主要作用是解決了數據持久化的實時性，目前已經是 Redis 持久化的主流方式。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 使用 AOF"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis 中 AOF 是默認關閉的，使用前要將配置參數 appendonly 改爲 yes（5.3 中會涉及一些配置參數，配置文件是安裝目錄下的 redis.windows.conf，參數在 APPEND ONLY FILE 一欄）。AOF 文件的保存文件名通過參數 appendfilename 參數配置，默認是 appendonly.aof。AOF 持久化策略的選擇由 appendfsync 參數進行選擇，有下面幾種："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"no：不執行 fsync，由操作系統保證數據同步到磁盤，速度快但不安全。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"always：每次寫入都執行 fsync，雖然保證數據同步到磁盤，但是效率很低。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"everysec：默認配置，每秒執行一次 fsync，可能會丟失這 1 秒的數據，但兼顧了安全性和效率，最常用的選擇。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 AOF工作流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AOF 的工作流程操作：命令寫入 (append)、文件同步 (sync)、文件重寫 (rewrite)、重啓加載 (load)。首先，所有的寫入命令都會被追加到 AOF 緩衝區中，然後 AOF 緩衝區根據對應的持久化策略對磁盤進行文件同步操作。當 AOF 文件的大小超過所設定的重寫閾值時對 AOF 文件進行重寫。當需要重寫時，父進程會進行 fork 操作創建一個子進程，子進程帶有父進程的數據副本，由子進程完成重寫過程，在此期間父進程仍然可以處理其他命令。當我們重啓 Redis 服務器時，可以加載 AOF 文件進行數據恢復，流程如下："}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ec/ecead74dc219deb4892f04398f817e33.png","alt":"持久化","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"持久化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從上圖我們也可以得知，"},{"type":"text","marks":[{"type":"strong"}],"text":"在同時開啓了 RDB 和 AOF 的情況下，Redis 會優先 AOF 文件的加載"},{"type":"text","text":"。遇到異常時，可以使用修復命令 redis-check-aof--fix 進行修復。在上述流程中，我們有幾點需要注意與知曉的："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"爲什麼需要 AOF 緩衝"},{"type":"text","text":"：由於 Redis 本身是單線程工作的，如果每次都直接把寫入命令追加到 AOF 文件中，那麼此時的性能取決於磁盤的 IO 性能，會降低性能。先寫入 AOF 緩衝區中，我們還可以選擇緩衝區同步到磁盤的策略，進一步兼顧安全性和性能。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"爲什麼需要 AOF 重寫"},{"type":"text","text":"：由於 AOF 持久化是不斷將寫命令記錄到 AOF 文件中，隨着時間的推移，文件必然會越來越大，這樣會增加恢復時的壓力。除此之外，例如一個 key 表示粉絲數，增加粉絲的過程我們會使用大量自增命令，而實際上在恢復時我們只需要知道在這段時間內總共增加了多少即可。這個例子也正指明瞭重寫機制的工作原理："},{"type":"text","marks":[{"type":"strong"}],"text":"AOF 文件重寫並非是對原文件進行整理，而是直接讀取服務器中現有的鍵值對，然後用一條命令代替記錄中改鍵值對的多條命令操作，生成一個新的文件替換原文件"},{"type":"text","text":"。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AOF 重寫觸發機制"},{"type":"text","text":"：AOF 重寫也分自動觸發和手動觸發。"}]}]}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動觸發涉及兩個配置參數：一個是 "},{"type":"text","marks":[{"type":"strong"}],"text":"auto-aof-rewrite-percent"},{"type":"text","text":"，默認值是 100；另一個是 "},{"type":"text","marks":[{"type":"strong"}],"text":"auto-aof-rewrite-min-size"},{"type":"text","text":"，默認值是 64MB。按照默認配置，Redis 會記錄上次重寫時 AOF 文件大小，並當目前 AOF 文件是上一次重寫後大小的一倍且文件大於 64MB 時自動觸發。"}]}]}]},{"type":"paragraph","attrs":{"indent":2,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"auto-aof-rewrite-percent"},{"type":"text","text":"：當目前 AOF 文件大小超過上一次重寫的 AOF 文件大小的百分之幾時進行重寫。"},{"type":"text","marks":[{"type":"strong"}],"text":"auto-aof-rewrite-min-size"},{"type":"text","text":"：設置允許重寫的最小 AOF 文件大小，避免了文件很小卻滿足百分比要求的重寫情況。"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"手動觸發直接調用 BGREWRITEAOF 命令。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 AOF 優缺點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AOF 的優點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"提供了多種同步頻率，即使使用默認的同步頻率每秒同步一次，Redis 最多也就失去 1 秒的數據。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AOF 文件使用 Redis 命令追加的形式來構造，即使只能向 AOF 文件寫入命令的片段，也很容易使用redis-check-aof 工具修正。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"AOF 文件的格式可讀性較強，可以爲使用者提供更靈活的處理方式。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AOF 的缺點："}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"在擁有相同數據的情況下，AOF 文件通常會比 RDB 文件體積更大。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"當 Redis 負載較高的情況下，RDB 會比 AOD 具有更好的性能保證。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"RDB 使用快照的形式來持久化整個 Redis 數據，而 AOF 只是將每次執行的命令追加到 AOF 文件中，因此從理論上說，RDB 比 AOF 方式更健壯。官方文檔也指出，AOF 的確也存在一些 BUG，這些 BUG 在RDB 沒有存在。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4. 如何選擇"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"介紹了 Redis 持久化的兩種方式，那麼我們在實際中應該如何選擇呢？對於數據庫而言，數據是相當重要的，RDB 相比於 AOF 而言出現異常丟失數據可能會更嚴重，除此之外，選擇 RDB 是更好的，定時生成快照是常用的數據庫備份方式，並且 RDB 文件是二進制文件，在恢復數據集時速度更快，使用 RDB 方式也可以規避 AOF 方式中的一些隱藏 bug。但是在一般情況下，我們應該同時開啓兩種持久化方式，而不是單獨使用某一種持久化機制。由於通常情況下 AOF 方式保存的數據更具實時性且更完整，因此相比 RDB 文件，Redis 重啓時會優先使用 AOF 文件來恢復數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}