aof,rdb是兩種 redis持久化的機制。對內存數據庫而言,系統宕機、硬件損壞等都會對數據造成損失。而redis對此有很強的修復能力,RDB (Redis DataBase)和 AOF (Append Only File)就是針對此類問題的解決方案。
RDB
Redis DataBase:在指定的時間間隔內將內存中的數據集快照寫入磁盤,也就是行話講的Snapshot快照,它恢復時是將快照文件直接讀到內存裏.
Redis會單獨創建(Fork)一個子進程來進行持久化,會先將數據寫入到一個臨時文件中,待持久化過程都結束了,再用這個臨時文件替換上次持久化好的文件。整個過程中,主進程是不進行任何IO操作的,這就確保了極高的性能
如果需要進行大規模數據的恢復,且對於數據恢復的完整性不是非常敏感,那RDB方式要比AOF方式更加的高效。RDB的缺點是最後一次持久化後的數據可能丟失。
Fork的作用是複製一個與當前進程一樣的進程。新進程的所有數據(變量、環境變量、程序計數器等)數值都和原進程一致,但是是一個全新的進程,並作爲原進程的子進程
配置文件中所處的位置:
################################ SNAPSHOTTING ################################ # # Save the DB on disk: # # save <seconds> <changes> # # Will save the DB if both the given number of seconds and the given # number of write operations against the DB occurred. # # In the example below the behaviour will be to save: # after 900 sec (15 min) if at least 1 key changed # after 300 sec (5 min) if at least 10 keys changed # after 60 sec if at least 10000 keys changed # # Note: you can disable saving completely by commenting out all "save" lines. # # It is also possible to remove all the previously configured save # points by adding a save directive with a single empty string argument # like in the following example: # # save ""
# save 秒鐘 操作次數 save 900 1 save 300 10 save 60 10000
首先RDB 保存的是dump.rdb文件,RDB是整個內存壓縮過的Snapshot,RDB的數據結構,可以配置複合的快照觸發條件,出廠默認是一分鐘內改一萬次,或五分鐘內改十次,或十五分鐘內改一次。
Redis的Save命令可以快速備份,也就是說不用等到配置文件中配置的時間點,手動操作備份數據.FlushAll命令也會觸發dump.rdb文件的改寫(會清空掉)。Shutdown後也會生成dump.rdb文件.備份的文件所在備機和要恢復的主機最好不是同一臺服務器.
Save:save時只管保存,其它不管,全部阻塞(在保存的這個時間範圍內,Redis只管保存,不處理寫入)
BGSAVE:Redis會在後臺異步進行快照操作,快照同時還可以響應客戶端請求。可以通過lastsave命令獲取最後一次成功執行快照的時間
執行flushall命令,也會產生dump.rdb文件,但裏面是空的,無意義
redis-server redis.windows.conf //redis啓動後,會自動加載dump.rdb文件實現數據的恢復
AOF
概念:以日誌的形式來記錄每個寫操作,將Redis執行過的所有寫指令記錄下來(讀操作不記錄),只許追加文件但不可以改寫文件,
redis啓動之初會讀取該文件重新構建數據,換言之,redis重啓後就根據日誌文件的內容將寫指令從前到後執行一次以完成數據的恢復工作
配置文件所處位置:
############################## APPEND ONLY MODE ############################### # By default Redis asynchronously dumps the dataset on disk. This mode is # good enough in many applications, but an issue with the Redis process or # a power outage may result into a few minutes of writes lost (depending on # the configured save points). # # The Append Only File is an alternative persistence mode that provides # much better durability. For instance using the default data fsync policy # (see later in the config file) Redis can lose just one second of writes in a # dramatic event like a server power outage, or a single write if something # wrong with the Redis process itself happens, but the operating system is # still running correctly. # # AOF and RDB persistence can be enabled at the same time without problems. # If the AOF is enabled on startup Redis will load the AOF, that is the file # with the better durability guarantees. # # Please check http://redis.io/topics/persistence for more information. appendonly no //默認不使用 # The name of the append only file (default: "appendonly.aof") //默認的文件名字爲appendonly.aof appendfilename "appendonly.aof" # The fsync() call tells the Operating System to actually write data on disk # instead of waiting for more data in the output buffer. Some OS will really flush # data on disk, some other OS will just try to do it ASAP. # # Redis supports three different modes: # # no: don't fsync, just let the OS flush the data when it wants. Faster. # always: fsync after every write to the append only log . Slow, Safest. # everysec: fsync only one time every second. Compromise. # # The default is "everysec", as that's usually the right compromise between # speed and data safety. It's up to you to understand if you can relax this to # "no" that will let the operating system flush the output buffer when # it wants, for better performances (but if you can live with the idea of # some data loss consider the default persistence mode that's snapshotting), # or on the contrary, use "always" that's very slow but a bit safer than # everysec. # # More details please check the following article: # http://antirez.com/post/redis-persistence-demystified.html # # If unsure, use "everysec". # appendfsync always //每一次寫的操作就立即同步 appendfsync everysec //默認每秒同步數據 # appendfsync no //不同步 # When the AOF fsync policy is set to always or everysec, and a background # saving process (a background save or AOF log background rewriting) is # performing a lot of I/O against the disk, in some Linux configurations # Redis may block too long on the fsync() call. Note that there is no fix for # this currently, as even performing fsync in a different thread will block # our synchronous write(2) call. # # In order to mitigate this problem it's possible to use the following option # that will prevent fsync() from being called in the main process while a # BGSAVE or BGREWRITEAOF is in progress. # # This means that while another child is saving, the durability of Redis is # the same as "appendfsync none". In practical terms, this means that it is # possible to lose up to 30 seconds of log in the worst scenario (with the # default Linux settings). # # If you have latency problems turn this to "yes". Otherwise leave it as # "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no # Automatic rewrite of the append only file. # Redis is able to automatically rewrite the log file implicitly calling # BGREWRITEAOF when the AOF log size grows by the specified percentage. # # This is how it works: Redis remembers the size of the AOF file after the # latest rewrite (if no rewrite has happened since the restart, the size of # the AOF at startup is used). # # This base size is compared to the current size. If the current size is # bigger than the specified percentage, the rewrite is triggered. Also # you need to specify a minimal size for the AOF file to be rewritten, this # is useful to avoid rewriting the AOF file even if the percentage increase # is reached but it is still pretty small. # # Specify a percentage of zero in order to disable the automatic AOF # rewrite feature.
#重寫機制,以下兩個是觸發條件 auto-aof-rewrite-percentage 100 //上次重寫後的一倍 auto-aof-rewrite-min-size 64mb //文件大於64M # An AOF file may be found to be truncated at the end during the Redis # startup process, when the AOF data gets loaded back into memory. # This may happen when the system where Redis is running # crashes, especially when an ext4 filesystem is mounted without the # data=ordered option (however this can't happen when Redis itself # crashes or aborts but the operating system still works correctly). # # Redis can either exit with an error when this happens, or load as much # data as possible (the default now) and start if the AOF file is found # to be truncated at the end. The following option controls this behavior. # # If aof-load-truncated is set to yes, a truncated AOF file is loaded and # the Redis server starts emitting a log to inform the user of the event. # Otherwise if the option is set to no, the server aborts with an error # and refuses to start. When the option is set to no, the user requires # to fix the AOF file using the "redis-check-aof" utility before to restart # the server. # # Note that if the AOF file will be found to be corrupted in the middle # the server will still exit with an error. This option only applies when # Redis will try to read more data from the AOF file but not enough bytes # will be found. aof-load-truncated yes
dump.rdb與appendononly.aof可以共存,先加載的是appendonly.aof文件
文件修復:Redis-check-aof --fix appendonly.aof //當出現文件錯損時就可以執行修復,redis會修復不規範的語法
redis-check-dump是用來修復dump.rdb文件的
aof的重寫機制
AOF採用文件追加方式,文件會越來越大爲避免出現此種情況,新增了重寫機制,當AOF文件的大小超過所設定的閾值時,Redis就會啓動AOF文件的內容壓縮,只保留可以恢復數據的最小指令集(redis會將重複的寫的指令進行去重等措施來重寫aof文件).
也可以使用命令bgrewriteaof來重寫。
AOF進行重寫是首先會有兩個觸發機制:Redis會記錄上次重寫時的AOF大小,默認配置是當AOF文件大小是上次rewrite後大小的一倍且文件大於64M時觸發,默認是這樣,但我們也可以改配置文件。
重寫原理:
AOF文件持續增長而過大時,會fork出一條新進程來將文件重寫(也是先寫臨時文件最後再rename),遍歷新進程的內存中數據,每條記錄有一條的Set語句。重寫aof文件的操作,並沒有讀取舊的aof文件,
而是將整個內存中的數據庫內容用命令的方式重寫了一個新的aof文件,這點和快照有點類似
aof與rdb比較:相同數據集的數據而言aof文件要遠大於rdb文件,恢復速度慢於rdb
Aof運行效率要慢於rdb,每秒同步策略效率較好,不同步效率和rdb相同
如有不當,望包涵!
😉