主從 + sentinel 實現redis的高可用

redis提供主從模式(也就是複製replication), 如果不太清楚主從搭建過程的請參考之前博客, 一主多從這種模式只是將讀寫進行了分類,如果主發生了故障,整個redis系統都將變的不可用. 然而redis 引進了哨兵, 哨兵可以獨立與redis運行的分佈式服務. 提供redis實時監控和故障檢測恢復的功能. 不瞭解哨兵特性的可參考之前關於哨兵的博客.

這裏記錄主從搭配合作實現redis的高可用(HA).

實戰圖解

       +----+
       | M1 |
       | S1 |
       +----+
          |
+----+    |    +----+
| R2 |----+----| R3 |
| S2 |         | S3 |
+----+         +----+

Configuration: quorum = 2

我們都知道哨兵的啓動方式:

redis-sentinel /path/to/sentinel_6379.conf --sentinel 

redis 版本

使用版本是3.2.1

27.0.0.1:6381> 
127.0.0.1:6381> info server
# Server
redis_version:3.2.1
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b3132c8ce7b475fa
redis_mode:standalone
os:Linux 3.13.0-32-generic x86_64
arch_bits:64
multiplexing_api:epoll

配置哨兵

自定義創建三個哨兵配置文件,這裏使用

sentinel_26379.conf

sentinel_26380.conf

sentinel_26381.conf

配置內容如下:

port 26379

sentinel monitor mymaster 127.0.0.1 6381 2

sentinel down-after-milliseconds mymaster 5000

# master 有密碼就要使用,
sentinel auth-pass mymaster ****

sentinel failover-timeout resque 180000.

sentinel parallel-syncs resque 5

其他兩個,端口換下就好.

配置啓動腳本文件

創建哨兵啓動腳本,sentinel_26379; sentinel_26380; sentinel_26381;

詳細內容如下:

#!/bin/sh
#
# Simple Redis init.d script conceived to work on Linux systems
# as it does use of the /proc filesystem.


 redis-sentinel /path/to/sentinel_26379.conf --sentinel &


esac

驗證運行狀態

分別啓動三個哨兵.

登陸6381(master),關閉 shutdown redis

宕機前的redis 主從狀態:

root@ubuntu:~/programs# redis-cli -p 6381
127.0.0.1:6381> 
127.0.0.1:6381> auth *****
OK
127.0.0.1:6381> 
127.0.0.1:6381> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6379,state=online,offset=1446113,lag=1
slave1:ip=127.0.0.1,port=6380,state=online,offset=1446113,lag=1
master_repl_offset:1446262
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:397687
repl_backlog_histlen:1048576
127.0.0.1:6381> 

執行關閉

127.0.0.1:6381> 
127.0.0.1:6381> 
127.0.0.1:6381> shutdown save
not connected> 
not connected> 

監控哨兵的打印狀態:

2078:X 28 Jul 19:50:11.967 # +sdown master mymaster 127.0.0.1 6381
2078:X 28 Jul 19:50:12.108 # +new-epoch 28
2078:X 28 Jul 19:50:12.175 # +vote-for-leader 1d9a0fc928f84a79b2e2aaa686db2ae735b6958d 28
2078:X 28 Jul 19:50:13.026 # +odown master mymaster 127.0.0.1 6381 #quorum 3/2
2078:X 28 Jul 19:50:13.026 # Next failover delay: I will not start a failover before Thu Jul 28 19:56:13 2016
2078:X 28 Jul 19:50:13.261 # +config-update-from sentinel 1d9a0fc928f84a79b2e2aaa686db2ae735b6958d 127.0.0.1 26380 @ mymaster 127.0.0.1 6381
2078:X 28 Jul 19:50:13.261 # +switch-master mymaster 127.0.0.1 6381 127.0.0.1 6380
2078:X 28 Jul 19:50:13.261 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6380
2078:X 28 Jul 19:50:13.262 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
2078:X 28 Jul 19:50:18.279 # +sdown slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380

根據上面信息可以看到: 其中一個哨兵發現6381 主觀下線, 然後發起了新的一輪投票;
quorum3/2 三個哨兵都認爲6381下線了。然後啓動了故障恢復, 最終選舉6380爲的主.

測試在登陸到6380 查看運行信息;

127.0.0.1:6381> shutdown save
not connected> 
not connected> 
not connected> 
not connected> quit
root@ubuntu:~/programs# 
root@ubuntu:~/programs# redis-cli -p 6380
127.0.0.1:6380> auth ****
OK
127.0.0.1:6380> 
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6379,state=online,offset=56725,lag=0
master_repl_offset:56860
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:56859
127.0.0.1:6380> 

可以看到6380已經是master,而且有一個從6379; 然後我們將6381重新啓動,看是否它加入到新的master-6380下麼.

127.0.0.1:6380> 
127.0.0.1:6380> quit
root@ubuntu:~/programs# 
root@ubuntu:~/programs# /etc/init.d/redis_6380 start
/var/run/redis_6380.pid exists, process is already running or crashed
root@ubuntu:~/programs# 
root@ubuntu:~/programs# 
root@ubuntu:~/programs# /etc/init.d/redis_6381 start
Starting Redis server...
root@ubuntu:~/programs# 
root@ubuntu:~/programs# redis-cli -p 6380
127.0.0.1:6380> auth ****
OK
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6379,state=online,offset=91546,lag=0
slave1:ip=127.0.0.1,port=6381,state=online,offset=91560,lag=0
master_repl_offset:91560
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:91559
127.0.0.1:6380> 

可以看到6381已經是6380的從機了.

在6381啓動過程中哨兵監控的內容

2104:X 28 Jul 19:57:38.988 # -sdown slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380
2104:X 28 Jul 19:57:48.947 * +convert-to-slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6380

6381啓動好了,客觀認爲6381不在是主,直接將角色轉換成了6380的從機.

此時redis 主從 + 哨兵 實現高可用方案搭建完畢.

繼續將6380宕機,監控如下:

2104:X 28 Jul 20:04:01.733 # +new-epoch 29
2104:X 28 Jul 20:04:01.767 # +vote-for-leader b24714af31039c93f6ad4173c059c4d11e86f302 29
2104:X 28 Jul 20:04:01.767 # +odown master mymaster 127.0.0.1 6380 #quorum 2/2
2104:X 28 Jul 20:04:01.767 # Next failover delay: I will not start a failover before Thu Jul 28 20:10:02 2016
2104:X 28 Jul 20:04:02.801 # +config-update-from sentinel b24714af31039c93f6ad4173c059c4d11e86f302 127.0.0.1 26379 @ mymaster 127.0.0.1 6380
2104:X 28 Jul 20:04:02.801 # +switch-master mymaster 127.0.0.1 6380 127.0.0.1 6381
2104:X 28 Jul 20:04:02.801 * +slave slave 127.0.0.1:6379 127.0.0.1 6379 @ mymaster 127.0.0.1 6381
2104:X 28 Jul 20:04:02.801 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381
2104:X 28 Jul 20:04:07.834 # +sdown slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 127.0.0.1 6381

歡迎大家評論,吐槽

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章