深入理解Redis系列之集羣環境搭建

原創 terryKing1992 2018-11-26 23:04
前面分別寫了關於單機版Redis搭建以及使用SpringBoot來訪問Redis服務, 後面也就順着寫一些關於分佈式環境的搭建以及高可用的大概原理;

分佈式環境準備

因爲在前面一篇深入理解Redis系列之單機Redis環境搭建中已經安裝好了Redis的介質, 後面只需要使用配置文件指定不同端口來做Master以及Slave節點即可;因爲是在一臺機器安裝, 所以只能通過端口來啓動不同角色的Redis進程; 規劃如下:

機器IP 端口 角色 配置文件目錄
127.0.0.1 6379 Master1 /usr/local/redis/conf/6379/redis.conf
127.0.0.1 6380 6380 /usr/local/redis/conf/6380/redis.conf
127.0.0.1 6381 6381 /usr/local/redis/conf/6381/redis.conf
127.0.0.1 26379 26379 /usr/local/redis/conf/26379/redis.conf
127.0.0.1 26380 26380 /usr/local/redis/conf/26380/redis.conf
127.0.0.1 26381 26381 /usr/local/redis/conf/26381/redis.conf

注: 本來使用的16379、16380、16381作爲slave節點的端口的, 但是發現每次啓動6379節點的時候總是把16379端口也同時佔用, 經查證發現每一個redis集羣的節點需要開通兩個TCP端口。一個是用於客戶端的Redis TCP，如6379。另一個由客戶端加10000所得，如16379，用於Redis集羣總線連接； 故改爲26379、26380、26381

生成配置文件

使用如下配置可以批量修改與端口相關內容

vi redis.conf
esc 命令之後
:%s/6379/{replace_port}/g

分別修改如下配置:

port 6379 //根據上面的規劃調整
daemonize yes // 從no => yes
pidfile /var/run/redis_6379.pid //根據規劃調整 
cluster-enabled yes //將該行註釋去掉, 如果沒有這一行就添加到配置文件
bind 127.0.0.1 //如果是外部訪問，改爲本機的IP地址，而不是本地迴路127.0.0.1
dbfilename dump_6379.rdb //根據端口設置
logfile "6379.log" //啓動日誌也可以打開, 便於看出錯日誌

啓動master與slave進程

cd /usr/local/redis/conf/6379
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/6380
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/6381
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26379
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26380
../../bin/redis-server redis.conf
cd /usr/local/redis/conf/26381
../../bin/redis-server redis.conf

我們可以通過ps -ef | grep redis 驗證 進程是否起來:

 501 82074     1   0 12:08上午 ??         0:00.11 ../../bin/redis-server 127.0.0.1:6379 [cluster] 
  501 82077     1   0 12:08上午 ??         0:00.09 ../../bin/redis-server 127.0.0.1:6380 [cluster] 
  501 82079     1   0 12:08上午 ??         0:00.07 ../../bin/redis-server 127.0.0.1:6381 [cluster] 
  501 82082     1   0 12:08上午 ??         0:00.05 ../../bin/redis-server 127.0.0.1:26379 [cluster] 
  501 82085     1   0 12:08上午 ??         0:00.04 ../../bin/redis-server 127.0.0.1:26380 [cluster] 
  501 82087     1   0 12:08上午 ??         0:00.02 ../../bin/redis-server 127.0.0.1:26381 [cluster] 
  501 82090 81954   0 12:08上午 ttys003    0:00.00 grep redis

將Redis各進程加入到Redis集羣

1、使用如下命令啓動集羣:

redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:26379 127.0.0.1:26380 127.0.0.1:26381 --cluster-replicas 1

啓動日誌輸出如下:

terrydeMacBook-Air:bin terrylmay$ ./redis-cli --cluster create 127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:26379 127.0.0.1:26380 127.0.0.1:26381 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:26379 to 127.0.0.1:6379
Adding replica 127.0.0.1:26380 to 127.0.0.1:6380
Adding replica 127.0.0.1:26381 to 127.0.0.1:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
M: d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
M: 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
S: 5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379
   replicates d0fc2996f417ac47d4ca174817d7485148534f97
S: 8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380
   replicates 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa
S: 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381
   replicates 3dbb14f3561c413a29c79eb75b39c670692bb1c1
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 127.0.0.1:6379)
M: 3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380
   slots: (0 slots) slave
   replicates 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa
S: 5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379
   slots: (0 slots) slave
   replicates d0fc2996f417ac47d4ca174817d7485148534f97
S: 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381
   slots: (0 slots) slave
   replicates 3dbb14f3561c413a29c79eb75b39c670692bb1c1
M: d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
terrydeMacBook-Air:bin terrylmay$

到此, Master-Slave集羣已經搭建成功了;

注:剛開始沒發覺, 後面驗證集羣master切換的時候才發現, 原來創建集羣的時候並不是按順序一對一的關係, 所以我們可以看到 127.0.0.1:26381 這個進程是 127.0.0.1:6379節點的Slave節點, 這也就是後面再啓動6379進程的時候日誌是連接到26381 master節點了;

那麼集羣master節點與slave節點都有主從複製的功能, 下面我們來驗證一下, 首先登錄redis任意主節點，放入內容之後, 登錄從節點

./redis-cli -c -h 127.0.0.1 -p 6380

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 6380
127.0.0.1:6380> set name 123456
OK
127.0.0.1:6380> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=26379,state=online,offset=110378,lag=0

./redis-cli -c -h 127.0.0.1 -p 26379登錄從節點, 發現通過name也可以拿到值

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 26379
127.0.0.1:26379> get name
-> Redirected to slot [5798] located at 127.0.0.1:6380
"123456"
127.0.0.1:6380>

通過上面日誌，我們看到該信息redirected到了master節點, 那麼問題來了? 從slave裏面讀取數據，會再走到master中讀數據麼? 答案應該不會, 那樣就沒人用slave來當本地緩存了. 但是爲什麼不會呢? 還是想要了解一下! 放在後面探索吧;

這時候，我們刪除其中的一個master 6379節點，看一下會發生什麼情況?

 ps -ef | grep :6379 | awk '{print $2}' |xargs  kill -9

我們在master6379的從節點26379節點上可以看到日誌輸出如下:

82082:S 05 Nov 2018 22:56:08.343 * FAIL message received from 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa about 3dbb14f3561c413a29c79eb75b39c670692bb1c1
82082:S 05 Nov 2018 22:56:08.345 # Cluster state changed: fail
82082:S 05 Nov 2018 22:56:09.153 # Cluster state changed: ok

這時候, 我們還看不出master節點切換了, 僅僅看着像是切換了;

那麼, 我們使用redis-cli命令進入到redis控制檯

./redis-cli -c -h 127.0.0.1 -p 6380

使用命令cluster nodes 查看節點狀態，可以看到

127.0.0.1:6380> cluster nodes
8d6edc0b4d95b989d94e12d9d7915b45db7a6a0b 127.0.0.1:26380@36380 slave 2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 0 1541431115317 5 connected
493d5dfe6d6aec9ce629dc55de58d4072ad1be93 127.0.0.1:26381@36381 master - 0 1541431115000 7 connected 0-5460
3dbb14f3561c413a29c79eb75b39c670692bb1c1 127.0.0.1:6379@16379 slave 493d5dfe6d6aec9ce629dc55de58d4072ad1be93 1541431110864 1541431109000 7 disconnected
5594d0d4e6114533c4b598307a94eeb56605970f 127.0.0.1:26379@36379 slave d0fc2996f417ac47d4ca174817d7485148534f97 0 1541431116330 4 connected
2dfe9608f3a3fc131f746a3336aa2e01cfb11faa 127.0.0.1:6381@16381 master - 0 1541431115000 3 connected 10923-16383
d0fc2996f417ac47d4ca174817d7485148534f97 127.0.0.1:6380@16380 myself,master - 0 1541431115000 2 connected 5461-10922

6379節點已經處於disconnected狀態了

使用命令cluster info 查看集羣狀態, 發現cluster 狀態正常

cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384

說明master節點已經切換到了26381節點;

然後, 我們試着重啓剛纔刪掉的6379節點，可以看到6379日誌打印如下:

90819:C 05 Nov 2018 23:05:56.067 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
90819:C 05 Nov 2018 23:05:56.067 # Redis version=5.0.0, bits=64, commit=00000000, modified=0, pid=90819, just started
90819:C 05 Nov 2018 23:05:56.067 # Configuration loaded
90820:M 05 Nov 2018 23:05:56.070 * Increased maximum number of open files to 10032 (it was originally set to 256).
90820:M 05 Nov 2018 23:05:56.073 * Node configuration loaded, I'm 3dbb14f3561c413a29c79eb75b39c670692bb1c1
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 5.0.0 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in cluster mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 90820
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

90820:M 05 Nov 2018 23:05:56.077 # Server initialized
90820:M 05 Nov 2018 23:05:56.077 * DB loaded from disk: 0.000 seconds
90820:M 05 Nov 2018 23:05:56.077 * Ready to accept connections
90820:M 05 Nov 2018 23:05:56.079 # Configuration change detected. Reconfiguring myself as a replica of 493d5dfe6d6aec9ce629dc55de58d4072ad1be93
90820:S 05 Nov 2018 23:05:56.079 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
90820:S 05 Nov 2018 23:05:56.080 # Cluster state changed: ok
90820:S 05 Nov 2018 23:05:57.088 * Connecting to MASTER 127.0.0.1:26381
90820:S 05 Nov 2018 23:05:57.096 * MASTER <-> REPLICA sync started
90820:S 05 Nov 2018 23:05:57.097 * Non blocking connect for SYNC fired the event.
90820:S 05 Nov 2018 23:05:57.097 * Master replied to PING, replication can continue...
90820:S 05 Nov 2018 23:05:57.098 * Trying a partial resynchronization (request 385e7338e98550e9dc9d1cd5a3ea77d4e39e5f51:1).
90820:S 05 Nov 2018 23:05:57.099 * Full resync from master: 306b56738fb37bd1d1fcb9e0f709584c56f2df52:107352
90820:S 05 Nov 2018 23:05:57.099 * Discarding previously cached master state.
90820:S 05 Nov 2018 23:05:57.188 * MASTER <-> REPLICA sync: receiving 178 bytes from master
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Flushing old data
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Loading DB in memory
90820:S 05 Nov 2018 23:05:57.189 * MASTER <-> REPLICA sync: Finished with success

Connecting to MASTER 127.0.0.1:26381 說明先前master節點切換到26381節點上; 集羣信息都有了更新; 當然我們可以通過./redis-server -c -h 127.0.0.1 -p 26381 然後使用 info replication查看slave節點信息

terrydeMacBook-Air:bin terrylmay$ ./redis-cli -c -h 127.0.0.1 -p 26381
127.0.0.1:26381> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6379,state=online,offset=108430,lag=0
master_replid:306b56738fb37bd1d1fcb9e0f709584c56f2df52
master_replid2:37b3866dc3956911c64eca1ea3df03aa786d4813
master_repl_offset:108430
second_repl_offset:107353
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:108430

到這裏, 基本上主從切換就OK了; 下一篇文章寫一下如何使用SpringBoot訪問Redis集羣;

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

ggit (git gui) --- 開發記錄 (一)

sewerganger 2019-02-24 22:45:04

git版本更新小記

raferz 2019-02-24 00:23:46

gitlab命令行使用（基礎篇）

cafebabe 2019-02-23 13:53:44

QuickRedis 是一款 永久免費 的 Redis 可視化管理工具

zhoudian2010 2022-03-25 16:55:09

Redis主從複製以及主從複製原理

LoyaltyLu 2019-02-24 23:12:38

Redis | 001能做什麼？

Java貓說 2019-02-24 21:03:51

數據工程師妹子養成手記——數據庫篇

青南 2019-02-23 16:33:37

redis數據庫隨筆

simuls 2019-02-23 13:34:47

2019年Java面試-併發容器篇

王知無 2019-02-24 15:12:46

深入理解Spring事務的傳播行爲

ゞ .邓澎波 2019-02-24 12:42:54

Spring Cloud Alibaba基礎教程：支持的幾種服務消費方式（RestTemplate、WebClient、Feign）

程序猿dd 2019-02-24 10:41:27

某熊的技術之路指北 ☯

王下邀月熊_Chevalier 2019-02-23 16:23:03

[spring boot] jdbc

locke 2019-02-23 16:13:10

Spring Security視頻地址

joseph999 2019-02-23 13:24:27

精華【分佈式微服務雲架構dubbo+zookeeper+springmvc+mybatis+shiro+redis】分佈式大型互聯網企業架構！

qq5930d18310d6b 2019-02-23 13:16:59