redis-cluster部署及數據遷移

原創 蘇黎世1995 2018-09-10 02:49

工作原理

節選自redis官方文檔：http://www.redis.cn/topics/cluster-tutorial.html

Redis集羣介紹

Redis 集羣是一個提供在多個Redis間節點間共享數據的程序集。
Redis集羣並不支持處理多個keys的命令,因爲這需要在不同的節點間移動數據,從而達不到像Redis那樣的性能,在高負載的情況下可能會導致不可預料的錯誤.
Redis 集羣通過分區來提供一定程度的可用性,在實際環境中當某個節點宕機或者不可達的情況下繼續處理命令. Redis 集羣的優勢:

  • 自動分割數據到不同的節點上。
  • 整個集羣的部分節點失敗或者不可達的情況下能夠繼續處理命令。

    Redis 集羣的數據分片

    Redis 集羣沒有使用一致性hash, 而是引入了 哈希槽的概念.
    Redis 集羣有16384個哈希槽,每個key通過CRC16校驗後對16384取模來決定放置哪個槽.集羣的每個節點負責一部分hash槽,舉個例子,比如當前集羣有3個節點,那麼:

  • 節點 A 包含 0 到 5500號哈希槽.
  • 節點 B 包含5501 到 11000 號哈希槽.
  • 節點 C 包含11001 到 16384號哈希槽.

這種結構很容易添加或者刪除節點. 比如如果我想新添加個節點D, 我需要從節點 A, B, C中得部分槽到D上. 如果我像移除節點A,需要將A中得槽移到B和C節點上,然後將沒有任何槽的A節點從集羣中移除即可. 由於從一個節點將哈希槽移動到另一個節點並不會停止服務,所以無論添加刪除或者改變某個節點的哈希槽的數量都不會造成集羣不可用的狀態.

Redis 集羣的主從複製模型

爲了使在部分節點失敗或者大部分節點無法通信的情況下集羣仍然可用，所以集羣使用了主從複製模型,每個節點都會有N-1個複製品.
在我們例子中具有A，B，C三個節點的集羣,在沒有複製模型的情況下,如果節點B失敗了，那麼整個集羣就會以爲缺少5501-11000這個範圍的槽而不可用.
然而如果在集羣創建的時候（或者過一段時間）我們爲每個節點添加一個從節點A1，B1，C1,那麼整個集羣便有三個master節點和三個slave節點組成，這樣在節點B失敗後，集羣便會選舉B1爲新的主節點繼續服務，整個集羣便不會因爲槽找不到而不可用了
不過當B和B1 都失敗後，集羣是不可用的.

redis安裝過程

1、下載redis安裝包

2、解壓

cd /app
tar zxf redis-3.2.11.tar.gz 
mv redis-3.2.11.tar.gz redis

3、編譯安裝

cd redis

make && make install

4、創建相關目錄,將相關命令從src目錄複製到bin目錄

mkdir -pv /app/redis/{bin,conf,data,logs}

5、使用utils中的install_server.sh安裝redis server

部署過程

Redis-cluster搭建

1、從測試打包已經編譯redis文件夾，上傳至需要安裝的服務器

2、解壓至app目錄，目錄結構如下，其中data,conf,logs爲編譯完成後自行創建的目錄，分別用來存放數據文件，配置文件，日誌文件，data目錄下自行創建以端口號命名的文件夾，分別用來存放各個端口號實例的數據文件

-rw-rw-r--  1 root root 92766 Sep 21 22:20 00-RELEASENOTES
drwxr-xr-x  2 root root  4096 Dec 20 16:40 bin
-rw-rw-r--  1 root root    53 Sep 21 22:20 BUGS
drwxr-xr-x  2 root root  4096 Jan  4 15:37 conf
-rw-rw-r--  1 root root  1805 Sep 21 22:20 CONTRIBUTING
-rw-rw-r--  1 root root  1487 Sep 21 22:20 COPYING
drwxr-xr-x 10 root root  4096 Jan  3 11:24 data
drwxrwxr-x  7 root root  4096 Dec 20 14:50 deps
-rw-rw-r--  1 root root    11 Sep 21 22:20 INSTALL
drwxr-xr-x  2 root root  4096 Jan  4 15:37 logs
-rw-rw-r--  1 root root   151 Sep 21 22:20 Makefile
-rw-rw-r--  1 root root  4223 Sep 21 22:20 MANIFESTO
-rw-rw-r--  1 root root  6834 Sep 21 22:20 README.md
-rw-rw-r--  1 root root 46695 Sep 21 22:20 redis.conf
-rwxrwxr-x  1 root root   271 Sep 21 22:20 runtest
-rwxrwxr-x  1 root root   280 Sep 21 22:20 runtest-cluster
-rwxrwxr-x  1 root root   281 Sep 21 22:20 runtest-sentinel
-rw-rw-r--  1 root root  7606 Sep 21 22:20 sentinel.conf
drwxrwxr-x  2 root root  4096 Dec 20 14:51 src
drwxrwxr-x 10 root root  4096 Sep 21 22:20 tests
drwxrwxr-x  7 root root  4096 Jan  4 14:36 utils

3、修改各個端口的配置文件，以node1:7000的配置文件爲例，根據情況修改

bind node1
protected-mode yes
port 7000
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile /var/run/redis_7000.pid
loglevel notice
logfile /app/redis/logs/redis_7000.log
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
dir /app/redis/data/7000
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

4、啓動對應端口的服務（所有節點的主從都啓動）,以7000爲例

/app/redis/bin/redis-server  /app/redis/conf/7000.conf

5、創建集羣

 /app/redis/bin/redis-trib.rb create node1:7000 node2:7000 node2:7000

[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:5461-10922 (5461 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:0-5460 (5461 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:10923-16383 (5461 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

6、從節點的添加需要在數據遷移之後

數據遷移

根據實際情況，參考：https://www.18188.org/articles/2016/04/23/1461374145366.html 進行數據遷移。

1、將所有的slot都轉移到一臺主實例


[root@node1 bin]# ./redis-trib.rb reshard --from b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f   --to 3b485951a72e16133464585dc2920c10ce75967b  --slots 5462 --yes node1:7000

[root@node1 bin]# ./redis-trib.rb reshard --from 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc   --to 3b485951a72e16133464585dc2920c10ce75967b  --slots 5462 --yes node1:7000

2、將原有的阿里雲redis備份rdb文件下載放到對應目錄

cp /app/hins2533955_data_20180105195042.rdb /app/redis/data/7000/dump.rdb

3、重啓將備份文件讀入,重啓完成連接進入使用dbsize查看數據是否導入

/app/redis/bin/redis-cli -h node1 -p 7000 shutdown
/app/redis/bin/redis-server /app/redis/conf/7000.conf

4、重新分配slot

[root@node1 bin]# ./redis-trib.rb reshard --from 3b485951a72e16133464585dc2920c10ce75967b   --to b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f  --slots 5462 --yes node2:7000

[root@node1 bin]# ./redis-trib.rb reshard --from 3b485951a72e16133464585dc2920c10ce75967b   --to 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc  --slots 5462 --yes node3:7000

5、增加從節點,各節點對應關係：

1.node1:7000(主)---》node4:7000(從)
2.node2:7000(主)---》node5:7000(從)
3.node3:7000(主)---》 node6:7000(從)
[root@node1 bin]# ./redis-trib.rb check node1.7000
Invalid IP or Port (given as node1.7000) - use IP:Port format
[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id 3b485951a72e16133464585dc2920c10ce75967b  node4:7000 node1:7000
>>> Adding node node4:7000 to cluster node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   0 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node4:7000 to make it join the cluster.
Waiting for the cluster to join...
>>> Configure node as replica of node1:7000.
[OK] New node added correctly.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f  node5:7000 node2:7000
>>> Adding node node5:7000 to cluster node2:7000
>>> Performing Cluster Check (using node node2:7000)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   0 additional replica(s)
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node5:7000 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of node2:7000.
[OK] New node added correctly.
[root@node1 bin]# ./redis-trib.rb add-node --slave --master-id 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc  node6:7000 node3:7000
>>> Adding node node6:7000 to cluster node3:7000
>>> Performing Cluster Check (using node node3:7000)
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   0 additional replica(s)
S: c09449b9dce5af4ac485ccedb664188229d75430 node5:7000
   slots: (0 slots) slave
   replicates b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   1 additional replica(s)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node node6:7000 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of node3:7000.
[OK] New node added correctly.

[root@node1 bin]# ./redis-trib.rb check node1:7000
>>> Performing Cluster Check (using node node1:7000)
M: 3b485951a72e16133464585dc2920c10ce75967b node1:7000
   slots:10924-16383 (5460 slots) master
   1 additional replica(s)
S: 8ff59145b1d3d7fd57923f3cb9c3444f7236d7b1 node4:7000
   slots: (0 slots) slave
   replicates 3b485951a72e16133464585dc2920c10ce75967b
S: 7501954b635724d66285afbb95841f4b49df5645 node6:7000
   slots: (0 slots) slave
   replicates 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc
S: c09449b9dce5af4ac485ccedb664188229d75430 node5:7000
   slots: (0 slots) slave
   replicates b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f
M: 54d5eb9415289228b05b28fe585bd5ab6e9bc0bc node3:7000
   slots:5462-10923 (5462 slots) master
   1 additional replica(s)
M: b6dd7c3d5af4ffc37e70d7eb538a3f9b0ae5a84f node2:7000
   slots:0-5461 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

QuickRedis 是一款 永久免費 的 Redis 可視化管理工具

zhoudian2010 2022-03-25 16:55:09

Redis主從複製以及主從複製原理

LoyaltyLu 2019-02-24 23:12:38

Redis | 001能做什麼？

Java貓說 2019-02-24 21:03:51

數據工程師妹子養成手記——數據庫篇

青南 2019-02-23 16:33:37

redis數據庫隨筆

simuls 2019-02-23 13:34:47

redis慢查詢日誌，php安裝redis擴展，redis存儲session，redis主從配置

13011150876 2019-02-23 00:41:11

redis慢查詢日誌，php安裝redis擴展，redis存儲session，redis主從配置

13011150876 2019-02-23 00:41:10

Redis 學習筆記

jasonhow 2019-02-23 00:39:36

Jedis連接池的使用

殺神之瘋子 2019-02-23 00:35:12

python 實現類似tweproxy 和codes的功能的redis代理功能

tianshuai369 2019-02-23 00:34:58

關於Mac安裝PHP相關擴展出現Zend/zend_config.h缺失的問題記錄

乄尐 2019-02-23 00:34:32

安裝redis支持php擴展

yfshare 2019-02-23 00:31:38

又一次redis被刪庫跑路,索要0.6比特幣

冷暖己知 2019-02-23 00:29:40

在Windows系統上安裝Redis

飛魚之戀 2019-02-23 00:27:22

redis

18518584522 2019-02-23 00:24:13