Redis熱遷移實戰分享

Redis cluster使用slots來分配集羣中的資源，因此官方提供了熱遷移slots的方案，以便於遷移redis cluster節點中的信息。此方案不僅可以使用於節點遷移，也可以根據資源的不同，配置不同的slots數量。

集羣原環境：

主機IP： 192.168.170.101

集羣節點端口：10001-10006

集羣當前主備關係：

 

遷移需求：現在有節點redis007,需要將redis002替換掉

遷移步驟：

1. 首先驗證環境當前的配置：

由上可以看出節點信息如下：

10001M<-10004S  slots=5461

10002M<-10005S      slots=5462

10003M<-10006S      slots=5461

2. 現在來插入10W條數據

from rediscluster import StrictRedisCluster

redis_nodes = [

        {"host": "192.168.170.101", "port": "10001"},

        {"host": "192.168.170.101", "port": "10002"},

        {"host": "192.168.170.101", "port": "10003"},

        {"host": "192.168.170.101", "port": "10004"},

        {"host": "192.168.170.101", "port": "10005"},

        {"host": "192.168.170.101", "port": "10006"}

    ]

redis_conn = StrictRedisCluster(startup_nodes=redis_nodes, decode_responses=True)

for key in range(0, 100000):

    print key

    value = key

    key = 'zhang%s' % key

    redis_conn.set(key, value)

3. 啓動redis007，並將redis007加入集羣環境：

./redis-server redis007.conf

[root@lab001 redis]# redis-trib.rb add-node 192.168.170.101:10007 192.168.170.101:10001

>>> Adding node 192.168.170.101:10007 to cluster 192.168.170.101:10001

>>> Performing Cluster Check (using node 192.168.170.101:10001)

M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005

   slots: (0 slots) slave

   replicates 6036af6afc6567b74ce2fecc734a2d3908d561d1

M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006

   slots: (0 slots) slave

   replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed

S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004

   slots: (0 slots) slave

   replicates b30fdabd07a4bef611c160828965b91a1cdd462a

M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

>>> Send CLUSTER MEET to node 192.168.170.101:10007 to make it join the cluster.

[OK] New node added correctly.

4. 查看當前節點狀態信息

[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001

>>> Performing Cluster Check (using node 192.168.170.101:10001)

M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007

   slots: (0 slots) master

   0 additional replica(s)

S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005

   slots: (0 slots) slave

   replicates 6036af6afc6567b74ce2fecc734a2d3908d561d1

M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006

   slots: (0 slots) slave

   replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed

S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004

   slots: (0 slots) slave

   replicates b30fdabd07a4bef611c160828965b91a1cdd462a

M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

5. 由上可以發現，redis007添加到節點當中，但當前未分配任何slots，這一步把redis002的slots完全遷移到redis007

由於過程較長，這裏不再輸出全部過程

[root@lab001 redis]# redis-trib.rb reshard --from 6036af6afc6567b74ce2fecc734a2d3908d561d1 --to 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 --slots 5462 --yes 192.168.170.101:10001

………………

Moving slot 10911 from 192.168.170.101:10002 to 192.168.170.101:10007: .....

Moving slot 10912 from 192.168.170.101:10002 to 192.168.170.101:10007: ..........

Moving slot 10913 from 192.168.170.101:10002 to 192.168.170.101:10007: ......

Moving slot 10914 from 192.168.170.101:10002 to 192.168.170.101:10007: .......

Moving slot 10915 from 192.168.170.101:10002 to 192.168.170.101:10007: .....

Moving slot 10916 from 192.168.170.101:10002 to 192.168.170.101:10007: .....

Moving slot 10917 from 192.168.170.101:10002 to 192.168.170.101:10007: ...

Moving slot 10918 from 192.168.170.101:10002 to 192.168.170.101:10007: ........

Moving slot 10919 from 192.168.170.101:10002 to 192.168.170.101:10007: .....

Moving slot 10920 from 192.168.170.101:10002 to 192.168.170.101:10007: .......

Moving slot 10921 from 192.168.170.101:10002 to 192.168.170.101:10007: ..

Moving slot 10922 from 192.168.170.101:10002 to 192.168.170.101:10007: ..........

6. 再次查看當前節點信息

[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001

>>> Performing Cluster Check (using node 192.168.170.101:10001)

M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005

   slots: (0 slots) slave

   replicates 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8

M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006

   slots: (0 slots) slave

   replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed

S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004

   slots: (0 slots) slave

   replicates b30fdabd07a4bef611c160828965b91a1cdd462a

M: 6036af6afc6567b74ce2fecc734a2d3908d561d1 192.168.170.101:10002

   slots: (0 slots) master

   0 additional replica(s)

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

由此可以發現，redis002的所有slots遷移到redis007完畢，並且redis005自動將主從關係從002轉移到了007

7. 最後來驗證下數據的完整性

from rediscluster import StrictRedisCluster 

redis_nodes = [

        {"host": "192.168.170.101", "port": "10001"},

        {"host": "192.168.170.101", "port": "10002"},

        {"host": "192.168.170.101", "port": "10003"},

        {"host": "192.168.170.101", "port": "10004"},

        {"host": "192.168.170.101", "port": "10005"},

        {"host": "192.168.170.101", "port": "10006"}

    ]

redis_conn = StrictRedisCluster(startup_nodes=redis_nodes, decode_responses=True)

for key in range(0, 100000):

    #value = key

    key = 'zhang%s' % key

    #redis_conn.set(key, value)

    print redis_conn.get(key)

執行完畢，數據較驗完成，數據完成完整遷移

8. 最後，將redis002從集羣中移除

[root@lab001 redis]# redis-trib.rb del-node 192.168.170.101:10001 6036af6afc6567b74ce2fecc734a2d3908d561d1

>>> Removing node 6036af6afc6567b74ce2fecc734a2d3908d561d1 from cluster 192.168.170.101:10001

>>> Sending CLUSTER FORGET messages to the cluster...

>>> SHUTDOWN the node.

[2]   Done                    ./redis001/bin/redis-server redis00$i/redis00$i.conf

9. 再次查看集羣狀態

[root@lab001 redis]# redis-trib.rb check 192.168.170.101:10001

>>> Performing Cluster Check (using node 192.168.170.101:10001)

M: b30fdabd07a4bef611c160828965b91a1cdd462a 192.168.170.101:10001

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8 192.168.170.101:10007

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 74805544792cc22f09af941fa117b2974c9c0abf 192.168.170.101:10005

   slots: (0 slots) slave

   replicates 7c1a0acf84b2cf08876036b07518fb8f2ea9b8f8

M: 9c6a98926b23585e2eeb49ae17cc77521041d3ed 192.168.170.101:10003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 57a2e7313b952b38e32d6a1030d4812ea4decb5f 192.168.170.101:10006

   slots: (0 slots) slave

   replicates 9c6a98926b23585e2eeb49ae17cc77521041d3ed

S: 58b7b25501fdba04aef9b8cb47edef89092f10a7 192.168.170.101:10004

   slots: (0 slots) slave

   replicates b30fdabd07a4bef611c160828965b91a1cdd462a

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@lab001 redis]# redis-trib.rb info 192.168.170.101:10001

192.168.170.101:10001 (b30fdabd...) -> 33357 keys | 5461 slots | 1 slaves.

192.168.170.101:10007 (7c1a0acf...) -> 33283 keys | 5462 slots | 1 slaves.

192.168.170.101:10003 (9c6a9892...) -> 33360 keys | 5461 slots | 1 slaves.

[OK] 100000 keys in 3 masters.

6.10 keys per slot on average.

總結：

1. slots遷移在日常工作中，通常用於節點遷移、擴展，也可以根據服務器本身的壓力將slots遷移至性能較優的服務器

2. slots遷移過程當中，不影響數據的讀寫，這點已經做過實際的測試。

3. 遷移過程當中，應用配置應當至少包含一個集羣master節點信息，否則有可能會造成數據訪問異常的情況。遷移完成並配置同步更新至最新後，方可刪除舊的空slots節點。