redis手工分片

原創 wzy0623 2020-07-06 18:32

目錄

一、redis環境

二、redis手工分片步驟

1. 配置級聯複製

2. 去掉老哨兵監控

3. 停止新實例從老實例的複製

4. 添加新哨兵監控

5. 重啓新哨兵       

6. 添加老哨兵監控

7. 重啓老哨兵

三、注意事項

四、自動化腳本

 

      隨着數據量越來越大，一個redis實例可能需要分成多個以形成數據分片。此時通常可以採取兩種方式操作：一是啓用cluster模式自動完成數據分片；二是手工分片，即配置需要分片redis實例的副本，再修改應用程序按一定方式（如取模等）訪問不同redis實例。cluster方式的配置可以參考“初學乍練redis：Redis 5.0.3單實例數據遷移到Cluster”。本文說明第二種方式的具體操作步驟。

一、redis環境

        假設現有三臺物理機192.168.1.36、192.168.1.37、192.168.1.38，分別配置爲redis一主兩從，通過哨兵進行訪問。192.168.1.36爲主實例，端口爲20007，master name爲redis7。三臺機器上分別啓動一個哨兵進程，端口爲30001。從哨兵查看複製信息如下：

[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3
[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3
[root~]#sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7
master1:name=redis7,status=ok,address= 192.168.1.36:20007,slaves=2,sentinels=3

        目標是將現有的一組redis（一主兩從）拆分爲四組（每組爲一主兩從），由應用程序實現數據分片策略。新增的三組redis實例所在物理機192.168.1.39、192.168.1.40、192.168.1.41，並且三臺機器上分別啓動一個哨兵進程，依然是通過哨兵進行訪問。

二、redis手工分片步驟

1. 配置級聯複製

（1）在192.168.1.39、192.168.1.40、192.168.1.41上配置三組標準一主兩從redis複製，39爲主，端口分別是20001、20002、20003。

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20001 -a 123456 slaveof 192.168.1.39 20001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20002 -a 123456 slaveof 192.168.1.39 20002
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20003 -a 123456 slaveof 192.168.1.39 20003

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20001 -a 123456 slaveof 192.168.1.39 20001
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20002 -a 123456 slaveof 192.168.1.39 20002
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456 slaveof 192.168.1.39 20003

（2）配置爲老實例的slave，進行數據複製

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20001 -a 123456 slaveof 192.168.1.36 20007
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20002 -a 123456 slaveof 192.168.1.36 20007
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 slaveof 192.168.1.36 20007

        此步完成後形成一個級聯複製，192.168.1.36:20007具有5個slave，分別是192.168.1.37:20007、192.168.1.38:20007、192.168.1.39:20001、192.168.1.39:20002、192.168.1.39:20003。而192.168.1.39上三個redis實例（20001、20002、20003）又分別帶有192.168.1.40、192.168.1.42兩個slave（端口同192.168.1.39）。可以使用以下命令查看各個實例上的鍵數：

# 查看key數量，在192.168.1.39執行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 20007 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 20007 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 20007 -a 123456 info | grep db0:keys=

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 info | grep db0:keys=

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 20003 -a 123456 info | grep db0:keys=

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20001 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20002 -a 123456 info | grep db0:keys=
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456 info | grep db0:keys=

2. 去掉老哨兵監控

# 在192.168.1.36執行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 sentinel remove redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 sentinel remove redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 sentinel remove redis7

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7

3. 停止新實例從老實例的複製

#  在192.168.1.39執行
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20003 -a 123456
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20004 -a 123456
cat ~/remove_repl.txt | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 20005 -a 123456

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20003 -a 123456 info replication
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20004 -a 123456 info replication
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 20005 -a 123456 info replication

~/remove_repl.txt文件內容：

slaveof no one
config rewrite

4. 添加新哨兵監控

# 在192.168.1.39執行
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 30001

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.39 -p 30001 info | grep master
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.40 -p 30001 info | grep master
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.41 -p 30001 info | grep master

/home/redis/tmp_sentinel_monitor文件內容：

sentinel monitor redis1  192.168.1.39 20001 2
sentinel set redis3 auth-pass 123456
sentinel set redis3 down-after-milliseconds 5000
sentinel set redis3 failover-timeout 10000

sentinel monitor redis2  192.168.1.39 20002 2
sentinel set redis4 auth-pass 123456
sentinel set redis4 down-after-milliseconds 5000
sentinel set redis4 failover-timeout 10000

sentinel monitor redis3  192.168.1.39 20003 2
sentinel set redis5 auth-pass 123456
sentinel set redis5 down-after-milliseconds 5000
sentinel set redis5 failover-timeout 10000

5. 重啓新哨兵       

# 分別在192.168.1.41、192.168.1.40、192.168.1.39執行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 info | grep master

6. 添加老哨兵監控

# 在192.168.1.36執行
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001
cat /home/redis/tmp_sentinel_monitor | sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.36 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.37 -p 30001 info | grep redis7
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -h  192.168.1.38 -p 30001 info | grep redis7

/home/redis/tmp_sentinel_monitor文件內容：

sentinel monitor redis7  192.168.1.36 20007 2
sentinel set redis7 auth-pass 123456
sentinel set redis7 down-after-milliseconds 5000
sentinel set redis7 failover-timeout 10000

7. 重啓老哨兵

# 在192.168.1.38、192.168.1.37、192.168.1.36執行
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &
sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 info

三、注意事項

  • 哨兵實例本身不能加密碼，否則應用程序不能正常連接。
  • 不要在級聯複製的中間層master上加哨兵監控，否則通過哨兵對頂層master的訪問會出問題。
  • 在去掉中間層master對上層的複製時，需要先去掉哨兵對頂層master的監控，否則即使在中間層master實例上執行了slaveof no one，依然會保持頂層master與中間層master的主從關係。
  • 爲保證哨兵工作正常，避免出現+sdown問題，在更改哨兵配置後需要重啓哨兵進程。

四、自動化腳本

        manual_shard.sh 文件內容如下：

#!/bin/bash

if [ ! -n "$1" ] ;then
    echo "請輸入原集羣端口號！"
    exit
fi

if [ ! -n "$2" ] ;then
    echo "請輸入拆分後的新集羣端口號！"
    exit
fi

para=$@

new_repl()
{
    case $1 in
    1)
    index=1
    for a in $para
    do
        if [ $index -gt 1 ]; then
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p $a -a 123456 info 2>/dev/null | grep db1:keys=
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 info 2>/dev/null | grep db1:keys=
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p $a -a 123456 info 2>/dev/null | grep db1:keys=       
        fi
        let index+=1   
    done
    ;;
    2)
    index=1
    for a in $para
    do
        if [ $index -gt 1 ]; then
            cat ~/remove_repl.txt | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 2>/dev/null
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p $a -a 123456 info replication 2>/dev/null
        fi
        let index+=1   
    done
    ;;
    3)
    index=1
    for a in $para
    do
        if [ $index -gt 1 ]; then
            cat /dev/null > /home/redis/tmp_sentinel_monitor
            clustername=`expr $a - 20000`
            echo "sentinel monitor redis${clustername} 192.168.1.40 $a 2" >> /home/redis/tmp_sentinel_monitor
            echo "sentinel set redis${clustername} auth-pass 123456" >> /home/redis/tmp_sentinel_monitor
            echo "sentinel set redis${clustername} down-after-milliseconds 5000" >> /home/redis/tmp_sentinel_monitor
            echo "sentinel set redis${clustername} failover-timeout 10000" >> /home/redis/tmp_sentinel_monitor

            cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001
            cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001
            cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001
  
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001 info | grep $a
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001 info | grep $a
            /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001 info | grep $a
        fi
        let index+=1   
    done
    ;;
    esac
}

get_char()
{
    SAVEDSTTY=`stty -g`
    stty -echo
    stty cbreak
    dd if=/dev/tty bs=1 count=1 2> /dev/null
    stty -raw
    stty echo
    stty $SAVEDSTTY
}

echo "腳本執行步驟："
echo -e "1. 查看key數量\n2. 去掉老哨兵監控\n3. 停止新實例從老實例的複製\n4. 添加新哨兵監控\n5. 重啓新哨兵\n6. 添加老哨兵監控\n7. 重啓老哨兵"

echo
echo -e "\n按任意鍵繼續 1. 查看key數量 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 查看key數量
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p $1 -a 123456 info 2>/dev/null | grep db1:keys=
 
new_repl 1

echo
echo -e "\n按任意鍵繼續 2. 去掉老哨兵監控 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 去掉老哨兵監控並確認
master_info=`/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $1`
master_name=`echo $master_info | awk -F"," '{print $1}' | awk -F"=" '{print $2}'`
master_ip=`echo $master_info | awk -F"," '{print $3}' | awk -F"=" '{print $2}' | awk -F":" '{print $1}'`

/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 sentinel remove $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 sentinel remove $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 sentinel remove $master_name
 
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep $master_name

echo
echo -e "\n按任意鍵繼續 3. 停止新實例從老實例的複製 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 停止新實例從老實例的複製並確認
new_repl 2

echo
echo -e "\n按任意鍵繼續 4. 添加新哨兵監控 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 添加新哨兵監控並確認
new_repl 3

echo
echo -e "\n按任意鍵繼續 5. 重啓新哨兵 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 重啓新哨兵並確認
/root/restart_sentinel.sh
ssh 192.168.1.40 /root/restart_sentinel.sh
ssh 192.168.1.41 /root/restart_sentinel.sh

wait
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.39 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.40 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.41 -p 30001 info | grep master

echo
echo -e "\n按任意鍵繼續 6. 添加老哨兵監控 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 添加老哨兵監控並確認
cat /dev/null > /home/redis/tmp_sentinel_monitor
echo "sentinel monitor ${master_name} ${master_ip} $1 2" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} auth-pass 123456" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} down-after-milliseconds 5000" >> /home/redis/tmp_sentinel_monitor
echo "sentinel set ${master_name} failover-timeout 10000" >> /home/redis/tmp_sentinel_monitor

cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001
cat /home/redis/tmp_sentinel_monitor | /home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001

/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep $master_name
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep $master_name

echo
echo -e "\n按任意鍵繼續 7. 重啓老哨兵 ...，組合鍵 CTRL+C 終止命令！\n"
char=`get_char`

# 重啓老哨兵並確認
echo "重啓老哨兵並確認，在192.168.1.36上執行/root/restart_all_sentinel.sh"

        restart_sentinel.sh 文件內容如下： 

sudo -u redis /home/redis/redis-5.0.3/src/redis-cli -p 30001 -a 123456 shutdown
sudo -u redis /home/redis/redis-5.0.3/src/redis-sentinel /home/redis/sentinel/sentinel.conf > /home/redis/sentinel/sentinel.log 2>&1 &

        restart_all_sentinel.sh 文件內容如下： 

#!/bin/bash

# 重啓本機哨兵
/root/restart_sentinel.sh

# 重啓遠程哨兵
ssh 192.168.1.37 /root/restart_sentinel.sh
ssh 192.168.1.38 /root/restart_sentinel.sh

wait
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.36 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.37 -p 30001 info | grep master
/home/redis/redis-5.0.3/src/redis-cli -h 192.168.1.38 -p 30001 info | grep master

        如將20011拆分爲兩個新的redis實例20006、20007：

manual_shard.sh 20011 20006 20007

 

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

定期全備redis

        首先申明我的觀點，redis本身只是緩存，不適合作爲數據庫使用，有說微博就是拿redis當DB用的，自己去證實吧。如果非要拿redis當數據庫，就不得不考慮數據丟失問題，這裏討論兩種常見的可能造成數據丟失的情況。    

wzy0623 2020-07-06 18:32:22

批量取redis key/value和批量刪除key

wzy0623 2020-02-24 09:59:26

定期全備redis

        首先申明我的觀點，redis本身只是緩存，不適合作爲數據庫使用，有說微博就是拿redis當DB用的，自己去證實吧。如果非要拿redis當數據庫，就不得不考慮數據丟失問題，這裏討論兩種常見的可能造成數據丟失的情況。    

wzy0623 2020-07-06 18:32:22

批量取redis key/value和批量刪除key

wzy0623 2020-02-24 09:59:26

爲什麼 redis 單線程卻能支撐高併發

wzy0623 2020-02-24 09:59:26

Redis Cluster FlushAll失敗

wzy0623 2019-06-11 14:56:09

初學乍練redis：Redis 5 Cluster手工指定主從關係

wzy0623 2019-04-21 18:58:42

初學乍練redis：Redis 5.0.3單實例數據遷移到Cluster

wzy0623 2019-04-20 11:39:49

初學乍練redis：使用rdr查看Redis key內存使用

wzy0623 2019-04-19 06:26:53