一、說明
redis 3.0集羣功能出來已經有一段時間了，目前最新穩定版是3.0.5，我瞭解到已經有很多互聯網公司在生產環境使用，比如唯品會、美團等等，剛好公司有個新項目，預估的量單機redis無法滿足，開發又不想在代碼層面做拆分，所以就推薦他們嘗試一下redis集羣，下面做了一些相關筆記，以備後用
二、環境
1、redis節點
10.10.2.70:6300 10.10.2.70:6301 主從 10.10.2.71:6300 10.10.2.71:6301 主從 10.10.2.85:6300 10.10.2.85:6301 主從
2、redis版本
Redis version 3.0.5
三、安裝配置
1、安裝redis
wget http://download.redis.io/releases/redis-3.0.5.tar.gz tar -zxvf redis-3.0.5.tar.gz cd redis-3.0.5 make cp redis-3.0.5/src/redis-trib.rb /bin/ cp redis-3.0.5/src/redis-server /bin/ cp redis-3.0.5/src/redis-cli /bin/
2、安裝ruby及ruby的redis模塊
yum -y install ruby rubygems gem install redis --version 3.0.5
3、內核調優
echo never > /sys/kernel/mm/transparent_hugepage/enabled echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf sysctl -p
4、建立目錄
mkdir /data/redis/6300 -p mkdir /data/redis/6301
5、撰寫redis配置文件（cp配置文件注意修改端口）
vim /etc/redis_6300.conf daemonize yes port 6300 tcp-backlog 511 timeout 0 tcp-keepalive 0 loglevel notice maxmemory 10gb databases 16 dir /data/redis/6300 slave-serve-stale-data yes loglevel notice logfile "/data/redis/6300/redis_6300.log" #slave只讀 slave-read-only yes #not use default repl-disable-tcp-nodelay yes slave-priority 100 #打開aof持久化 appendonly yes #每秒一次aof寫 appendfsync everysec #關閉在aof rewrite的時候對新的寫操作進行fsync no-appendfsync-on-rewrite yes auto-aof-rewrite-min-size 64mb lua-time-limit 5000 #打開redis集羣 cluster-enabled yes cluster-config-file /data/redis/6300/nodes-6300.conf #節點互連超時的閥值（單位毫秒） cluster-node-timeout 15000 #一個主節點在擁有多少個好的從節點的時候就要割讓一個從節點出來給其他沒有從節點或者從節點掛掉的主節點 cluster-migration-barrier 1 #如果某一些key space沒有被集羣中任何節點覆蓋，最常見的就是一個node掛掉，集羣將停止接受寫入 cluster-require-full-coverage no #部署在同一機器的redis實例，把auto-aof-rewrite搓開，防止瞬間fork所有redis進程做rewrite,佔用大量內存 auto-aof-rewrite-percentage 80-100 slowlog-log-slower-than 10000 slowlog-max-len 128 notify-keyspace-events "" hash-max-ziplist-entries 512 hash-max-ziplist-value 64 list-max-ziplist-entries 512 list-max-ziplist-value 64 set-max-intset-entries 512 zset-max-ziplist-entries 128 zset-max-ziplist-value 64 activerehashing yes client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 hz 10 aof-rewrite-incremental-fsync yes
6、啓動服務
redis-server /etc/redis_6300.conf redis-server /etc/redis_6301.conf echo "redis-server /etc/redis_6300.conf" >> /etc/rc.local echo "redis-server /etc/redis_6301.conf" >> /etc/rc.local
7、初始化集羣
#節點角色由順序決定,先master之後是slave，本文中6300是master，6301是slave redis-trib.rb create --replicas 1 10.10.2.70:6300 10.10.2.71:6300 10.10.2.85:6300 10.10.2.70:6301 10.10.2.71:6301 10.10.2.85:6301
8、查看集羣狀態
redis-trib.rb check 10.10.2.70:6300
PS：
redis-trib.rb是一個ruby工具，封裝了redis集羣的一些命令，用這個工具操作集羣非常方便，比如上面初始化集羣，查看集羣狀態，還有添加、刪除節點，遷移slot等等功能
四、redis集羣維護
A、場景1
線上的集羣已經有瓶頸，集羣需要擴容，比如我們已經準備了一主一從（10.10.2.85:6302、10.10.2.85:6303），如下：
1、添加一個主節點
[root@yw_0_0 ~]# redis-trib.rb add-node 10.10.2.85:6302 10.10.2.70:6300 >>> Adding node 10.10.2.85:6302 to cluster 10.10.2.70:6300 Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6301: OK >>> Performing Cluster Check (using node 10.10.2.70:6300) S: cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slots: (0 slots) slave replicates 85412cf3d8e69354115fc0991f470b32b9213cd7 M: 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 10.10.2.85:6300 slots:0-5460 (5461 slots) master 1 additional replica(s) S: a74642c0fbc98f921be477eabcdd22eccd89891f 10.10.2.85:6301 slots: (0 slots) slave replicates 2568dbd91fffa16ff93ea8db19275fd7ec8af41a M: 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: 22d2dec483824b84571a60e8c037fff957615552 10.10.2.71:6301 slots: (0 slots) slave replicates 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. Connecting to node 10.10.2.85:6302: OK >>> Send CLUSTER MEET to node 10.10.2.85:6302 to make it join the cluster. [OK] New node added correctly.
10.10.2.85:6302是要加的新節點，10.10.2.70:6300是集羣中已存在的任意節點
2、給主節點添加從節點
[root@yw_0_0 ~]# redis-trib.rb add-node --slave --master-id 5ef18f95f75756891aa948ea1f200044f1d3947c 10.10.2.85:6303 10.10.2.70:6300 >>> Adding node 10.10.2.85:6303 to cluster 10.10.2.70:6300 Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.85:6302: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6301: OK >>> Performing Cluster Check (using node 10.10.2.70:6300) S: cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slots: (0 slots) slave replicates 85412cf3d8e69354115fc0991f470b32b9213cd7 M: 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 10.10.2.85:6300 slots:0-5460 (5461 slots) master 1 additional replica(s) M: 5ef18f95f75756891aa948ea1f200044f1d3947c 10.10.2.85:6302 slots: (0 slots) master 0 additional replica(s) S: a74642c0fbc98f921be477eabcdd22eccd89891f 10.10.2.85:6301 slots: (0 slots) slave replicates 2568dbd91fffa16ff93ea8db19275fd7ec8af41a M: 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: 22d2dec483824b84571a60e8c037fff957615552 10.10.2.71:6301 slots: (0 slots) slave replicates 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. Connecting to node 10.10.2.85:6303: OK >>> Send CLUSTER MEET to node 10.10.2.85:6303 to make it join the cluster. Waiting for the cluster to join. >>> Configure node as replica of 10.10.2.85:6302. [OK] New node added correctly.
--slave 指定要加的是從節點，--master-id 指定這個從節點的主節點ID，10.10.2.85:6303是需要新加的從節點，10.10.2.70:6300是集羣已存在的任意節點
3、遷移一些slot給新節點
[root@yw_0_0 ~]# redis-trib.rb reshard 10.10.2.70:6300 Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.85:6303: OK Connecting to node 10.10.2.85:6302: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6301: OK >>> Performing Cluster Check (using node 10.10.2.70:6300) S: cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slots: (0 slots) slave replicates 85412cf3d8e69354115fc0991f470b32b9213cd7 M: 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 10.10.2.85:6300 slots:0-5460 (5461 slots) master 1 additional replica(s) S: fc90d090fae909fd4f962752941c039d081d3854 10.10.2.85:6303 slots: (0 slots) slave replicates 5ef18f95f75756891aa948ea1f200044f1d3947c M: 5ef18f95f75756891aa948ea1f200044f1d3947c 10.10.2.85:6302 slots: (0 slots) master 1 additional replica(s) S: a74642c0fbc98f921be477eabcdd22eccd89891f 10.10.2.85:6301 slots: (0 slots) slave replicates 2568dbd91fffa16ff93ea8db19275fd7ec8af41a M: 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slots:10923-16383 (5461 slots) master 1 additional replica(s) S: 22d2dec483824b84571a60e8c037fff957615552 10.10.2.71:6301 slots: (0 slots) slave replicates 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 3000 #設置需要把3000個slot做移動 What is the receiving node ID? 5ef18f95f75756891aa948ea1f200044f1d3947c #設置接收這3000個slot的節點ID，也就是剛纔新加的10.10.2.85:6302的ID Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1:85412cf3d8e69354115fc0991f470b32b9213cd7 #設置這3000slot的來源ID，這裏我從集羣之前的3個節點分別去取一部分slot Source node #2:6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 #設置這3000slot的來源ID，這裏我從集羣之前的3個節點分別去取一部分slot Source node #3:2568dbd91fffa16ff93ea8db19275fd7ec8af41a #設置這3000slot的來源ID，這裏我從集羣之前的3個節點分別去取一部分slot Source node #4:done #輸入done開始做一些初始化操作 此處省略 Do you want to proceed with the proposed reshard plan (yes/no)? yes 輸入yes確認開始遷移slot
B、場景二
上面的例子是集羣擴容，相對的，由於各種原因集羣可能也需要縮容，下面的例子把上文擴容的節點下線，步驟如下：
1、遷移這個節點的slot到其他節點（有slot的節點是不可以直接下線的）
[root@yw_0_0 ~]# redis-trib.rb reshard 10.10.2.70:6300 Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.85:6303: OK Connecting to node 10.10.2.85:6302: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6301: OK >>> Performing Cluster Check (using node 10.10.2.70:6300) S: cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slots: (0 slots) slave replicates 85412cf3d8e69354115fc0991f470b32b9213cd7 M: 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 10.10.2.85:6300 slots:999-5460 (4462 slots) master 1 additional replica(s) S: fc90d090fae909fd4f962752941c039d081d3854 10.10.2.85:6303 slots: (0 slots) slave replicates 5ef18f95f75756891aa948ea1f200044f1d3947c M: 5ef18f95f75756891aa948ea1f200044f1d3947c 10.10.2.85:6302 slots:0-998,5461-6461,10923-11921 (2999 slots) master 1 additional replica(s) S: a74642c0fbc98f921be477eabcdd22eccd89891f 10.10.2.85:6301 slots: (0 slots) slave replicates 2568dbd91fffa16ff93ea8db19275fd7ec8af41a M: 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 slots:6462-10922 (4461 slots) master 1 additional replica(s) M: 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slots:11922-16383 (4462 slots) master 1 additional replica(s) S: 22d2dec483824b84571a60e8c037fff957615552 10.10.2.71:6301 slots: (0 slots) slave replicates 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. How many slots do you want to move (from 1 to 16384)? 3000 #上文給這個節點遷入了3000個slot，所以這裏還選擇遷出3000個slot What is the receiving node ID? 85412cf3d8e69354115fc0991f470b32b9213cd7 #接收這3000slot節點的主ID Please enter all the source node IDs. Type 'all' to use all the nodes as source nodes for the hash slots. Type 'done' once you entered all the source nodes IDs. Source node #1:5ef18f95f75756891aa948ea1f200044f1d3947c #要下線節點的主ID Source node #4:done 此處省略 Do you want to proceed with the proposed reshard plan (yes/no)?yes
2、然後查看10.10.2.85:6302這個maser上已經沒有slot了
10.10.2.71:6300> cluster nodes 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 master - 0 1445853133399 12 connected 0-999 6462-7460 10923-16383 22d2dec483824b84571a60e8c037fff957615552 10.10.2.71:6301 slave 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 0 1445853132898 10 connected 6bea6afa2ee8dfb0cc3c96f804eb3fa77ce98013 10.10.2.85:6300 master - 0 1445853134400 10 connected 1000-5461 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 myself,master - 0 0 11 connected 5462-6461 7461-10922 cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slave 85412cf3d8e69354115fc0991f470b32b9213cd7 0 1445853131395 12 connected fc90d090fae909fd4f962752941c039d081d3854 10.10.2.85:6303 slave 5ef18f95f75756891aa948ea1f200044f1d3947c 0 1445853133899 8 connected a74642c0fbc98f921be477eabcdd22eccd89891f 10.10.2.85:6301 slave 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 0 1445853129394 11 connected 5ef18f95f75756891aa948ea1f200044f1d3947c 10.10.2.85:6302 master - 0 1445853132397 8 connected
3、下線slave節點
[root@yw_0_0 ~]# redis-trib.rb del-node 10.10.2.85:6303 fc90d090fae909fd4f962752941c039d081d3854 >>> Removing node fc90d090fae909fd4f962752941c039d081d3854 from cluster 10.10.2.85:6303 Connecting to node 10.10.2.85:6303: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.85:6302: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.71:6301: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6300: OK >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
4、下線master節點
redis-trib.rb del-node 10.10.2.70:6301 5ef18f95f75756891aa948ea1f200044f1d3947c >>> Removing node 5ef18f95f75756891aa948ea1f200044f1d3947c from cluster 10.10.2.70:6301 Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6301: OK Connecting to node 10.10.2.85:6302: OK Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
C、場景三
集羣中一個節點的master掛掉，從節點提升爲主節點，還沒有來的急給這個新的主節點加從節點，這個新的主節點就又掛掉了，那麼集羣中這個節點就徹底不可以用了，爲了解決這個問題，我們至少保證每個節點的maser下面有兩個以上的從節點，這樣一來，需要的內存資源或者服務器資源就翻倍了，有沒有一個折中的方法呢，答案是肯定的，還節點上文配置文件中的cluster-migration-barrier參數不，我們只需要給集羣中其中一個節點的master掛多個從庫，當其他節點的master下沒有可用的從庫時，有多個從庫的master會割讓一個slave給他，保證整個集羣的可用性
1、給10.10.2.70:6300 10.10.2.70:6301 這組節點下面加一個從庫10.10.2.85:6302
[root@yw_0_0 ~]# redis-trib.rb add-node --slave --master-id cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.85:6302 10.10.2.70:6300 >>> Adding node 10.10.2.85:6302 to cluster 10.10.2.70:6300 Connecting to node 10.10.2.70:6300: OK Connecting to node 10.10.2.85:6300: OK Connecting to node 10.10.2.71:6300: OK Connecting to node 10.10.2.70:6301: OK Connecting to node 10.10.2.85:6301: OK Connecting to node 10.10.2.71:6301: OK >>> Performing Cluster Check (using node 10.10.2.70:6300) M: cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 slots:3000-5461,6462-7460,10923-16383 (8922 slots) master 1 additional replica(s) M: e36cdef7a26ed59e8d9db2cf1dbc1997bfc9dfde 10.10.2.85:6300 slots:0-2999 (3000 slots) master 1 additional replica(s) M: 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 slots:5462-6461,7461-10922 (4462 slots) master 1 additional replica(s) S: 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slots: (0 slots) slave replicates cd1f2c1f348bb4359337e7462c1e21dc82f1551b S: 89fcc4994a99ed2fe9bbb908c58dfda2cf31e7d2 10.10.2.85:6301 slots: (0 slots) slave replicates e36cdef7a26ed59e8d9db2cf1dbc1997bfc9dfde S: 1f3ea36eacbe005a4b9ac52aeef6d83337dac051 10.10.2.71:6301 slots: (0 slots) slave replicates 2568dbd91fffa16ff93ea8db19275fd7ec8af41a [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. Connecting to node 10.10.2.85:6302: OK >>> Send CLUSTER MEET to node 10.10.2.85:6302 to make it join the cluster. Waiting for the cluster to join. >>> Configure node as replica of 10.10.2.70:6300. [OK] New node added correctly.
2、把10.10.2.71:6300 10.10.2.71:6301這組的從節點停掉
redis-cli -h 10.10.2.71 -p 6301 shutdown
3、查看10.10.2.85:6302這個節點是否成爲10.10.2.71：6300的從庫
10.10.2.71:6300> CLUSTER nodes 85412cf3d8e69354115fc0991f470b32b9213cd7 10.10.2.70:6301 slave cd1f2c1f348bb4359337e7462c1e21dc82f1551b 0 1445911596844 17 connected 89fcc4994a99ed2fe9bbb908c58dfda2cf31e7d2 10.10.2.85:6301 slave e36cdef7a26ed59e8d9db2cf1dbc1997bfc9dfde 0 1445911594841 20 connected 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 10.10.2.71:6300 myself,master - 0 0 11 connected 5462-6461 7461-10922 cd1f2c1f348bb4359337e7462c1e21dc82f1551b 10.10.2.70:6300 master - 0 1445911593839 17 connected 3000-5461 6462-7460 10923-16383 2b34532cd6937063d1da26cd4652881b73d97a06 10.10.2.85:6302 slave 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 0 1445911592838 17 connected #已成功掛到了10.10.2.71:6300下 1f3ea36eacbe005a4b9ac52aeef6d83337dac051 10.10.2.71:6301 slave,fail 2568dbd91fffa16ff93ea8db19275fd7ec8af41a 1445911561982 1445911559778 11 disconnected e36cdef7a26ed59e8d9db2cf1dbc1997bfc9dfde 10.10.2.85:6300 master - 0 1445911595843 20 connected 0-2999
五、cluster相關命令
集羣 CLUSTER INFO 打印集羣的信息 CLUSTER NODES 列出集羣當前已知的所有節點（node），以及這些節點的相關信息。 節點 CLUSTER MEET <ip> <port> 將 ip 和 port 所指定的節點添加到集羣當中，讓它成爲集羣的一份子。 CLUSTER FORGET <node_id> 從集羣中移除 node_id 指定的節點。 CLUSTER REPLICATE <node_id> 將當前節點設置爲 node_id 指定的節點的從節點。 CLUSTER SAVECONFIG 將節點的配置文件保存到硬盤裏面。 槽(slot) CLUSTER ADDSLOTS <slot> [slot ...] 將一個或多個槽（slot）指派（assign）給當前節點。 CLUSTER DELSLOTS <slot> [slot ...] 移除一個或多個槽對當前節點的指派。 CLUSTER FLUSHSLOTS 移除指派給當前節點的所有槽，讓當前節點變成一個沒有指派任何槽的節點。 CLUSTER SETSLOT <slot> NODE <node_id> 將槽 slot 指派給 node_id 指定的節點，如果槽已經指派給另一個節點，那麼先讓另一個節點刪除該槽>，然後再進行指派。 CLUSTER SETSLOT <slot> MIGRATING <node_id> 將本節點的槽 slot 遷移到 node_id 指定的節點中。 CLUSTER SETSLOT <slot> IMPORTING <node_id> 從 node_id 指定的節點中導入槽 slot 到本節點。 CLUSTER SETSLOT <slot> STABLE 取消對槽 slot 的導入（import）或者遷移（migrate）。 鍵 CLUSTER KEYSLOT <key> 計算鍵 key 應該被放置在哪個槽上。 CLUSTER COUNTKEYSINSLOT <slot> 返回槽 slot 目前包含的鍵值對數量。 CLUSTER GETKEYSINSLOT <slot> <count> 返回 count 個 slot 槽中的鍵。
參考文章:
http://www.redis.cn/topics/cluster-tutorial.html
http://www.redis.cn/topics/cluster-spec.html
http://redisdoc.com/topic/cluster-tutorial.html