通過xargs實現redis cluster批量keys操作
爲什麼會出現批量刪除keys的腳本，那是因爲redis不支持刪除（DEL）帶有通配符“*”的key，而查詢支持帶通配符“*”。
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD KEYS hello*
1) "hello8"
2) "hello5"
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD DEL hello*
(integer) 0工具簡介
xargs
This manual page documents the GNU version of xargs. xargs reads items from the standard input, delimited by blanks (which can be protected with double or single quotes or a backslash) or newlines, and executes the command (default is /bin/echo) one or more times with any initial-arguments followed by items read from standard input. Blank lines on the standard input are ignored.
The command line for command is built up until it reaches a system-defined limit (unless the -n and -L options are used). The specified com‐mand will be invoked as many times as necessary to use up the list of input items. In general, there will be many fewer invocations of com‐mand than there were items in the input. This will normally have significant performance benefits. Some commands can usefully be executed in parallel too; see the -P option.
Because Unix filenames can contain blanks and newlines, this default behaviour is often problematic; filenames containing blanks and/or new‐lines are incorrectly processed by xargs. In these situations it is better to use the -0 option, which prevents such problems. When using this option you will need to ensure that the program which produces the input for xargs also uses a null character as a separator. If that program is GNU find for example, the -print0 option does this for you.
If any invocation of the command exits with a status of 255, xargs will stop immediately without reading any further input. An error message is issued on stderr when this happens.redis-cli
Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]
-h <hostname> Server hostname (default: 127.0.0.1).
-p <port> Server port (default: 6379).
-s <socket> Server socket (overrides hostname and port).
-a <password> Password to use when connecting to the server.
-r <repeat> Execute specified command N times.
-i <interval> When -r is used, waits <interval> seconds per command.
It is possible to specify sub-second times like -i 0.1.
-n <db> Database number.
-x Read last argument from STDIN.
-d <delimiter> Multi-bulk delimiter in for raw formatting (default: \n).
-c Enable cluster mode (follow -ASK and -MOVED redirections).
--scan List all keys using the SCAN command.
--pattern <pat> Useful with --scan to specify a SCAN pattern.key-value操作
查詢
KEYS查詢
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD KEYS hello*
1) "hello8"
2) "hello5"
3) "hello1"GET值查詢
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD GET hello8
"955"
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD GET hello5
"555"
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD GET hello1
"999"SET重置
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD SET hello1 666
OK
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD GET hello1
"666"DEL刪除
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD DEL hello1
(integer) 1
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD GET hello1
(nil)
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD KEYS hello1
(empty list or set)通過xargs實現批量刪除
第一種方式：
redis-cli -a "password" -n 0 -p 6379 keys "*:login" | xargs -i redis-cli -a "password" -n 0 -p 6379 del {}[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 KEYS hello*
1) "hello8"
2) "hello5"
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 KEYS "hello*" | xargs -i redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 DEL {}
(integer) 1
(integer) 1
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 KEYS hello*
(empty list or set)這樣一個壞處每次都要建立一個連接，量小的話還可以接受，量大的話，效率不行。
第二種方式：
自從redis2.8以後就開始支持scan命令，模式匹配可以採取下面的形式來批刪除大量的key
redis-cli -a "password" -n 0 -p 6379 --scan --pattern "*:login" | xargs -i redis-cli -a "password" -n 0 -p 6379 DEL {}[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 --scan --pattern "hello*"
hello1
hello4
hello5
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 --scan --pattern "hello*" | xargs -i redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 DEL {}
(integer) 1
(integer) 1
(integer) 1
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 --scan --pattern "hello*"
[root@localhost ~]# redis-cli -c -h 10.1.37.113 -p 7002 -a PASSWORD -n 0 KEYS "hello*"
(empty list or set)速度處理也是非常快的。
第三種方式：
redis-cli -c -h $ip -p $port keys $pkey | xargs -r -t -n1 redis-cli -c -h $ip -p $port del批量刪除kyes腳本
腳本造key測試
#!/bin/bash
redis_list=("10.1.37.113:7002")
pkey_list=("ValuationRuleSummary" "ValuationRuleDetail" "MerchantValuation" "QueryValuationRules" "GetMerchantByUserId")
for info in ${redis_list[@]}
do
echo "開始執行:$info"
ip=`echo $info | cut -d \: -f 1`
port=`echo $info | cut -d \: -f 2`
for pkey in ${pkey_list[@]}
do
for i in `seq 0 10000`
do
echo $pkey"_"$i $i
redis-cli -c -h $ip -p $port -a PASSWORD SET $pkey"_"$i $i
done
done
done
echo "完成"第一種批量刪除方式
#!/bin/bash
redis_list=("10.1.37.113:7000" "10.1.37.113:7001" "10.1.37.113:7002" "10.1.37.113:7003" "10.1.37.113:7004" "10.1.37.113:7005")
pkey_list=("ValuationRuleSummary*" "ValuationRuleDetail*" "MerchantValuation*" "QueryValuationRules*" "GetMerchantByUserId*")
for info in ${redis_list[@]}
do
echo "開始執行:$info"
ip=`echo $info | cut -d \: -f 1`
port=`echo $info | cut -d \: -f 2`
for pkey in ${pkey_list[@]}
do
redis-cli -c -h $ip -p $port -a PASSWORD KEYS $pkey | xargs -n 1 -t -i redis-cli -c -h $ip -p $port -a PASSWORD DEL {}
done
done
echo "完成"通過keys文件列表操作
key.txt –待刪除的key
key1
key2
key3redis_delete_key.sh
redis_list 填入集羣節點
redis_list=("127.0.0.1:6379" "127.0.0.1:6380")
for info in ${redis_list[@]}
do
echo "開始執行:$info"
ip=`echo $info | cut -d \: -f 1`
port=`echo $info | cut -d \: -f 2`
cat key.txt |xargs -t -n1 redis-cli -h $ip -p $port -c del
done
echo "完成"總結
這種方式怎麼刪除還是要看xargs怎麼使用了，xargs用什麼參數看自己需求，詳細請man查看。
但是這種方式還是比較慢，當keys非常巨大時，就能體現出來有多慢了，有可能你刪一天都刪不完，詳細可以查看redis DEL 命令的時間複雜度。
要想加快速度怎麼辦？還得使用程序進行，或者使用 Lua 腳本等方法，畢竟shell腳本還是一條一條去刪除的。請大佬看到多多批評指正，獻醜了！