突然想到項目中每天要生產各個環境的索引...時間一長會越來越多.
這裏寫一個定時自動清理elasticsearch的腳本
首先.我們進入到elasticsearch的索引目錄中去 發現索引數據是真多啊...
root@iZbp1ig4nql7imcw4ssn7vZ:/home/idouall# cd /opt/elasticsearch-6.4.3/data/nodes/0/indices/
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices# pwd
/opt/elasticsearch-6.4.3/data/nodes/0/indices
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices# ll
total 140
drwxrwxr-x 35 idouall idouall 4096 Feb 27 16:32 ./
drwxrwxr-x 4 idouall idouall 4096 Feb 20 21:42 ../
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 335HxTLoTEyVGtNBI3QnUw/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 18:06 8vzxHNv3SOSYEmyleo6G-A/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 93mpD1vsQ2OeiZoTVHw4WA/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 15:10 9OhuYCcKRoywBY-9RcfWLg/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 ahtHtEOoQ32tKLys8-PPfQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 15:08 AmsG8zAZRv2-mxhQe6se6g/
drwxrwxr-x 8 idouall idouall 4096 Feb 25 09:15 bfhdi1FMRIOwQ1m1KtKGXA/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 09:41 ChxRHY7SQsGBqbDI3q7AEg/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 15:46 CKAnJBrdQd2A_gT4YW898g/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 dEQDNpMlRtidwIaoZbWXuQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 15:46 Dsct-CMWQK6Fd6mdRra-Yg/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 08:00 DuoIvH0sRcmwcqFPJrUWOw/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 FRHlFniRS8CprAwM-RHfqw/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 16:45 IrtW2PWqSc2s1L8OSVJh0A/
drwxrwxr-x 8 idouall idouall 4096 Feb 25 09:15 JNaCO72jSrKVWCpVUowVjA/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 17:25 JOONjvP2QSavERnM1oLIEA/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 k3E5EExhSBKf2uR97AKVVw/
drwxrwxr-x 8 idouall idouall 4096 Feb 25 09:00 K-G-bjDkRVaw_k6gTXsE0w/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 17:12 kKlZrmUPSqerTPckUfUlZw/
drwxrwxr-x 8 idouall idouall 4096 Feb 22 17:32 LBui4Vx_TEmXgxMJD7ZbeA/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 17:12 n2Y43XW8Q_KjsBlZyJG-yQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 15:46 pN94eztrSTiMM_mFHe_0ag/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 18:06 pzxyRtccRaehjmKAGH4SKA/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 16:54 rL2GRdr-TIShYkX0G5W-QQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 24 17:12 sCQtad6lT1ijTozuvynSEQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 08:00 uLW0N7PoTo-aKjoHbocOag/
drwxrwxr-x 4 idouall idouall 4096 Feb 20 21:42 uPF1CNvbRYmuBgXDRaHb0Q/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 10:09 Uz9OpsQ0SB6Joa2o6IdNbg/
drwxrwxr-x 8 idouall idouall 4096 Feb 26 22:52 V-7cpx38TKempGZG9rdEJQ/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 vt6rw-XVReeDTqWnskD5Mw/
drwxrwxr-x 8 idouall idouall 4096 Feb 25 08:00 x0Cc53-OSQOCddvvsrwEPA/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 10:19 yCwUDpaiTA2VKVbJhJ-9tw/
drwxrwxr-x 8 idouall idouall 4096 Feb 27 16:13 YrqZzoiTSLCXaPbVmfzetQ/
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices#
但是都是看不懂的...我們利用curl調用elasticsearch的api進行索引的查看.
案例如下:
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices# curl -XGET 'http://101.37.32.7:9200/_cat/indices/?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open prodbacklog-2020.02.27 dEQDNpMlRtidwIaoZbWXuQ 5 1 219134 0 104.8mb 104.8mb
yellow open testlog-2020.02.25 K-G-bjDkRVaw_k6gTXsE0w 5 1 20988 0 18.8mb 18.8mb
yellow open testlog-2020.02.27 vt6rw-XVReeDTqWnskD5Mw 5 1 17668 0 19.7mb 19.7mb
yellow open uatbacklog-2020.02.27 335HxTLoTEyVGtNBI3QnUw 5 1 27694 0 4.9mb 4.9mb
yellow open uatlog-2020.02.26 rL2GRdr-TIShYkX0G5W-QQ 5 1 120 0 342.9kb 342.9kb
yellow open prodlog-2020.02.27 k3E5EExhSBKf2uR97AKVVw 5 1 30147 0 36.9mb 36.9mb
yellow open devbacklog-2020.02.24 JOONjvP2QSavERnM1oLIEA 5 1 48 0 199.9kb 199.9kb
yellow open testbacklog-2020.02.25 x0Cc53-OSQOCddvvsrwEPA 5 1 1390581 0 108.9mb 108.9mb
green open .kibana uPF1CNvbRYmuBgXDRaHb0Q 1 0 31 1 69.8kb 69.8kb
yellow open devlog-2020.02.25 JNaCO72jSrKVWCpVUowVjA 5 1 138 0 339.8kb 339.8kb
yellow open taskrecordaudiobranchwords pN94eztrSTiMM_mFHe_0ag 5 1 6 0 99.9kb 99.9kb
yellow open devbacklog-2020.02.26 ChxRHY7SQsGBqbDI3q7AEg 5 1 1956 0 2.4mb 2.4mb
yellow open devbacklog-2020.02.25 bfhdi1FMRIOwQ1m1KtKGXA 5 1 1184 0 2mb 2mb
yellow open alllogback-2020.02.26 V-7cpx38TKempGZG9rdEJQ 5 1 878 0 245.9kb 245.9kb
yellow open devlog-2020.02.24 kKlZrmUPSqerTPckUfUlZw 5 1 62 0 168.3kb 168.3kb
yellow open testlog-2020.02.26 uLW0N7PoTo-aKjoHbocOag 5 1 9480 0 8mb 8mb
yellow open prodlog-2020.02.26 9OhuYCcKRoywBY-9RcfWLg 5 1 536 0 833.5kb 833.5kb
yellow open devlog-2020.02.22 LBui4Vx_TEmXgxMJD7ZbeA 5 1 53 0 245.2kb 245.2kb
yellow open devlog-2020.02.26 Uz9OpsQ0SB6Joa2o6IdNbg 5 1 141 0 474kb 474kb
yellow open testbacklog-2020.02.27 yCwUDpaiTA2VKVbJhJ-9tw 5 1 2120884 0 168.3mb 168.3mb
yellow open testlog-2020.02.24 n2Y43XW8Q_KjsBlZyJG-yQ 5 1 4503 0 4.8mb 4.8mb
yellow open devlog-2020.02.27 FRHlFniRS8CprAwM-RHfqw 5 1 194 0 327.2kb 327.2kb
yellow open uatbacklog-2020.02.26 IrtW2PWqSc2s1L8OSVJh0A 5 1 9394 0 2mb 2mb
yellow open taskrecordaudiotranslate CKAnJBrdQd2A_gT4YW898g 5 1 0 0 1.2kb 1.2kb
yellow open uatlog-2020.02.27 93mpD1vsQ2OeiZoTVHw4WA 5 1 306 0 962.3kb 962.3kb
yellow open test 8vzxHNv3SOSYEmyleo6G-A 5 1 0 0 1.2kb 1.2kb
yellow open testbacklog-2020.02.26 DuoIvH0sRcmwcqFPJrUWOw 5 1 582574 0 46.5mb 46.5mb
yellow open tasksurveyandmissingimagedto Dsct-CMWQK6Fd6mdRra-Yg 5 1 35 0 879.3kb 879.3kb
yellow open alllogback-2020.02.27 YrqZzoiTSLCXaPbVmfzetQ 5 1 1014962 0 142.2mb 142.2mb
yellow open province pzxyRtccRaehjmKAGH4SKA 5 1 0 0 1.2kb 1.2kb
yellow open devbacklog-2020.02.27 ahtHtEOoQ32tKLys8-PPfQ 5 1 1225 0 3.3mb 3.3mb
yellow open testbacklog-2020.02.24 sCQtad6lT1ijTozuvynSEQ 5 1 869793 0 66.2mb 66.2mb
yellow open prodbacklog-2020.02.26 AmsG8zAZRv2-mxhQe6se6g 5 1 29892 0 5.7mb 5.7mb
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices#
是不是清晰多了....然後我們利用刪除命令進行刪除一個試試. 如下證明刪除成功.
root@iZbp1ig4nql7imcw4ssn7vZ:/opt/elasticsearch-6.4.3/data/nodes/0/indices# curl -XDELETE 'http://101.37.32.7:9200/devlog-*'
{"acknowledged":true}
那麼有了刪除命令....我們寫一個刪除腳本利用上面的命令進行批量的模糊刪除就行了.
我們在/home/idouall下面建立es-index-delete.sh腳本,
touch es-index-delete.sh
修改權限.
chmod 777 es-index-delete.sh
然後寫入腳本如下:
vim es-index-delete.sh
#/bin/bash
#指定日期(7天前)
DATA=`date -d "1 week ago" +%Y.%m.%d`
#當前日期
time=`date`
#刪除7天前的日誌
curl -XDELETE http://101.37.32.7:9200/testlog-${DATA}
curl -XDELETE http://101.37.32.7:9200/devlog-${DATA}
curl -XDELETE http://101.37.32.7:9200/prodlog-${DATA}
curl -XDELETE http://101.37.32.7:9200/uatlog-${DATA}
curl -XDELETE http://101.37.32.7:9200/devbacklog-${DATA}
curl -XDELETE http://101.37.32.7:9200/testbacklog-${DATA}
curl -XDELETE http://101.37.32.7:9200/uatbacklog-${DATA}
curl -XDELETE http://101.37.32.7:9200/prodbacklog-${DATA}
curl -XDELETE http://101.37.32.7:9200/alllogback-${DATA}
if [ $? -eq 0 ];then
echo $time"-->del $DATA log success.." >> /home/idouall/es-index-sucess-clear.log
else
echo $time"-->del $DATA log fail.." >> /home/idouall/es-index-fail-clear.log
fi
接下來我將腳本加入linux的定時任務中
執行
crontab -e
0 1 1 * * /script/es-index-delete.sh
然後執行crontab -l查看是否添加成功.