最近在工作中,遇到了IO使用率過高告警。但是移動的系統又很嚴格,不能隨便安裝監控開源軟件。這裏我只能通過向方法使用腳本去完成監控。
監控方法如下:
1、安裝iotop軟件(centos7默認是有這個包):
2、使用iotop -obt >> testfile.txt 重定向到testfile文件(意思是將實時在使用磁盤的程序顯示):
3、自己寫個腳本查看監控情況,寫好腳本。我這裏抓取的信息是時間,磁盤讀寫轉數、使用率、進程名。然後寫到計劃任務裏面去就可以去做日誌提取了。(提取日誌自己研究了)
[root@master ~]# cat iotest.sh
#!/bin/bash
IFS=$'\n'
for i in $(grep `date +"%H:%M"` testfile.txt | grep "%" | awk '{if($11>=0.5) {print $1"\t",$5,$6"\t",$7,$8"\t",$9,$10"\t",$11,$12"\t",$13}}')
do
echo `date +"%Y-%m-%d"` $i
done
4、最後做個計劃任務每十秒去執行腳本(這個根據自己情況定);
* * * * * sleep 00; /root/iotest.sh >> iotest.txt &
* * * * * sleep 10; /root/iotest.sh >> iotest.txt &
* * * * * sleep 20; /root/iotest.sh >> iotest.txt &
* * * * * sleep 30; /root/iotest.sh >> iotest.txt &
* * * * * sleep 40; /root/iotest.sh >> iotest.txt &
* * * * * sleep 50; /root/iotest.sh >> iotest.txt &
5、測試每時每刻監控IO使用率情況:
[root@master ~]# ./iotest.sh
2018-12-14 00:40:00 0.00 B/s 0.00 B/s 0.00 % 0.65 % [kworker/0:1]
2018-12-14 00:40:04 0.00 B/s 0.00 B/s 0.00 % 0.66 % [kworker/0:1]
2018-12-14 00:40:06 0.00 B/s 0.00 B/s 0.00 % 0.54 % [kworker/0:1]
2018-12-14 00:40:10 0.00 B/s 0.00 B/s 0.00 % 0.96 % [kworker/0:1]
2018-12-14 00:40:23 0.00 B/s 0.00 B/s 0.00 % 1.13 % [kworker/0:1]
2018-12-14 00:40:24 0.00 B/s 0.00 B/s 0.00 % 1.15 % [kworker/0:1]
2018-12-14 00:40:28 0.00 B/s 0.00 B/s 0.00 % 0.59 % [kworker/0:1]
2018-12-14 00:40:30 0.00 B/s 0.00 B/s 0.00 % 0.56 % [kworker/0:1]
2018-12-14 00:40:36 0.00 B/s 0.00 B/s 0.00 % 0.76 % [kworker/0:1]
[root@master ~]# cat iotest.txt
2018-12-14 00:48:01 0.00 B/s 0.00 B/s 0.00 % 0.59 % [kworker/0:2]
2018-12-14 00:48:01 0.00 B/s 0.00 B/s 0.00 % 0.59 % [kworker/0:2]
2018-12-14 00:48:05 0.00 B/s 0.00 B/s 0.00 % 0.52 % [kworker/0:2]
2018-12-14 00:48:07 0.00 B/s 0.00 B/s 0.00 % 0.54 % [kworker/0:2]
2018-12-14 00:48:09 0.00 B/s 0.00 B/s 0.00 % 0.56 % [kworker/0:2]
2018-12-14 00:49:00 0.00 B/s 0.00 B/s 0.00 % 0.77 % [kworker/0:2]
2018-12-14 00:49:00 0.00 B/s 0.00 B/s 0.00 % 0.77 % [kworker/0:2]
2018-12-14 00:49:02 0.00 B/s 0.00 B/s 0.00 % 0.61 % [kworker/0:2]
2018-12-14 00:49:08 0.00 B/s 0.00 B/s 0.00 % 1.71 % [kworker/0:2]
2018-12-14 00:49:19 0.00 B/s 0.00 B/s 0.00 % 0.58 % [kworker/0:2]
2018-12-14 00:49:23 0.00 B/s 0.00 B/s 0.00 % 0.54 % [kworker/0:2]
2018-12-14 00:50:00 0.00 B/s 0.00 B/s 0.00 % 0.50 % [kworker/0:2]
2018-12-14 00:53:02 0.00 B/s 0.00 B/s 0.00 % 1.08 % [kworker/0:1]
2018-12-14 00:53:08 0.00 B/s 0.00 B/s 0.00 % 1.15 % [kworker/0:1]
2018-12-14 00:53:18 0.00 B/s 0.00 B/s 0.00 % 0.67 % [kworker/0:1]
2018-12-14 00:53:20 0.00 B/s 0.00 B/s 0.00 % 0.52 % [kworker/0:1]
2018-12-14 00:53:22 0.00 B/s 0.00 B/s 0.00 % 0.69 % [kworker/0:1]
2018-12-14 00:53:29 0.00 B/s 0.00 B/s 0.00 % 0.61 % [kworker/0:1]
2018-12-14 00:53:33 0.00 B/s 0.00 B/s 0.00 % 0.55 % [kworker/0:1]
2018-12-14 00:53:35 0.00 B/s 0.00 B/s 0.00 % 0.55 % [kworker/0:1]
2018-12-14 00:53:39 0.00 B/s 0.00 B/s 0.00 % 0.51 % [kworker/0:1]
2018-12-14 00:59:01 0.00 B/s 0.00 B/s 0.00 % 0.51 % [kworker/0:1]
2018-12-14 00:59:03 0.00 B/s 0.00 B/s 0.00 % 0.57 % [kworker/0:1]
2018-12-14 01:00:02 0.00 B/s 0.00 B/s 0.00 % 0.62 % [kworker/0:1]
2018-12-14 01:01:03 0.00 B/s 0.00 B/s 0.00 % 0.93 % [kworker/0:1]
2018-12-14 01:02:01 31.06 K/s 0.00 B/s 0.00 % 0.67 % systemd-journald
2018-12-14 01:02:05 0.00 B/s 0.00 B/s 0.00 % 0.58 % [kworker/0:1]
2018-12-14 01:01:03 0.00 B/s 0.00 B/s 0.00 % 0.93 % [kworker/0:1]
2018-12-14 01:05:05 0.00 B/s 0.00 B/s 0.00 % 1.12 % [kworker/0:2]
2018-12-14 01:06:00 0.00 B/s 0.00 B/s 0.00 % 0.55 % [kworker/0:2]
2018-12-14 01:06:04 0.00 B/s 0.00 B/s 0.00 % 0.56 % [kworker/0:2]
2018-12-14 01:06:06 0.00 B/s 0.00 B/s 0.00 % 0.60 % [kworker/0:2]
6、總結:
1)使用for循環的時候需要使用 IFS=$'\n' 去轉義,否則你輸出的結果是這樣的:
[root@master ~]# ./abc123.sh
2018-12-14 00:40:00
2018-12-14 0.00
2018-12-14 B/s
2018-12-14 0.00
2018-12-14 B/s
2018-12-14 0.00
2018-12-14 %
2018-12-14 0.65
2018-12-14 %
2018-12-14 [kworker/0:1]
2018-12-14 00:40:04
2018-12-14 0.00
2018-12-14 B/s
2018-12-14 0.00
2018-12-14 B/s
2018-12-14 0.00
2018-12-14 %
2018-12-14 0.66
2018-12-14 %
2018-12-14 [kworker/0:1]
好啦!寫好的監控IO使用率的腳本可以直接拿去使用,希望能夠幫到大家。