自動化巡檢---AIX腳本分享一例

巡檢是大多數SA的日常工作之一,但對於如此枯燥乏味的活兒想來大多數SA都提不起興趣,這時候腳本就派上用場了,跑起來真是多快好省.下面就分享一個簡單的AIX巡檢腳本.

巡檢項:

1. 文件系統使用率的監控(不能超過85%)

2. 交換空間的監控(不能超過70%)

3. errdemon進程運行是否正常

4. 檢查系統報錯

5. 檢查rootvg中邏輯卷是否同步

6. 檢查適配卡狀態

來看一下腳本,這個腳本會在每臺主機上以普通用戶執行(結合Crontab),生成一個日誌文件,然後調用工具上傳到特定主機,以方便檢查,整個過程均自動,無需人工操作.

#! /usr/bin/ksh
# Version 1.0
# AUTHOR:       Xin23   http://weibo.com/231988
NextLine()
{
        echo ' '>> $(date +%Y_%m_%d)_$(hostname)_Total.log
}
EchoTitle()
{
        echo "-------------------$TITLE-------------------" >> $(date +%Y_%m_%d)_$(hostname)_Total.log
        NextLine
}
#第一部分:  檢查並記錄異常到日誌
df -k | awk ' {printf "%-33s%8s\n",$1,$4}' | awk '{split($2,b,"%");if(b[1]>85)print}'| sort -nrk 2 >> /tmp/$(date +%Y_%m_%d)_$(hostname)_FileSystem.log
#當文件系統使用率超過85%時報警
ps -ef | grep errdemon | grep -v grep >> /tmp/$(date +%Y_%m_%d)_$(hostname)_ErrDemon.log
#檢查errdemon進程是否啓動
errpt >> /tmp/$(date +%Y_%m_%d)_$(hostname)_Error.log
#檢查系統報錯
lsvg -l rootvg | tail +3 | grep -v syncd >> /tmp/$(date +%Y_%m_%d)_$(hostname)_Disk.log
#檢查rootvg中邏輯卷狀態
lsps -a | awk ' $5 > 70 {printf "%-13s%8s%8s\n",$1,$4,$5}'| tail +2 >> /tmp/$(date +%Y_%m_%d)_$(hostname)_Paging.log
#檢查交換空間,使用率高於70%時報警
lsdev -Cc adapter | grep -v Available >> /tmp/$(date +%Y_%m_%d)_$(hostname)_Adapter.log
#檢查適配卡狀態
#第二部分:  彙總異常
cd /tmp
TITLE=HACMP
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_HACMP.log ]
then echo 'HACMP SERVICE IP IS OK!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
else echo 'Warning: HACMP FAILED!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
fi
NextLine
TITLE=FileSystem
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_FileSystem.log ]
then echo 'Warning: FileSystem has an error!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
        cat $(date +%Y_%m_%d)_$(hostname)_FileSystem.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'FileSystem is ok' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
TITLE=ErrDemon
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_ErrDemon.log ]
then echo 'ErrDemon is ok' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'Warning: ErrDemon was Stoped!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
TITLE=Error
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_Error.log ]
then echo 'Warning: There is an error in system!'>> $(date +%Y_%m_%d)_$(hostname)_Total.log
        NextLine
        cat $(date +%Y_%m_%d)_$(hostname)_Error.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'No Error in System!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
TITLE=Disk
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_Disk.log ]
then echo 'Warning: There is something wrong with disk!' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
        cat $(date +%Y_%m_%d)_$(hostname)_Disk.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'Disk State is ok' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
#cat $(date +%Y_%m_%d)_$(hostname)_Mem.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
TITLE=Paging
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_Paging.log ]
then echo 'Warning: There is something wrong with paging space' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
        cat $(date +%Y_%m_%d)_$(hostname)_Paging.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'Paging space is ok' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
TITLE=Adapter
EchoTitle
if [ -s $(date +%Y_%m_%d)_$(hostname)_Adapter.log ]
then echo 'Warning: Adapter error' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
        cat $(date +%Y_%m_%d)_$(hostname)_Adapter.log >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
else echo 'Adapter is ok' >> $(date +%Y_%m_%d)_$(hostname)_Total.log;
fi
NextLine
TITLE='FileSystem Status'
EchoTitle
df -k | awk '{printf "%-22s%12s%10s%6s%10s%7s %s\n", $1,$2,$3,$4,$5,$6,$7}' >> $(date +%Y_%m_%d)_$(hostname)_Total.log
NextLine
#第三部分:  調用腳本上傳到主機
DATE=$(date +%Y_%m_%d)
HOSTNAME=$(hostname)
scp.sh $DATE $HOSTNAME
#請使用絕對路徑

再來看一下上傳工具scp.sh

#!/usr/bin/expect
# Version 1.0
# AUTHOR:       Xin23   http://weibo.com/231988
set timeout 3
set IP xxx
set USER xxx
set PASSWORD xxx
set DATE [lindex $argv 0]
set HOSTNAME [lindex $argv 1]
spawn scp /tmp/$DATE\_$HOSTNAME\_Total.log $USER@$IP:/tmp/log
expect {
                "(yes/no)?" {send "yes\r"}
                "*assword:" {send "$PASSWORD\r"}
}
expect {
                "$" {send "\r"}
                "*assword:" {send "$PASSWORD\r"}
        }
exit

來看一下效果:

被上傳的巡檢日誌:

巡檢:

grep Warning 即可,因爲有任何異常日誌都會記錄關鍵字Warning,這樣就一目瞭然了.

$ grep Warning *

$

一切正常

The End