古人云“工欲善其事，必先利其器”。可以說，shell腳本語言編程就是系統管理員的一個編程利器、一個好幫手。熟練掌握和運用shell腳本語言編程，對於一個合格系統管理員來講，可以說是基本素養。
shell腳本語言，可實現複雜程序邏輯，數據處理。除去強大的編程功能外，其無需編譯環境的解釋運行的特色，更顯示其強大的靈活性。
學習shell編程，好處很多，但是困難是什麼呢？有的說，沒有shell編程學習範例。其實，在linux操作系統內，存在大量的腳本學習範例可供學習借鑑。
有的講，沒有編程需求。仔細回想一下，很多工作包含大量複雜的操作，是否有必要通過邏輯條件判斷和數據抽取處理實現，並封裝成自動化的腳本，以供今後重複調用。
在掌握shell編程的基礎指令、正則表達和邏輯判斷後，通過一些例子的練習，實現用戶需求後，就可以達到shell腳本語言編程能力的進一步提升。.
本文向給出以下思路和框架。
實戰.格式化文本數據
一些指令的多行輸出結果，或一些文件的多行內容，可能已經結構化好了，但大部分沒有格式化。所以編程進階面對的一個重要問題，是抽取有用的數據並格式化數據，以便後續的加工處理。
格式化文本類數據基本方法如下：
（1）篩選數據
主要是運用grep、sed，抽取有用內容、剔除無用內容
# df -k|sed 1,2d 剔除指令輸出中的前兩行
# cat /etc/host| grep -v ^# |grep -v ^$ 剔除文件中的空行和註釋行
# cat /etc/host| grep -v ^# |grep -v ^$ 剔除文件中的空行和註釋行
# df -k |grep '[0-9][0-9]%' 抽取指令輸出中帶%且爲2位數的行
# awk -F ":" '{print $1}' /etc/passwd 抽取文件的第一列
# df -k |grep '[0-9][0-9]%' |sed -e 's/%//g' 剔除指令輸出中中%字符
# sed -i 's/APPLICATION//g' /etc/hosts 剔除文件中APPLICATION字符
（2）結構化數據
主要利用awk的指定字段分割符方法，完成數據的結構化
# ip a |grep "<" |awk -F ":" '{print $2}' |grep eth
（3）排序數據
主要是運用sort，對於文件差異的比對非常有用
# awk -F ":" '{print $1}' /etc/passwd |sort 按正序排列用戶
# awk -F ":" '{print $1,$3}' /etc/group|sort -rn -k 2 按倒序排列用戶組ID
實戰.LINUX系統CPU空閒率指標檢查
例：sar指令連續執行6次，求字段9%idle的最小值，如果小於80，則提示異常，否則顯示正常：
腳本實現了外部參數的代入，通過統計後，得出判斷結果
# cat check.sh
CPU_IDLE_STR="檢查CPU空閒IDLE,大於百分"
CPU_IDLE_VAL=80
sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { min=0} {if ($9 < min ) min=(min<$9)?min:$9} END {if (min < var2) {printf "\t[ %-40s %-02d]\t\t*異常*\t\t當前值[ %-02d ] \n",var1,var2,min } else {printf "\t[ %-40s %-02d ]\t\t正 常\t\t當前值[ %-02d ] \n",var1,var2,min }}'
# sh check.sh
例：sar指令連續執行6次，求字段9%idle的平均值，如果小於80，則提示異常，否則顯示正常：
腳本實現了外部參數的代入，通過統計後，得出判斷結果
# cat check.sh
CPU_IDLE_STR="檢查CPU空閒IDLE,大於百分"
CPU_IDLE_VAL=80
sar 1 6 |sed 1,3d| awk -v var1=$CPU_IDLE_STR -v var2=$CPU_IDLE_VAL 'BEGIN { sum=0;count=0 } {sum +=$9; count++;} END {if (sum/count < var2) {printf "\t[ %-40s %-02d]\t\t*異常*\t\t當前值[ %-02d ] \n",var1,var2,sum/count } else {printf "\t[ %-40s %-02d ]\t\t正 常\t\t當前值[ %-02d ] \n",var1,var2,sum/count }}'
# sh check.sh
例：統計不同日期內的報錯
# cat m1.txt
Server1023/Server1023.log,Oct 17, 2017 7:31:12 PM GMT+08:00,Error> <Server1023>
Server1023/Server1023.log,Oct 17, 2017 7:32:12 PM GMT+08:00,Error> <Server1023>
Server1023/Server1023.log,Oct 17, 2017 8:05:20 PM GMT+08:00,Server1023> <[STUCK]
Server1113/Server1113.log,Sep 25, 2017 9:40:32 AM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 25, 2017 9:41:32 AM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 25, 2017 9:58:08 AM GMT+08:00,Server1113> <[STUCK]
Server1113/Server1113.log,Sep 29, 2017 3:59:22 PM GMT+08:00,Error> <Server1113>
Server1113/Server1113.log,Sep 29, 2017 4:00:22 PM GMT+08:00,Error> <Server1113>
# awk -F "," 'BEGIN { max="Oct 17" ;count=0 } { if ($2 == max ) { ++count } else { printf " %s %d \n",max,count ;max=$2;count=1} } END { printf " %s %d \n",max,count }' m1.txt
Oct 17 3
Sep 25 3
Sep 29 2
例：統計不同錯誤代碼出現的次數
# cat m1.txt
####<Apr 7, 2017 2:07:27 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491545247275> <BEA-310003> <Free memory in the server is 4,905,048 bytes. There is danger of OutOfMemoryError>
####<Apr 8, 2017 9:06:33 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491613593096> <BEA-310003> <Free memory in the server is 4,728,536 bytes. There is danger of OutOfMemoryError>
####<Apr 9, 2017 3:46:38 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491680798625> <BEA-310003> <Free memory in the server is 4,992,928 bytes. There is danger of OutOfMemoryError>
####<Apr 9, 2017 10:54:44 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491749684634> <BEA-310003> <Free memory in the server is 4,772,736 bytes. There is danger of OutOfMemoryError>
####<Apr 10, 2017 5:51:50 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491817910249> <BEA-310003> <Free memory in the server is 4,794,072 bytes. There is danger of OutOfMemoryError>
####<Apr 11, 2017 12:51:55 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491886315817> <BEA-310003> <Free memory in the server is 4,960,776 bytes. There is danger of OutOfMemoryError>
####<Apr 12, 2017 7:52:01 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1491954721248> <BEA-310003> <Free memory in the server is 4,989,464 bytes. There is danger of OutOfMemoryError>
####<Apr 13, 2017 2:55:06 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492023306984> <BEA-310003> <Free memory in the server is 4,845,776 bytes. There is danger of OutOfMemoryError>
####<Apr 13, 2017 9:59:12 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492091952865> <BEA-310003> <Free memory in the server is 4,833,352 bytes. There is danger of OutOfMemoryError>
####<Apr 14, 2017 5:00:05 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492160405970> <BEA-310003> <Free memory in the server is 4,979,432 bytes. There is danger of OutOfMemoryError>
####<Apr 15, 2017 12:04:11 PM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492229051734> <BEA-310003> <Free memory in the server is 4,957,360 bytes. There is danger of OutOfMemoryError>
####<Apr 16, 2017 7:08:17 AM GMT+08:00> <Critical> <Health> <tsptsvc> <Server1002> <weblogic.GCMonitor> <<anonymous>> <> <> <1492297697464> <BEA-310003> <Free memory in the server is 4,798,184 bytes. There is danger of OutOfMemoryError>
####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099718> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<Sep 23, 2017 9:41:39 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1506174099719> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189359> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 27, 2017 9:36:29 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495892189360> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776640> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 27, 2017 10:02:56 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495893776641> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000415> <System has file descriptor limits of - soft: 8,192, hard: 8,192>
####<May 28, 2017 7:04:24 PM GMT+08:00> <Info> <Socket> <tsptsvc> <Server1042> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1495969464161> <BEA-000416> <Using effective file descriptor limit of: 8,192 open sockets/files.>
# awk -F"<" '/BEA/ {print $6,$13 } END {}' o1.txt |awk '{printf "\t%s %s %s\n",S[$NF],$1,$2;++S[$NF];printf "\t %s %s %s\n",S[$NF],$1,$2 } END {for(a in S) printf "END %s %d \n",a,S[a]}'
END BEA-000415> 4
END BEA-000416> 4
END BEA-310003> 12