使用AWSTATS自動分析Nginx日誌
環境說明：
Nginx安裝目錄：/usr/local/nginx
Nginx配置文件存放目錄：/usr/local/nginx/conf/nginx.conf
Nginx日誌目錄：/usr/local/nginx/logs
Awstats安裝目錄：/usr/local/awstats
Awstats配置文件存放目錄：/etc/awstats
日誌切割腳本存放目錄：/usr/local/nginx/sbin/cut_nginx_log.sh
一、Nginx日誌切割
1、修改nginx.conf配置文件，使AWSTATS支持分析日誌格式
# vim /usr/local/nginx/conf/nginx.conf
|
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
|
2、編輯Nginx日誌切割腳本
# vim /usr/local/nginx/sbin/cut_nginx_log.sh
|
#!/bin/bash
mv /usr/local/nginx/logs/access.log /usr/local/nginx/logs/access_`date +%Y%m%d`.log
mv /usr/local/nginx/logs/error.log /usr/local/nginx/logs/error_`date +%Y%m%d`.log
killall -s USER1 nginx
|
# chmod +x /usr/local/nginx/sbin/cut_nginx_log.sh
3、
二、下載安裝Awstats
1、下載安裝
|
# tar -zxvf awstats-7.0.tar.gz
# mv awstats-7.0 /usr/local/awstats
# mkdir -p /var/lib/awstats
|
2、配置
|
# mkdir - p /tmp/src/wwwroot/cgi-bin/
# cp
/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf /tmp/src/wwwroot/cgi-bin/
# /usr/local/awstats/tools/awstats_configure.pl
-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/tmp/src
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ?輸入y
|
回車
|
-----> Check for web server install
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> none 這裏不是apache，寫none，跳過#
|
回車
|
-----> Update model config file '/tmp/src/wwwroot/cgi-bin/awstats.model.conf'
File awstats.model.conf updated.
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ?輸入 y #創建一個全新的統計
|
回車
|
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> 192.168.2.143被統計網站的域名 #
|
回車
|
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
> #回車繼續
-----> Create config file '/etc/awstats/awstats.192.168.2.143.conf'
Config file /etc/awstats/awstats.192.168.2.143.conf created.
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/tmp/src/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143
Or if you have several config files and prefer having only one command:
/tmp/src/tools/awstats_updateall.pl now
Press ENTER to continue...
A SIMPLE config file has been created: /etc/awstats/awstats.192.168.2.143.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for '192.168.2.143' with command:
> perl awstats.pl -update -config=192.168.2.143
You can also build static report pages for '192.168.2.143' with command:
> perl awstats.pl -output=pagetype -config=192.168.2.143
Press ENTER to finish...
|
三、修改統計日誌文件的路徑
|
# vim /etc/awstats/awstats.192.168.2.143.conf -
LogFile="/var/log/httpd/mylog.log"修改成--->
LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-0.log"
|
這裏是對應上面Nginx日誌切割所生成的目錄存放位置，注意awstats的年月日格式，分析的執行順序是：
Nginx 產生日誌 –> 日誌切割 –> Nginx 繼續產生日誌 –> 另存切割日誌 –> 交由Awstats統計 –> 生成結果
四、執行Awstats 日誌更新程序開始統計分析
|
#/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=192.168.2.143
Create/Update database for config "/etc/awstats/awstats.192.168.2.143.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/nginx/logs/access_20121207.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 66466
Found 0 dropped records,
Found 0 comments,
Found 1 blank records,
Found 58472 corrupted records,
Found 0 old records,
Found 7993 new qualified records.
|
看到以上顯示，證明日誌切割和 Awstats 都已經運行無誤了。統計分析完成後，結果還在 Awstats 的數據庫中。在 Apache 上，可以直接打開 Perl 程序的網頁查看統計。但本文開始時已經提到，Nginx 對 Perl 支持並不好，所以我們要換個方法，利用 awstats 的工具將統計的結果生成靜態文件。
五、Awstats生成靜態文件
|
# cp -rf /usr/local/awstats/wwwroot/ /usr/local/nginx/logs/
# mkdir /usr/local/nginx/logs/wwwroot/awstats
# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
|
/usr/local/awstats/tools/awstats_buildstaticpages.pl #Awstats 靜態頁面生成腳本
-update -config=192.168.2.143 #更新配置選項
-lang=cn #語音中文
-dir=/usr/local/nginx/logs/wwwroot/awstats/ #統計結果輸出目錄
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl #日誌更新程序路徑
六、修改nginx配置文件
|
# vim /usr/local/nginx/conf/nginx.conf
server {
listen 8080;
server_name localhost;
location ~ ^/awstats/ { # html 靜態頁面目錄
root /usr/local/nginx/logs/wwwroot/awstats;
index index.html;
access_log off;
error_log off;
charset gb2312; }
location ~ ^/icon/ { # 圖標目錄
root /usr/local/nginx/logs/wwwroot;
index index.html;
access_log off;
error_log off;
charset gb2312;
}
}
|
七、配置Awstats自動運行
|
# vim /usr/local/nginx/sbin/cut_nginx_log.sh
23 59 * * * root /usr/local/nginx/sbin/cut_nginx_log.sh
00 01 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=192.168.2.143 -lang=cn -dir=/usr/local/nginx/logs/wwwroot/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl
|
