五分鐘上手 Nginx 日誌調優，較爲全面的 Nginx 日誌解析！

原創 酋长rosen 2019-08-28 16:55

爲什麼要針對配置 Nginx 日誌？

優點：
在使用nginx進行服務器管理時候，日誌對於統計、審查、排錯來說非常有利，通過日誌能更準確的定位信息，快速解決問題！

缺點：
配置後的日誌可能較爲複雜，而且日誌文件的體積會比常規日誌的要大，更佔據空間；需要對日誌文件進行切割，一段時間後需要清理無用的日誌文件

一、常規日誌

nginx常規日誌配置：

日誌配置存放的位置：
日誌設置可存在 http 中，也可存在 server 虛擬主機中

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';
                  
access_log  /var/log/nginx/access.log;	  #常規日誌
error_log /var/log/nginx/error.log error; #錯誤日誌

Nginx 日誌默認爲普通文本的格式，例如，下面是 Nginx 的一行訪問日誌：

# tail -n 10 access.log 
192.168.10.1 - - [23/Aug/2019:15:29:57 +0800] "GET /www/%E6%89%8B%E5%8A%A8%E9%98%80%E5%8D%81%E5%88%86 HTTP/1.1" 404 555 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:03 +0800] "GET /www/ HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:03 +0800] "GET /www/text.png HTTP/1.1" 404 555 "http://deng.rosen.cn/www/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:03 +0800] "GET /www/ HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:03 +0800] "GET /www/text.png HTTP/1.1" 404 555 "http://deng.rosen.cn/www/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:04 +0800] "GET /www/ HTTP/1.1" 200 755 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:04 +0800] "GET /www/text.png HTTP/1.1" 404 555 "http://deng.rosen.cn/www/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:08 +0800] "GET / HTTP/1.1" 200 63 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:09 +0800] "GET / HTTP/1.1" 200 36 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.10.1 - - [23/Aug/2019:15:30:09 +0800] "GET / HTTP/1.1" 200 63 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"

二、Json格式日誌（常規日誌參數也存在下面解析中）

設置爲Json格式（配置重啓Nginx可觀察）：

日誌配置存放的位置：
日誌設置可存在 http 中，也可存在 server 虛擬主機中

log_format json_main '{			 #定義日誌的格式
	"timestamp":"$time_local",'  #通用日誌格式下的本地時間，也可使用“$time_iso8601”標準格式下的本地時間
	'"host":"$server_addr",'	 #訪問服務器的IP
	'"remote_user":"$remote_user",'   #記錄客戶端用戶名稱
	'"clientip":"$remote_addr",'      #客戶端請求的IP
	'"request":"$request",' 		  #記錄請求的URL和HTTP協議
	'"request_time":"$request_time",' #請求處理時間
	'"request":"$request",'		      #請求的類型
	'"request_method":"$request_method",'	#請求的方法
	'"connection_requests":"$connection_requests",' #當前通過一個連接獲得的請求數量
	'"http_user_agent":"$http_user_agent",'  #記錄客戶端瀏覽器相關信息
	'"body_bytes_sent":"$body_bytes_sent",'  #發送給客戶端的字節數，發送給客戶端的字節數，也可選擇 $bytes_sent 發送給客戶端的總字節數
	'"upstream_response_time":"$upstream_response_time",' #上游請求時間
	'"upstream_addr":"$upstream_addr",' #上游請求的地址去向
	'"url":"$uri",'		#請求的URL
	'"domain":"$host",'	#請求的域名
	'"http_x_forwarded_for":"$http_x_forwarded_for",' #（反向）記錄客戶端IP地址,也可以用 $remote_addr
	'"http_referrer":"$http_referer",'  #記錄從哪個頁面鏈接訪問過來的
	'"status":"$status"}'; #返回的狀態

access_log  /var/log/nginx/access.log  json_main;	 #json_main 格式日誌
error_log /var/log/nginx/error.log error;			 #錯誤日誌

爲了便於利用 ELK 日誌平臺收集展示 Nginx 的日誌，可以將 Nginx 的日誌改成 json 的格式。修改後的 json 日誌格式如下所示：

# tail -n 10 access.log 
{"timestamp":"23/Aug/2019:15:47:32 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/text.png HTTP/1.1","request_time":"0.001","request":"GET /www/text.png HTTP/1.1","request_method":"GET","connection_requests":"8","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"555","upstream_response_time":"0.000","upstream_addr":"192.168.10.135:80","url":"/www/text.png","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"http://deng.rosen.cn/www/","status":"404"}
{"timestamp":"23/Aug/2019:15:47:33 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/ HTTP/1.1","request_time":"0.001","request":"GET /www/ HTTP/1.1","request_method":"GET","connection_requests":"9","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"755","upstream_response_time":"0.000","upstream_addr":"192.168.10.134:80","url":"/www/","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"200"}
{"timestamp":"23/Aug/2019:15:47:33 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/text.png HTTP/1.1","request_time":"0.001","request":"GET /www/text.png HTTP/1.1","request_method":"GET","connection_requests":"10","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"555","upstream_response_time":"0.000","upstream_addr":"192.168.10.135:80","url":"/www/text.png","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"http://deng.rosen.cn/www/","status":"404"}
{"timestamp":"23/Aug/2019:15:47:34 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/ HTTP/1.1","request_time":"0.001","request":"GET /www/ HTTP/1.1","request_method":"GET","connection_requests":"11","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"655","upstream_response_time":"0.001","upstream_addr":"192.168.10.135:80","url":"/www/","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"200"}
{"timestamp":"23/Aug/2019:15:47:34 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/text.png HTTP/1.1","request_time":"0.001","request":"GET /www/text.png HTTP/1.1","request_method":"GET","connection_requests":"12","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"555","upstream_response_time":"0.001","upstream_addr":"192.168.10.135:80","url":"/www/text.png","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"http://deng.rosen.cn/www/","status":"404"}
{"timestamp":"23/Aug/2019:15:47:36 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET / HTTP/1.1","request_time":"0.001","request":"GET / HTTP/1.1","request_method":"GET","connection_requests":"13","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"36","upstream_response_time":"0.001","upstream_addr":"192.168.10.134:80","url":"/","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"200"}
{"timestamp":"23/Aug/2019:15:47:37 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET / HTTP/1.1","request_time":"0.002","request":"GET / HTTP/1.1","request_method":"GET","connection_requests":"14","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"63","upstream_response_time":"0.001","upstream_addr":"192.168.10.135:80","url":"/","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"200"}
{"timestamp":"23/Aug/2019:15:47:37 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET / HTTP/1.1","request_time":"0.001","request":"GET / HTTP/1.1","request_method":"GET","connection_requests":"15","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"36","upstream_response_time":"0.001","upstream_addr":"192.168.10.134:80","url":"/","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"200"}
{"timestamp":"23/Aug/2019:15:47:41 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /sdfasf HTTP/1.1","request_time":"0.001","request":"GET /sdfasf HTTP/1.1","request_method":"GET","connection_requests":"16","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"555","upstream_response_time":"0.001","upstream_addr":"192.168.10.135:80","url":"/sdfasf","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"-","status":"404"}
{"timestamp":"23/Aug/2019:15:47:45 +0800","host":"192.168.10.133","remote_user":"-","clientip":"192.168.10.1","request":"GET /www/text.png HTTP/1.1","request_time":"0.001","request":"GET /www/text.png HTTP/1.1","request_method":"GET","connection_requests":"17","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36","body_bytes_sent":"555","upstream_response_time":"0.001","upstream_addr":"192.168.10.134:80","url":"/www/text.png","domain":"deng.rosen.cn","http_x_forwarded_for":"-","http_referrer":"http://deng.rosen.cn/www/","status":"404"}

三、nginx 配置 Json格式，可直引用！

log_format json_main '{"timestamp":"$time_local",'
		'"host":"$server_addr",'
		'"remote_user":"$remote_user",'
		'"clientip":"$remote_addr",'
		'"request":"$request",'
		'"request_time":"$request_time",'
		'"request":"$request",'
		'"request_method":"$request_method",'
		'"connection_requests":"$connection_requests",' 
		'"http_user_agent":"$http_user_agent",'
		'"body_bytes_sent":"$body_bytes_sent",'
		'"upstream_response_time":"$upstream_response_time",'
		'"upstream_addr":"$upstream_addr",'
		'"url":"$uri",'
		'"domain":"$host",'
		'"http_x_forwarded_for":"$http_x_forwarded_for",'
		'"http_referrer":"$http_referer",'
		'"status":"$status"}';

access_log  /var/log/nginx/access.log  json_main;
error_log /var/log/nginx/error.log error;

修改 Nginx 的配置，重啓 Nginx ，便可以看到 json 格式的日誌，重啓 Nginx：

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

五分鐘上手nginx、熟悉nginx配置詳解

nginx.conf配置詳解析 #定義nginx運行用戶和用戶組 user nginx; #nginx進程數，建議等於CPU核心數或者 auto worker_processes auto; #全局錯誤日誌定義類型：[ d

ぃ小小宇宙 2020-06-16 09:01:55

php-fpm配置優化總結

酋长rosen 2019-10-25 16:55:46