關於keepalive 問題處理
昨天新配置了Nginx Keepalived架構，今天上午來了VIP無法訪問了，於是查看日誌開始解決問題。
主服務器日誌：
Sep 25 16:08:28 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 16:08:33 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:08:23 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:08:23 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:49 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:49 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:15:50 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1Sep 25 17:30:09 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Received lower prio advert, forcing new electionSep 25 17:30:51 modoor Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 1.1.1.1
從服務器日誌：
Sep 25 15:58:19 wap Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 25 15:58:19 wap Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 1.1.1.1
Sep 25 15:58:24 wap Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 1.1.1.1
Sep 25 16:08:25 wap Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 25 16:11:32 wap Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 25 16:48:01 wap Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 25 16:48:02 wap Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 25 16:48:02 wap Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 25 16:48:02 wap Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 1.1.1.1
Sep 25 16:48:05 wap Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Sep 25 16:48:05 wap Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 25 16:48:05 wap Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Sep 25 17:08:19 wap Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 25 17:08:19 wap Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Sep 25 17:08:19 wap Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
Sep 25 17:08:23 wap Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 25 17:08:24 wap Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 25 17:08:24 wap Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 25 17:08:24 wap Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth1 for 1.1.1.1
Sep 25 17:08:25 wap Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
從日誌分析：主備在反覆的切換。於是查看自己的配置文件
我的配置文件是這樣的：
global_defs {
notification_email {
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_http_port {
script "/opt/check_nginx.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 50
mcast_src_ip 1.1.1.100
priority 100
advert_int 1
nopreempt
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_http_port ### 執行監控的服務
}
virtual_ipaddress {
1.1.1.1
}
}
檢查配置文件沒有問題。
然後懷疑問題出在/opt/check_nginx.sh 這個檢測上面
/opt/check_nginx.sh 的內容是：
!/bin/bash
if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
/etc/init.d/nginx start
fi
sleep 2
if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
/etc/init.d/keepalived stop
fi
仔細的讀者會發現我這個腳本有問題，對!/bin/bash 前面少個#號。於是添加上，但是問題還是沒有解決。 最後我再keepalive中取消了nginx的檢測，keepalived正常了。
#!/bin/bash
if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
/etc/init.d/nginx start
fi
sleep 2
if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
/etc/init.d/keepalived stop
fi
以上這個腳本是有問題的。下面是修改後正確的腳本：
#!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
then
/usr/local/nginx/sbin/nginx
sleep 5
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ]
then
killall keepalived
fi
fi
後期補充：
日誌中出現Process [6528] didn't respond to SIGTERM 這種問題check_nginx.sh也是這個腳本的問題，
大家注意一下。
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章