HAProxy雙機高可用方案之HAProxy+Keepalived

 HAProxy雙機高可用方案之HAProxy+Keepalived

環境:centos 5.4 x64
軟件版本:haproxy-1.3.26.tar.gz 、keepalived-1.2.7.tar.gz
網絡拓撲:
                  192.168.48.143
             +-----------VIP----------+  
             |           eth1         |
             |                        |
           Master                   Backup
        192.168.48.139            192.168.48.140
        +----------+             +----------+
        | HAProxy  |    eth0     | HAProxy  |
        |keepalived|-------------|keepalived|
        +----------+1.139   1.140+----------+
             | 
             v 
        +--------+
        |        |     
        |        |     
        v        v  eth1   
    +------+  +------+ 
    | WEB1 |  | WEB2 | 
    +------+  +------+ 
    48.141     48.142

haproxy安裝
主頁:http://haproxy.1wt.eu/
下載地址:http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.26.tar.gz
[root@localhost tools]# wget http://haproxy.1wt.eu/download/1.3/src/haproxy-1.3.26.tar.gz
[root@localhost tools]# tar zxf haproxy-1.3.26.tar.gz
[root@localhost tools]# cd haproxy-1.3.26
[root@localhost haproxy-1.3.26]# make TARGET=linux26 PREFIX=/usr/local/haproxy
[root@localhost haproxy-1.3.26]# make install PREFIX=/usr/local/haproxy
[root@localhost haproxy-1.3.26]# cd /usr/local/haproxy/
[root@localhost haproxy]# mkdir conf
主備配置一樣
[root@localhost haproxy]# cat haproxy.cfg
global 
        log 127.0.0.1   local0 
        maxconn 65535 
        chroot /usr/local/haproxy 
        uid 99 
        gid 99 
        daemon 
        nbproc 2
        #pidfile /usr/local/haproxy/haproxy.pid 
        debug 

defaults 
         log     127.0.0.1       local3 
         mode   http 
         option httplog 
         option httpclose 
         option dontlognull 
         option forwardfor 
         option redispatch 
         retries 2 
         maxconn 2000 
#         balance roundrobin
#         stats   uri     /haproxy-stats 
         contimeout      5000 
         clitimeout      50000 
         srvtimeout      50000 

frontend web_proxy 
         bind *:80
         mode http
         option httplog 
         log global  
         default_backend phppool  

backend phppool
        mode http
        balance source
        option httpchk HEAD /
        server web1 192.168.48.141:80  check inter 2000
        server web2 192.168.48.142:80  check inter 2000       

listen  web_ha
        bind 0.0.0.0:8080
        mode http 
        stats   uri     /haproxy-stats 
        stats hide-version
#        stats realm Haproxy\statistics
        stats auth gao:gao   # 用戶名/密碼
         
haproxy啓動腳本
[root@ha139 sbin]# cat haproxy.sh
#!/bin/sh 
# chkconfig 35 on 
# description: HAProxy is a TCP/HTTP reverse proxy which is particularly suited for high availability environments. 

# Source function library. 
if [ -f /etc/init.d/functions ]; then 
  . /etc/init.d/functions 
elif [ -f /etc/rc.d/init.d/functions ] ; then 
  . /etc/rc.d/init.d/functions 
else 
  exit 0 
fi 

# Source networking configuration. 
. /etc/sysconfig/network 

# Check that networking is up. 
[ ${NETWORKING} = "no" ] && exit 0 

[ -f /usr/local/haproxy/conf/haproxy.cfg ] || exit 1 

RETVAL=0

start() { 
  /usr/local/haproxy/sbin/haproxy -c -q -f /usr/local/haproxy/conf/haproxy.cfg 
  if [ $? -ne 0 ]; then 
    echo "Errors found in configuration file." 
    return 1 
  fi 

  echo -n "Starting HAproxy: " 
  daemon /usr/local/haproxy/sbin/haproxy -D -f /usr/local/haproxy/conf/haproxy.cfg -p /var/run/haproxy.pid 
  RETVAL=$? 
  echo 
  [ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy 
  return $RETVAL 


stop() { 
  echo -n "Shutting down HAproxy: " 
  killproc haproxy -USR1 
  RETVAL=$? 
  echo 
  [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy 
  [ $RETVAL -eq 0 ] && rm -f /var/run/haproxy.pid 
  return $RETVAL 


restart() { 
  /usr/local/haproxy/sbin/haproxy -c -q -f /usr/local/haproxy/conf/haproxy.cfg 
  if [ $? -ne 0 ]; then 
    echo "Errors found in configuration file, check it with 'haproxy check'." 
    return 1 
  fi 
  stop 
  start 


check() { 
  /usr/local/haproxy/sbin/haproxy -c -q -V -f /usr/local/haproxy/conf/haproxy.cfg 


rhstatus() { 
  status haproxy 


condrestart() { 
  [ -e /var/lock/subsys/haproxy ] && restart || : 


# See how we were called. 
case "$1" in 
  start) 
    start 
    ;; 
  stop) 
    stop 
    ;; 
  restart) 
    restart 
    ;; 
  reload) 
    restart 
    ;; 
  condrestart) 
    condrestart 
    ;; 
  status) 
    rhstatus 
;; 
  check) 
    check 
    ;; 
  *) 
    echo $"Usage: haproxy {start|stop|restart|reload|condrestart|status|check}" 
    RETVAL=1
esac 
exit $RETVAL
         
haproxy加日誌:
編輯/etc/syslog.conf文件, 添加內容如下:
local3.*        /var/log/haproxy.log
local0.*        /var/log/haproxy.log
編輯/etc/sysconfig/syslog文件,修改內容如下:
SYSLOGD_OPTIONS="-r -m 0"
然後重啓syslog服務,命令如下:
service syslog restart

keepalived安裝
主頁:www.keepalived.org
[root@localhost tools]# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
[root@ha tools]# tar zxf keepalived-1.2.7.tar.gz
[root@ha tools]# cd keepalived-1.2.7
[root@ha keepalived-1.2.7]# ./configure --prefix=/usr/local/keepalived
過程中出錯:
configure: error:
  !!! OpenSSL is not properly installed on your system. !!!
  !!! Can not include OpenSSL headers files.            !!!
因爲openssl沒有安裝好
yum install -y openssl-devel
[root@ha keepalived-1.2.7]# make
[root@ha keepalived-1.2.7]# make install

keepalived做成服務模式啓動
[root@ha keepalived-1.2.7]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
[root@ha keepalived-1.2.7]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
[root@ha keepalived-1.2.7]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
[root@ha keepalived-1.2.7]# mkdir /etc/keepalived
[root@ha keepalived-1.2.7]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
主的配置文件
[root@ha keepalived-1.2.7]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS_1
}
vrrp_script chk_haproxy {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
!state MASTER
state BACKUP
priority 100
advert_int 1
virtual_router_id 50
garp_master_delay 1
interface eth1
nopreempt
authentication {
auth_type PASS
auth_pass Kxiaokk345Pix
}
track_interface {
eth0
eth1
}
virtual_ipaddress {
192.168.48.143
}
track_script {
chk_haproxy
}
! Configuration File for keepalived
global_defs {
router_id LVS_1
}
vrrp_script chk_haproxy {
script "/etc/keepalived/check_haproxy.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
!state MASTER
state BACKUP
priority 100
advert_int 1
virtual_router_id 50
garp_master_delay 1
interface eth1
nopreempt
authentication {
auth_type PASS
auth_pass Kxiaokk345Pix
}
track_interface {
eth0
eth1
}
virtual_ipaddress {
192.168.48.143
}
track_script {
chk_haproxy
}
!notify_master "/etc/keepalived/mailnotify.py master"
!notify_backup "/etc/keepalived/mailnotify.py backup"
!notify_fault "/etc/keepalived/mailnotify.py fault"
}
}

配置說明:
! Configuration File for keepalived
global_defs {
     notification_email { #發生事件切換的時候,發送的郵箱,可以有多個,每行一個
     [email protected]
     }
     notification_email_from [email protected]  #指定發件人
     smtp_server 127.0.0.1  #指定smtp服務器地址
     smtp_connect_timeout 30    #指定smtp連接超時時間
     router_id LVS_2  #運行keepalived機器的一個標識主備不能相同
     }

#vrrp_script區域定義腳本名字和腳本執行的間隔和腳本執行的優先級變更
vrrp_script check_haproxy {
#定義監控nginx的命令,命令的$?返回1就改變優先級。
        script "/etc/keepalived/check_haproxy.sh" 
        interval 5     #腳本執行間隔
        weight -10   #腳本結果1導致的優先級變更:10表示優先級+10;-10則表示優先級-10
        fall   2     # 檢測幾次返回1爲失敗
        rise   1     # 檢測幾次返回0成功爲ok
        }

#VIP
vrrp_instance VI_1 {    #監控多個網段的實例
     state BACKUP     #全部服務器都設爲BACKUP,讓優先級高的自動爲master
#    dont track primary     #忽略VRRP的interface錯誤
     track_interface {
     eth0                 #跟蹤接口,設置額外的監控,裏面任意一塊網卡出現問題,都會進入故障(FAULT)狀態
     eth1
     }
     interface eth0         #實例綁定的網卡
     mcast_src_ip 192.168.48.139 #指定主或備的eth0的Ip    
     virtual_router_id 51   #這裏非常重要,相同的VRID爲一個組
     priority 100           #優先級,
     advert_int 1           #檢查間隔,默認1s
#     nopreempt            #不設置搶佔,優先級高就做master,當主恢復時不搶佔。nopreempt  //master 設置爲BACKUP ,通過priority來判斷哪成爲主,設置nopreempt 是可以在MASTER 出現問題修復好後不搶用vip,直接充當slave的角色。這樣,可以確保session不會丟失
     authentication {       #認證
     auth_type PASS         #認證的方式,支持PASS和AH
     auth_pass 123         #認證的密碼
     }
    track_interface {      #設置監聽本機的接口
    eth0
    eth1
    }     
     virtual_ipaddress {     #指定漂移地址(VIP)
     192.168.48.143         #如果有多個VIP,繼續換行填寫
     }
    track_script {     #執行定義的命令
    check_haproxy
    }
}

haproxy檢測腳本
[root@ha139 keepalived]# cat check_haproxy.sh
#!/bin/bash
#
# desc: check haproxy service
#
A=`ip address show eth1 | grep 192.168.48.143 | wc -l`
B=`ps -C haproxy --no-heading | wc -l`

if [ $A -eq 1 ]; then
if [ $B -eq 0 ]; then
/usr/local/haproxy/sbin/haproxy.sh start
sleep 3
fi
fi

主或備服務要求搶佔:不要配置nopreempt
track_interface配置監聽沒有配置時,如果是多網卡,只能監控之中的一張網卡。
track_interface配置同時監聽本機的eth0,eth1時,兩張網卡之中的一張有問題,vip就會漂移
track_interface配置分別監聽本機的eth0或eth1時,只是監聽指定的本機網卡有問題時,vip纔會漂移

主不要求搶佔:加nopreempt,做到主不搶佔,優先級高就做master,當主恢復時不搶佔。//master 設置爲BACKUP ,通過priority來判斷哪成爲主,設置nopreempt 是可以在MASTER 出現問題修復好後不搶用vip,直接充當slave的角色。這樣,可以確保session不會丟失
搶佔問題處理:http://bbs.linuxtone.org/thread-6788-1-1.html

高可用(keepalived+haproxy)切換測試:(兩臺機)
主或備斷eth0或eth1網卡,看能否漂移,是否搶佔。
主或備關keepaviled,看能否漂移。

併發測試
webbench最多可以模擬3萬個併發連接去測試網站的負載能力,個人感覺要比Apache自帶的ab壓力測試工具好,安裝使用也特別方便。
wget http://blog.s135.com/soft/linux/webbench/webbench-1.5.tar.gz
tar zxvf webbench-1.5.tar.gz
cd webbench-1.5
make &&mkdir -p /usr/local/man&& make install
使用:
webbench -c 500 -t 30 http://192.168.48.143
參數說明:-c表示併發數,-t表示時間(秒)

haproxy參考資料
http://network.51cto.com/art/201110/295955.htm  HAProxy雙機高可用方案之HAProxy+Keepalived(1)
http://dngood.blog.51cto.com/446195/738634 haproxy 安裝與配置 (有配置說明)
http://os.51cto.com/art/201202/317741_2.htm HAProxy配置語法及實例(3)(有配置說明)
http://hi.baidu.com/gaolongquan/item/51a8f6c68f44ad61f7c95db0  HAProxy+Keepalived
http://xukaizijian.blog.163.com/blog/static/17043311920115283358709/   web] Haproxy配置(有配置說明)
http://blog.51cto.com/zt/232     HAProxy 能夠做些什麼?51cto專題

keepalived參考資料
http://xzregg.blog.51cto.com/3829250/976793 ngnix+keepalived  實現N主高可用負載均衡web羣集 (有配置說明)
http://www.jb51.net/os/RedHat/55468.html  haproxy+keepalived  負載均衡之主備切換
http://bbs.linuxtone.org/thread-6788-1-1.html                 vip搶佔問題處理
http://www.tumblr.com/tagged/keepalived                       HAProxy + KeepAlived構建高可用的反向代理系統(帶併發測試)
http://blog.s135.com/post/288/                                併發測試
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章