前言

對於訪問量較大的網站來說，隨着流量的增加單臺服務器已經無法處理所有的請求，這時候需要多臺服務器對大量的請求進行分流處理，即負載均衡。而如果實現負載均衡，必須在網站的入口部署服務器（不只是一臺）對這些請求進行分發，這臺服務器即反向代理。

由於反向代理服務器是網站的入口，其負載壓力大且易遭到攻擊，存在單點故障的風險，所以我們需要一個高可用的方案來實現當一臺反向代理服務器宕機的時候，另一臺服務器會自動接管服務。基於以上要求，我們使用HAProxy，KeepAlived來構建高可用的反向代理系統。

介紹

HAProxy是高性能的代理服務器，其可以提供7層和4層代理，具有healthcheck，負載均衡等多種特性，性能卓越，包括Twitter，Reddit，StackOverflow，GitHub在內的多家知名互聯網公司在使用。

KeepAlived是一個高可用方案，通過VIP(即虛擬IP)和心跳檢測來實現高可用。其原理是存在一組（兩臺）服務器，分別賦予Master,Backup兩個角色，默認情況下Master會綁定VIP到自己的網卡上，對外提供服務。Master,Backup會在一定的時間間隔向對方發送心跳數據包來檢測對方的狀態，這個時間間隔一般爲2秒鐘，如果Backup發現Master宕機，那麼Backup會發送ARP包到網關，把VIP綁定到自己的網卡，此時Backup對外提供服務，實現自動化的故障轉移，當Master恢復的時候會重新接管服務。

環境

OS: CentOS Linux release 6.0 (Final) 2.6.32-71.29.1.el6.x86_64
HAProxy: 1.4.18
KeepAlived: 1.2.2
VIP: 192.168.1.99
M: 192.168.1.222
S: 192.168.1.189

架構

                    192.168.1.99
             +-----------VIP----------+   
             |                        |
             |                        |
           Master                   Backup
        192.168.1.189            192.168.1.222
        +----------+             +----------+
        | HAProxy  |             | HAProxy  |
        |keepalived|             |keepalived|
        +----------+             +----------+
             |  
             v  
    +--------+---------+ 
    |        |         |
    |        |         |
    v        v         v
+------+  +------+  +------+
| WEB1 |  | WEB2 |  | WEB3 |
+------+  +------+  +------+

安裝HAProxy

安裝pcre

$ yum install pcre
$ wget http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.18.tar.gz
$ tar -zxvf haproxy-1.4.18.tar.gz
$ cd haproxy-1.4.18

注意編譯參數：

TARGET是指自己系統的內核版本 ARCH指定系統是32位還是64位
CPU=native: use the build machine's specific processor optimizations
更多編譯參數內容見源碼中的README
$ make TARGET=linux26 ARCH=x86_64 USE_PCRE=1 CPU=native $ make install

配置文件 /etc/haproxy.cfg

global
    log 127.0.0.1   local3
    maxconn 20000   
    uid 535  #uid和gid按照實際情況進行配置
    gid 520  
    chroot /var/chroot/haproxy
    daemon 
    nbproc 1 
 
defaults
   log     127.0.0.1       local3
   mode    http            
   option  httplog
   option  httpclose
   option  dontlognull
   option  forwardfor
   retries 2
   balance roundrobin 
   stats   uri     /haproxy-stats
   contimeout      5000
   clitimeout      50000
   srvtimeout      50000
 
frontend http-in
        bind *:80 
        default_backend pool1
 
backend pool1
        option httpchk HEAD / HTTP/1.0
        stats refresh 2
        server WEB1 192.168.1.189:81 weight 3 maxconn 10000 check 
        server WEB2 192.168.1.222:81 weight 3 maxconn 10000 check

查看HAProxy的狀態：http://192.168.1.99/haproxy-stats，這個頁面會顯示HAProxy本身以及後端服務器的狀態。

日誌

haproxy會把日誌記錄發送到syslog server(CentOS6下是rsyslogd，UDP514端口)，編輯/etc/rsyslog.conf文件，添加如下內容：

$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 127.0.0.1
local3.*                /var/log/haproxy.log

重啓rsyslog

$ /etc/init.d/rsyslog restart

自動輪轉日誌，編輯/etc/logrotate.d/haproxy.cfg，添加如下內容：

/var/log/haproxy.log
{
    rotate 4
    daily
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
    postrotate
    reload rsyslog > /dev/null 2>&1 || true
    endscript
}

啓動腳本

$ wget -O haproxy https://raw.github.com/gist/3665034/4125bd5b81977a72e5eec30650fb21f3034782a0/haproxy-init.d 
$ cp haproxy /etc/init.d/haproxy
$ chmod +x /etc/init.d/haproxy
#使用方式
$ /etc/init.d/haproxy start|stop|restart

安裝KeepAlived

安裝依賴庫

$ yum install popt popt-devel

安裝KeepAlived

$ wget http://www.keepalived.org/software/keepalived-1.2.2.tar.gz
$ tar -zxvf keepalived-1.2.2.tar.gz
$ cd keepalived-1.2.2
$ ./configure --prefix=/usr/local/keepalived
$ make && make install
 
$ cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived
$ cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
$ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
$ mkdir -p /etc/keepalived/
$ cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf 
$ chmod +x /etc/init.d/keepalived

使用方式

$ /etc/init.d/keepalived start|stop|restart

Master服務器上的配置 /etc/keepalived/keepalived.conf

global_defs {
 
   notification_email {
       [email protected]
   }
 
   notification_email_from [email protected]
   smtp_server 192.168.x.x
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
 
#監測haproxy進程狀態，每2秒執行一次
vrrp_script chk_haproxy {
    script "/usr/local/keepalived/chk_haproxy.sh"
    interval 2
    weight 2
}
 
vrrp_instance VI_1 {
    state MASTER #標示狀態爲MASTER
    interface eth0
    virtual_router_id 51
    priority 101   #MASTER權重要高於BACKUP
    advert_int 1
    mcast_src_ip 192.168.1.189 #Master服務器IP
 
    authentication {
        auth_type PASS #主從服務器驗證方式
        auth_pass 1111
    }
 
    track_script {
        chk_haproxy #監測haproxy進程狀態
    }
 
    #VIP
    virtual_ipaddress {
        192.168.1.99 #虛擬IP
    }
}

Bakcup服務器上的配置 /etc/keepalived/keepalived.conf

global_defs {
   notification_email {
   [email protected]
   }
 
   notification_email_from [email protected]
   smtp_server 192.168.x.x
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
 
#監測haproxy進程狀態，每2秒執行一次
vrrp_script chk_haproxy {
    script "/usr/local/keepalived/chk_haproxy.sh"
    interval 2
    weight 2
}
 
vrrp_instance VI_1 {
    state BACKUP #狀態爲BACKUP
    interface eth0
    virtual_router_id 51
    priority 100  #權重要低於MASTER
    advert_int 1
    mcast_src_ip 192.168.1.222 #Backup服務器的IP
 
    authentication {
        auth_type PASS
        auth_pass 1111
    }
 
    track_script {
        chk_haproxy #監測haproxy進程狀態
    }
 
    #VIP
    virtual_ipaddress {
        192.168.1.99 #虛擬IP
    }
}

chk_haproxy.sh內容

#!/bin/bash
#
# author: weizhifeng
# description: 
# 定時查看haproxy是否存在，如果不存在則啓動haproxy，
# 如果啓動失敗，則停止keepalived
# 
status=$(ps aux|grep haproxy | grep -v grep | grep -v bash | wc -l)
if [ "${status}" = "0" ]; then
    /etc/init.d/haproxy start
 
    status2=$(ps aux|grep haproxy | grep -v grep | grep -v bash |wc -l)
 
    if [ "${status2}" = "0"  ]; then
            /etc/init.d/keepalived stop
    fi
fi

高可用測試

在Master上停止keepalived，查看系統日誌，發現MASTER釋放了VIP

$ /etc/init.d/keepalived stop
 $ tail -f /var/log/message
 Keepalived: Terminating on signal Keepalived: Stopping Keepalived v1.2.2 (11/03,2011) 
 Keepalived_vrrp: Terminating VRRP child process on signal 
 Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

在Backup上查看系統日誌，發現Backup已經進入MASTER角色，並且綁定了VIP 192.168.1.99

$ tail -f /var/log/message
 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs
 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.99 #在Backup上查看VIP是否已經綁定

在Master上重新啓動keepalived，查看系統日誌，發現重新獲得MASTER角色，並且綁定VIP 192.168.1.99

$ /etc/init.d/keepalived start
 $ tail -f /var/log/message
 Keepalived_vrrp: VRRP_Instance(VI_1) Transition to MASTER STATE
 Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTER STATE
 Keepalived_vrrp: VRRP_Instance(VI_1) setting protocol VIPs.
 Keepalived_vrrp: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.99

在Backup上查看系統日誌，發現其重新回到BACKUP角色，並且釋放VIP

$ tail -f /var/log/message
 Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
 Keepalived_vrrp: VRRP_Instance(VI_1) Entering BACKUP STATE
 Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.

併發測試

我們使用webbench來對HAProxy進行併發測試

$ yum install ctags
$ wget http://home.tiscali.cz/~cz210552/distfiles/webbench-1.5.tar.gz
$ tar -zxvf webbench-1.5.tar.gz
$ cd webbench-1.5
$ make 
$ mkdir -p /usr/local/man && make install

測試環境：
CPU：Intel 雙核 x86_64 主頻3191MHZ
Mem：2G

修改php-fpm.conf，設置PHP-FPM spawn的進程數量爲100：

pm.start_servers = 100
pm.max_spare_servers = 100

測試方法：

$ webbench -c 100 -t 3000 http://192.168.1.99/check.txt
$ webbench -c 100 -t 3000 http://192.168.1.99/test.php

測試結果：

併發訪問txt文件，HAProxy的session數量爲10000左右，這說明HAProxy能夠hold住10000個併發連接；併發訪問php文件，HAProxy的session峯值爲200左右，接近於後端PHP的併發處理能力(100x2)。

HAProxy負載均衡單點故障解決方案：HAProxy+keepAlived

前言

介紹

環境

架構

安裝HAProxy

日誌

安裝KeepAlived

高可用測試

併發測試

【HTML】H5實現倒計時

HAProxy負載均衡單點故障解決方案：HAProxy+keepAlived

redis隊列的實現

【Android】Android實現切換主題皮膚功能

【交易所】山哥的撮合系統火力加強升級版

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結