使用haproxy+keepalived來實現mariadb galera cluster的高可用架構

更多博文請關注：沒有傘的孩子必須努力奔跑 (www.xuchanggang.cn)

   在上篇mariadb galera cluster博文中,實現了數據庫之間的多主複製,數據庫對外提供了好幾個IP地址,我們在這裏通過haproxy的虛擬IP(就如mysql-proxy一樣),來對外提供一個IP地址來實現數據庫的負載均衡,通過keepalived來判斷haproxy的存活情況。

一.Haproxy和keepalived的概述：

   haproxy是一款免費的提供高可用性,負載均衡以及基於TCP(第四層)和HTTP(第七層)應用的代理軟件,藉助haproxy可以快速並且可靠的提供基於TCP和HTTP應用的代理解決方案.

   keepalived主要作用是LoadBalance master和LoadBalance backup之間的健康檢查,實現故障轉換.

   haproxy官方網站：http://haproxy.1wt.eu/

   keepalived官方網站：http://www.keepalived.org/

二.haproxy的特點和算法：

1.haproxy的特點是：

  (1).免費開源,穩定性也是非常好

  (2).能夠補充Nginx的一些缺點比如Session的保持，Cookie的引導等工作

  (3).支持虛擬主機

  (4).是一款負載均衡軟件,單純從效率上來講HAProxy比Nginx有更出色的負載均衡速度,在併發處理上也是優於Nginx的

  (5).可以對Mysql讀進行負載均衡,對後端的MySQL節點進行檢測和負載均衡[這點是我門dba需要關注的重點]

2.haproxy的算法現在也越來越多了,具體有如下8種：

  (1).roundrobin：表示簡單的輪詢[數據庫負載的話,建議使用此方式,使用source的話,後端數據庫宕掉的話,會有一個連接出錯]

  (2).static-rr：表示根據權重

  (3).leastconn：表示最少連接者先處理

  (4).source：表示根據請求源IP,這個跟Nginx的IP_hash機制類似,我們用其作爲解決session問題的一種方法[此方法針對數據庫待定]

  (5).ri：表示根據請求的URI

  (6).rl_param：表示根據請求的URl參數'balance url_param' requires an URL parameter name

  (7).hdr(name)：表示根據HTTP請求頭來鎖定每一次HTTP請求

  (8).rdp-cookie(name)：表示根據cookie(name)來鎖定並哈希每一次TCP請求

三.haproxy+keepalived+mariadb galera cluster高可用負載搭建演示：

# 這裏配置的主要目的其實是針對mariadb galera cluster來實現數據庫的負載均衡,但這裏爲了方便演示,使用mysql數據庫來替代galera cluster(mysql數據庫不配置主從,方便顯示效果)

# 再次聲明：如果僅僅是mysql數據庫讀寫負載,是不需要此架構的,這個架構僅針對mariadb galera cluster

1.環境：

os：red hat linux 6（64位,2.6內核）
haproxy1/keepalived1：192.168.1.100 (master)
haproxy2/keepalived2：192.168.1.101 (backup)
mysql1:192.168.1.102
mysql2:192.168.1.103
haproxy vip:192.168.1.220

2.配置實現圖：

3.詳細配置步驟如下：

(1).在192.168.1.100上安裝配置haproxy

[root@client100 ~]# tar -xf haproxy-1.4.24.tar.gz
[root@client100 ~]# cd haproxy-1.4.24
# 注意下面的TARGET必須大寫，不然會報錯(如果報gcc錯誤，安裝gcc軟件)
[root@client100 haproxy-1.4.24]# make TARGET=linux26 prefix=/usr/local/haproxy
[root@client100 haproxy-1.4.24]# make install PREFIX=/usr/local/haproxy
[root@client100 haproxy-1.4.24]# cd /usr/local/haproxy/
# conf目錄如果不存在，請先建立該目錄[下面這步複製配置的模板文件可以忽略]
[root@client100 haproxy-1.4.24]# cp examples/haproxy.cfg  /usr/local/haproxy/conf/haproxy.conf
# 在當前目錄下，建立兩個目錄conf logs(分別存放HAproxy的配置文件，PID文件和日誌文件)
[root@client100 haproxy]# mkdir conf logs
[root@client100 haproxy]# vim conf/haproxy.conf
# 以下有部分參數其實是不需要設置的
global
        log 127.0.0.1   local0
        maxconn 4096                   # 最大連接數
        chroot /usr/local/haproxy    # 安裝路徑
        uid 99                               # 所屬用戶id
        gid 99                               # 所屬用戶組id[用戶和組可以自己創建的]
        daemon                            # 後臺運行
        quiet
        nbproc 1                           # 進程數,可以同時開啓多個
        pidfile /usr/local/haproxy/logs/haproxy.pid
defaults
        log global
        mode http                        # 所處理的類別[7層：http;4層：tcp]
        retries 3                          # 3次連接失敗就認爲服務不可用
        option httplog                  # 日誌類別http日誌格式
        option dontlognull            # 不記錄健康檢查的日誌信息
        option redispatch            # serverid對應服務器宕掉後,強制定向到其他健康的服務器
        option abortonclose         #當服務器負載很高的話,自動結束到當前處理比較久的連接
        maxconn 4096                # 最大連接數
        contimeout 50000           # 連接超時
        clitimeout 50000             # 客戶端連接超時
        srvtimeout 50000            # 心跳檢測超時
listen mysql_proxy 0.0.0.0:3306
        mode tcp                       # 監聽4層 模式
        balance roundrobin        # 負載均衡方式爲輪詢
    #  balance source              # 此負載方式數據庫負載不建議使用,http可以使用
        option tcpka
        option httpchk               # 心跳檢測
    #  option mysql-check user haproxy
        server mysql1 192.168.1.102:3306 weight 1    # 後端真是數據庫ip地址和端口,權重
        server mysql2 192.168.1.103:3306 weight 1    # 後端真是數據庫ip地址和端口,權重
# 啓動測試,看haproxy啓動是否正常
[root@client100 haproxy]# /usr/local/haproxy/sbin/haproxy-f haproxy.conf
[root@client100 haproxy]# ps -ef |grep haproxy
# 編寫haproxy啓動腳本[可編寫也可以不編寫,建議編寫]
[root@client100 haproxy]# vim /etc/init.d/haproxy
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="$@"
start()
{
echo"START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy-f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo"STOP HAPoxy Listen"
kill-TTOU $(cat$BASE_DIR/logs/haproxy.pid)
echo"STOP HAPoxy process"
kill-USR1 $(cat$BASE_DIR/logs/haproxy.pid)
}
case$ARGVin
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo"hactl.sh [start|restart|stop]"
esac
exit$ERROR
# 賦予腳本可執行權限
[root@client100 haproxy]# chmod +x /etc/init.d/haproxy

(2).在192.168.1.101上安裝配置haproxy

[root@client101 ~]# tar -xf haproxy-1.4.24.tar.gz
[root@client101 ~]# cd haproxy-1.4.24
[root@client101 haproxy-1.4.24]# make TARGET=linux26 prefix=/usr/local/haproxy
[root@client101 haproxy-1.4.24]# make install PREFIX=/usr/local/haproxy
[root@client101 haproxy-1.4.24]# cd /usr/local/haproxy/
[root@client101 haproxy]# mkdir conf logs
[root@client101 haproxy]# vim conf/haproxy.conf
global
        log 127.0.0.1   local0
        maxconn 4096
        chroot /usr/local/haproxy
        uid 99
        gid 99
        daemon
        quiet
        nbproc 1
        pidfile /usr/local/haproxy/logs/haproxy.pid
defaults
        log global
        mode http
        retries 3
        option httplog
        option dontlognull
        option redispatch
        option abortonclose
        maxconn 4096
        contimeout 50000
        clitimeout 50000
        srvtimeout 50000
listen mysql_proxy 0.0.0.0:3306
        mode tcp
        balance roundrobin
    #  balance source
        option tcpka
        option httpchk
    #  option mysql-check user haproxy
        server mysql1 192.168.1.102:3306 weight 1
        server mysql2 192.168.1.103:3306 weight 1
[root@client100 haproxy]# /usr/local/haproxy/sbin/haproxy-f haproxy.conf
[root@client100 haproxy]# ps -ef |grep haproxy
# 編寫haproxy啓動腳本[可編寫也可以不編寫,建議編寫]
[root@client100 haproxy]# vim /etc/init.d/haproxy
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="$@"
start()
{
echo"START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy-f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo"STOP HAPoxy Listen"
kill-TTOU $(cat$BASE_DIR/logs/haproxy.pid)
echo"STOP HAPoxy process"
kill-USR1 $(cat$BASE_DIR/logs/haproxy.pid)
}
case$ARGVin
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo"hactl.sh [start|restart|stop]"
esac
exit$ERROR
# 賦予腳本可執行權限
[root@client100 haproxy]# chmod +x /etc/init.d/haproxy

(3).在192.168.1.100上安裝配置keepalived

# keepalived需要openssl依賴包
[root@client100 ~]# yum -y install openssl* gcc make
# 建立keepalived安裝目錄
[root@client100 ~]# mkdir /usr/local/keepalived
[root@client100 ~]# tar -xf keepalived-1.2.9.tar.gz
[root@client100 ~]# cd keepalived-1.2.9
[root@client100 keepalived-1.2.9]#  ./configure  --prefix=/usr/local/keepalived && make && make install
# 複製相應文件到指定目錄
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/sbin/keepalived  /usr/sbin/
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
[root@client100 keepalived-1.2.9]# mkdir /etc/keepalived
[root@client100 keepalived-1.2.9]# cd /etc/keepalived/
# 編輯配置文件
[root@client100 keepalived]# vim keepalived.conf
global_defs {
    notification_email {
[email protected]            # e-mail地址
    }
    notification_email_from [email protected]
    smtp_server smtp.163.com   # 郵件服務器地址
    smtp_connect_timeout 30     # 連接超時時間
    router_id LVS_Master
}
vrrp_script chk_http_port {
    script  "/etc/keepalived/check_haproxy.sh"    # haproxy運行檢測腳本[haproxy宕掉重啓haproxy服務]
    interval        5                     # 腳本執行間隔
    weight         -5                    #執行腳本後優先級變更：5表示優先級+5;-5則表示優先級-5
}
vrrp_instance VI_A {
    state MASTER                    # 主上此值爲MASTER,從上爲BACKUP
    interface eth0
    virtual_router_id 50            # 此值主從必須一致
    priority 100
    advert_int 1
    authentication {                  # authentication兩個參數值,主從也必須一致
        auth_type PASS
        auth_pass kongzhong
    }
track_script {
    chk_http_port
    }
    virtual_ipaddress {
        192.168.1.220                # haproxy提供的虛擬IP地址
    }
}
#啓動keepalived服務
[root@client100 keepalived]# /etc/init.d/keepalivedstart
# 編輯check_haproxy.sh腳本
[root@client100 keepalived]# vim/etc/keepalived/check_haproxy.sh
# 此腳本用來檢測haproxy是否正常,不正常啓動
#!/bin/bash
A=`ps-C haproxy --no-header |wc-l`
if[ $A -eq0 ];then
/etc/init.d/haproxyrestart
echo"Start haproxy"&>/dev/null
sleep3
if[ `ps-C haproxy --no-header |wc-l` -eq0 ];then
/etc/init.d/keepalivedstop
echo"Stop keepalived"&>/dev/null
fi
fi
# 賦予腳本可執行權限
[root@client100 keepalived]#chmod +x /etc/keepalived/check_haproxy.sh

(4).在192.168.1.101上安裝配置keepalived

# keepalived需要openssl依賴包
[root@client101 ~]# yum -y install openssl* gcc make
# 建立keepalived安裝目錄
[root@client101 ~]# mkdir /usr/local/keepalived
[root@client101 ~]# cd keepalived-1.2.9
[root@client101 keepalived-1.2.9]# ./configure  --prefix=/usr/local/keepalived && make && make install
# 複製相應文件到指定目錄
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/sbin/keepalived  /usr/sbin/
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
[root@client101 keepalived-1.2.9]# mkdir /etc/keepalived
[root@client101 keepalived-1.2.9]# cd /etc/keepalived/
# 編輯配置文件
[root@client101 keepalived]# vim keepalived.conf
global_defs {
    notification_email {
[email protected]
    }
    notification_email_from [email protected]
    smtp_server smtp.163.com
    smtp_connect_timeout 30
    router_id LVS_Master
}
vrrp_script chk_http_port {
    script  "/etc/keepalived/check_haproxy.sh"
    interval        5
    weight         -5
}
vrrp_instance VI_A {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass kongzhong
    }
track_script {
    chk_http_port
    }
    virtual_ipaddress {
        192.168.1.220
    }
}
#啓動keepalived服務
[root@client101 keepalived]# /etc/init.d/keepalivedstart
# 編輯check_haproxy.sh腳本
[root@client101 keepalived]# vim/etc/keepalived/check_haproxy.sh
# 此腳本用來檢測haproxy是否正常,不正常啓動
#!/bin/bash
A=`ps-C haproxy --no-header |wc-l`
if[ $A -eq0 ];then
/etc/init.d/haproxyrestart
echo"Start haproxy"&>/dev/null
sleep3
if[ `ps-C haproxy --no-header |wc-l` -eq0 ];then
/etc/init.d/keepalivedstop
echo"Stop keepalived"&>/dev/null
fi
fi
# 賦予腳本可執行權限
[root@client101 keepalived]#chmod +x /etc/keepalived/check_haproxy.sh

(5).192.168.1.102,192.168.1.103兩臺數據庫啓動

[root@client102 ~]# /etc/init.d/mysqld start
[root@client103 ~]# /etc/init.d/mysqld start
# 兩臺數據庫上分別建立kz數據庫,創建允許登陸的用戶
mysql> create database kz;
mysql> grant all privileges on kz.* to 'kz'@'192.168.1.%' identified by 'kz';
mysql> flush privileges
# 下面爲了測試登陸到那臺數據庫服務器上
# 我們在192.168.1.102上建立kz102這張表
mysql> use kz
mysql> create table kz102(a int);
# 我們在192.168.1.103上建立kz103這張表
mysql> use kz
mysql> create table kz103(a int);

4.下面就可以測試,這裏就不演示詳細內容,簡要說明一下大概步驟

  (1).haproxy是否正常啓動：ps -ef |grep haproxy

  (2).keepalived是否啓動正常/IP是否可以正常切換：ip add|grep 192.168.1.220

  (3).使用kz登陸數據庫,是否能正常登陸,登陸的是那一臺數據庫服務器

      mysql -ukz -pkz -h192.168.1.220

      use kz

      show tables

  (4).後端數據庫宕掉1臺,再次登陸看是否能正常登陸

到此,測試完畢,有問題歡迎反饋,拍磚.