使用haproxy+keepalived来实现mariadb galera cluster的高可用架构

更多博文请关注：没有伞的孩子必须努力奔跑 (www.xuchanggang.cn)

   在上篇mariadb galera cluster博文中,实现了数据库之间的多主复制,数据库对外提供了好几个IP地址,我们在这里通过haproxy的虚拟IP(就如mysql-proxy一样),来对外提供一个IP地址来实现数据库的负载均衡,通过keepalived来判断haproxy的存活情况。

一.Haproxy和keepalived的概述：

   haproxy是一款免费的提供高可用性,负载均衡以及基于TCP(第四层)和HTTP(第七层)应用的代理软件,借助haproxy可以快速并且可靠的提供基于TCP和HTTP应用的代理解决方案.

   keepalived主要作用是LoadBalance master和LoadBalance backup之间的健康检查,实现故障转换.

   haproxy官方网站：http://haproxy.1wt.eu/

   keepalived官方网站：http://www.keepalived.org/

二.haproxy的特点和算法：

1.haproxy的特点是：

  (1).免费开源,稳定性也是非常好

  (2).能够补充Nginx的一些缺点比如Session的保持，Cookie的引导等工作

  (3).支持虚拟主机

  (4).是一款负载均衡软件,单纯从效率上来讲HAProxy比Nginx有更出色的负载均衡速度,在并发处理上也是优于Nginx的

  (5).可以对Mysql读进行负载均衡,对后端的MySQL节点进行检测和负载均衡[这点是我门dba需要关注的重点]

2.haproxy的算法现在也越来越多了,具体有如下8种：

  (1).roundrobin：表示简单的轮询[数据库负载的话,建议使用此方式,使用source的话,后端数据库宕掉的话,会有一个连接出错]

  (2).static-rr：表示根据权重

  (3).leastconn：表示最少连接者先处理

  (4).source：表示根据请求源IP,这个跟Nginx的IP_hash机制类似,我们用其作为解决session问题的一种方法[此方法针对数据库待定]

  (5).ri：表示根据请求的URI

  (6).rl_param：表示根据请求的URl参数'balance url_param' requires an URL parameter name

  (7).hdr(name)：表示根据HTTP请求头来锁定每一次HTTP请求

  (8).rdp-cookie(name)：表示根据cookie(name)来锁定并哈希每一次TCP请求

三.haproxy+keepalived+mariadb galera cluster高可用负载搭建演示：

# 这里配置的主要目的其实是针对mariadb galera cluster来实现数据库的负载均衡,但这里为了方便演示,使用mysql数据库来替代galera cluster(mysql数据库不配置主从,方便显示效果)

# 再次声明：如果仅仅是mysql数据库读写负载,是不需要此架构的,这个架构仅针对mariadb galera cluster

1.环境：

os：red hat linux 6（64位,2.6内核）
haproxy1/keepalived1：192.168.1.100 (master)
haproxy2/keepalived2：192.168.1.101 (backup)
mysql1:192.168.1.102
mysql2:192.168.1.103
haproxy vip:192.168.1.220

2.配置实现图：

3.详细配置步骤如下：

(1).在192.168.1.100上安装配置haproxy

[root@client100 ~]# tar -xf haproxy-1.4.24.tar.gz
[root@client100 ~]# cd haproxy-1.4.24
# 注意下面的TARGET必须大写，不然会报错(如果报gcc错误，安装gcc软件)
[root@client100 haproxy-1.4.24]# make TARGET=linux26 prefix=/usr/local/haproxy
[root@client100 haproxy-1.4.24]# make install PREFIX=/usr/local/haproxy
[root@client100 haproxy-1.4.24]# cd /usr/local/haproxy/
# conf目录如果不存在，请先建立该目录[下面这步复制配置的模板文件可以忽略]
[root@client100 haproxy-1.4.24]# cp examples/haproxy.cfg  /usr/local/haproxy/conf/haproxy.conf
# 在当前目录下，建立两个目录conf logs(分别存放HAproxy的配置文件，PID文件和日志文件)
[root@client100 haproxy]# mkdir conf logs
[root@client100 haproxy]# vim conf/haproxy.conf
# 以下有部分参数其实是不需要设置的
global
        log 127.0.0.1   local0
        maxconn 4096                   # 最大连接数
        chroot /usr/local/haproxy    # 安装路径
        uid 99                               # 所属用户id
        gid 99                               # 所属用户组id[用户和组可以自己创建的]
        daemon                            # 后台运行
        quiet
        nbproc 1                           # 进程数,可以同时开启多个
        pidfile /usr/local/haproxy/logs/haproxy.pid
defaults
        log global
        mode http                        # 所处理的类别[7层：http;4层：tcp]
        retries 3                          # 3次连接失败就认为服务不可用
        option httplog                  # 日志类别http日志格式
        option dontlognull            # 不记录健康检查的日志信息
        option redispatch            # serverid对应服务器宕掉后,强制定向到其他健康的服务器
        option abortonclose         #当服务器负载很高的话,自动结束到当前处理比较久的连接
        maxconn 4096                # 最大连接数
        contimeout 50000           # 连接超时
        clitimeout 50000             # 客户端连接超时
        srvtimeout 50000            # 心跳检测超时
listen mysql_proxy 0.0.0.0:3306
        mode tcp                       # 监听4层 模式
        balance roundrobin        # 负载均衡方式为轮询
    #  balance source              # 此负载方式数据库负载不建议使用,http可以使用
        option tcpka
        option httpchk               # 心跳检测
    #  option mysql-check user haproxy
        server mysql1 192.168.1.102:3306 weight 1    # 后端真是数据库ip地址和端口,权重
        server mysql2 192.168.1.103:3306 weight 1    # 后端真是数据库ip地址和端口,权重
# 启动测试,看haproxy启动是否正常
[root@client100 haproxy]# /usr/local/haproxy/sbin/haproxy-f haproxy.conf
[root@client100 haproxy]# ps -ef |grep haproxy
# 编写haproxy启动脚本[可编写也可以不编写,建议编写]
[root@client100 haproxy]# vim /etc/init.d/haproxy
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="$@"
start()
{
echo"START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy-f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo"STOP HAPoxy Listen"
kill-TTOU $(cat$BASE_DIR/logs/haproxy.pid)
echo"STOP HAPoxy process"
kill-USR1 $(cat$BASE_DIR/logs/haproxy.pid)
}
case$ARGVin
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo"hactl.sh [start|restart|stop]"
esac
exit$ERROR
# 赋予脚本可执行权限
[root@client100 haproxy]# chmod +x /etc/init.d/haproxy

(2).在192.168.1.101上安装配置haproxy

[root@client101 ~]# tar -xf haproxy-1.4.24.tar.gz
[root@client101 ~]# cd haproxy-1.4.24
[root@client101 haproxy-1.4.24]# make TARGET=linux26 prefix=/usr/local/haproxy
[root@client101 haproxy-1.4.24]# make install PREFIX=/usr/local/haproxy
[root@client101 haproxy-1.4.24]# cd /usr/local/haproxy/
[root@client101 haproxy]# mkdir conf logs
[root@client101 haproxy]# vim conf/haproxy.conf
global
        log 127.0.0.1   local0
        maxconn 4096
        chroot /usr/local/haproxy
        uid 99
        gid 99
        daemon
        quiet
        nbproc 1
        pidfile /usr/local/haproxy/logs/haproxy.pid
defaults
        log global
        mode http
        retries 3
        option httplog
        option dontlognull
        option redispatch
        option abortonclose
        maxconn 4096
        contimeout 50000
        clitimeout 50000
        srvtimeout 50000
listen mysql_proxy 0.0.0.0:3306
        mode tcp
        balance roundrobin
    #  balance source
        option tcpka
        option httpchk
    #  option mysql-check user haproxy
        server mysql1 192.168.1.102:3306 weight 1
        server mysql2 192.168.1.103:3306 weight 1
[root@client100 haproxy]# /usr/local/haproxy/sbin/haproxy-f haproxy.conf
[root@client100 haproxy]# ps -ef |grep haproxy
# 编写haproxy启动脚本[可编写也可以不编写,建议编写]
[root@client100 haproxy]# vim /etc/init.d/haproxy
#!/bin/bash
BASE_DIR="/usr/local/haproxy"
ARGV="$@"
start()
{
echo"START HAPoxy SERVERS"
$BASE_DIR/sbin/haproxy-f $BASE_DIR/conf/haproxy.conf
}
stop()
{
echo"STOP HAPoxy Listen"
kill-TTOU $(cat$BASE_DIR/logs/haproxy.pid)
echo"STOP HAPoxy process"
kill-USR1 $(cat$BASE_DIR/logs/haproxy.pid)
}
case$ARGVin
start)
start
ERROR=$?
;;
stop)
stop
ERROR=$?
;;
restart)
stop
start
ERROR=$?
;;
*)
echo"hactl.sh [start|restart|stop]"
esac
exit$ERROR
# 赋予脚本可执行权限
[root@client100 haproxy]# chmod +x /etc/init.d/haproxy

(3).在192.168.1.100上安装配置keepalived

# keepalived需要openssl依赖包
[root@client100 ~]# yum -y install openssl* gcc make
# 建立keepalived安装目录
[root@client100 ~]# mkdir /usr/local/keepalived
[root@client100 ~]# tar -xf keepalived-1.2.9.tar.gz
[root@client100 ~]# cd keepalived-1.2.9
[root@client100 keepalived-1.2.9]#  ./configure  --prefix=/usr/local/keepalived && make && make install
# 复制相应文件到指定目录
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/sbin/keepalived  /usr/sbin/
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
[root@client100 keepalived-1.2.9]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
[root@client100 keepalived-1.2.9]# mkdir /etc/keepalived
[root@client100 keepalived-1.2.9]# cd /etc/keepalived/
# 编辑配置文件
[root@client100 keepalived]# vim keepalived.conf
global_defs {
    notification_email {
[email protected]            # e-mail地址
    }
    notification_email_from [email protected]
    smtp_server smtp.163.com   # 邮件服务器地址
    smtp_connect_timeout 30     # 连接超时时间
    router_id LVS_Master
}
vrrp_script chk_http_port {
    script  "/etc/keepalived/check_haproxy.sh"    # haproxy运行检测脚本[haproxy宕掉重启haproxy服务]
    interval        5                     # 脚本执行间隔
    weight         -5                    #执行脚本后优先级变更：5表示优先级+5;-5则表示优先级-5
}
vrrp_instance VI_A {
    state MASTER                    # 主上此值为MASTER,从上为BACKUP
    interface eth0
    virtual_router_id 50            # 此值主从必须一致
    priority 100
    advert_int 1
    authentication {                  # authentication两个参数值,主从也必须一致
        auth_type PASS
        auth_pass kongzhong
    }
track_script {
    chk_http_port
    }
    virtual_ipaddress {
        192.168.1.220                # haproxy提供的虚拟IP地址
    }
}
#启动keepalived服务
[root@client100 keepalived]# /etc/init.d/keepalivedstart
# 编辑check_haproxy.sh脚本
[root@client100 keepalived]# vim/etc/keepalived/check_haproxy.sh
# 此脚本用来检测haproxy是否正常,不正常启动
#!/bin/bash
A=`ps-C haproxy --no-header |wc-l`
if[ $A -eq0 ];then
/etc/init.d/haproxyrestart
echo"Start haproxy"&>/dev/null
sleep3
if[ `ps-C haproxy --no-header |wc-l` -eq0 ];then
/etc/init.d/keepalivedstop
echo"Stop keepalived"&>/dev/null
fi
fi
# 赋予脚本可执行权限
[root@client100 keepalived]#chmod +x /etc/keepalived/check_haproxy.sh

(4).在192.168.1.101上安装配置keepalived

# keepalived需要openssl依赖包
[root@client101 ~]# yum -y install openssl* gcc make
# 建立keepalived安装目录
[root@client101 ~]# mkdir /usr/local/keepalived
[root@client101 ~]# cd keepalived-1.2.9
[root@client101 keepalived-1.2.9]# ./configure  --prefix=/usr/local/keepalived && make && make install
# 复制相应文件到指定目录
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/sbin/keepalived  /usr/sbin/
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
[root@client101 keepalived-1.2.9]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/init.d/
[root@client101 keepalived-1.2.9]# mkdir /etc/keepalived
[root@client101 keepalived-1.2.9]# cd /etc/keepalived/
# 编辑配置文件
[root@client101 keepalived]# vim keepalived.conf
global_defs {
    notification_email {
[email protected]
    }
    notification_email_from [email protected]
    smtp_server smtp.163.com
    smtp_connect_timeout 30
    router_id LVS_Master
}
vrrp_script chk_http_port {
    script  "/etc/keepalived/check_haproxy.sh"
    interval        5
    weight         -5
}
vrrp_instance VI_A {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass kongzhong
    }
track_script {
    chk_http_port
    }
    virtual_ipaddress {
        192.168.1.220
    }
}
#启动keepalived服务
[root@client101 keepalived]# /etc/init.d/keepalivedstart
# 编辑check_haproxy.sh脚本
[root@client101 keepalived]# vim/etc/keepalived/check_haproxy.sh
# 此脚本用来检测haproxy是否正常,不正常启动
#!/bin/bash
A=`ps-C haproxy --no-header |wc-l`
if[ $A -eq0 ];then
/etc/init.d/haproxyrestart
echo"Start haproxy"&>/dev/null
sleep3
if[ `ps-C haproxy --no-header |wc-l` -eq0 ];then
/etc/init.d/keepalivedstop
echo"Stop keepalived"&>/dev/null
fi
fi
# 赋予脚本可执行权限
[root@client101 keepalived]#chmod +x /etc/keepalived/check_haproxy.sh

(5).192.168.1.102,192.168.1.103两台数据库启动

[root@client102 ~]# /etc/init.d/mysqld start
[root@client103 ~]# /etc/init.d/mysqld start
# 两台数据库上分别建立kz数据库,创建允许登陆的用户
mysql> create database kz;
mysql> grant all privileges on kz.* to 'kz'@'192.168.1.%' identified by 'kz';
mysql> flush privileges
# 下面为了测试登陆到那台数据库服务器上
# 我们在192.168.1.102上建立kz102这张表
mysql> use kz
mysql> create table kz102(a int);
# 我们在192.168.1.103上建立kz103这张表
mysql> use kz
mysql> create table kz103(a int);

4.下面就可以测试,这里就不演示详细内容,简要说明一下大概步骤

  (1).haproxy是否正常启动：ps -ef |grep haproxy

  (2).keepalived是否启动正常/IP是否可以正常切换：ip add|grep 192.168.1.220

  (3).使用kz登陆数据库,是否能正常登陆,登陆的是那一台数据库服务器

      mysql -ukz -pkz -h192.168.1.220

      use kz

      show tables

  (4).后端数据库宕掉1台,再次登陆看是否能正常登陆

到此,测试完毕,有问题欢迎反馈,拍砖.