負載均衡集羣LVS實戰篇

Keepalived+LVS的高可用集羣系統

前面兩章已經詳細介紹了負載均衡LVS原理和安裝,這篇文章實際操作一遍,讓大家能更深刻理解,閒話不多說,直接進入正題。

一.拓撲圖結構:

wKiom1dhBbKQEqIOAAHZOQTsykA798.png-wh_50

二.Keepalived的安裝

1.主服務器的安裝

[root@wjb10000-master ~]# wget http://www.keepalived.org/software/keepalived-1.2.20.tar.gz


[root@wjb10000-master ~]# tar -zxvf keepalived-1.2.20.tar.gz


[root@wjb10000-master ~]# cd keepalived-1.2.20/


[root@wjb10000-master keepalived-1.2.20]# ./configure --prefix=/usr/local/keepalived


configure: error:

  !!! OpenSSL is not properly installed on your system. !!!

  !!! Can not include OpenSSL headers files.            !!!

[root@wjb10000-master keepalived-1.2.20]# yum -y install openssl-devel libnl-devel libnfnetlink-devel ipvsadm


[root@wjb10000-master keepalived-1.2.20]# ./configure --prefix=/usr/local/keepalived


Keepalived configuration

------------------------

Keepalived version       : 1.2.20

Compiler                 : gcc

Compiler flags           : -g -O2 -DFALLBACK_LIBNL1

Extra Lib                : -lssl -lcrypto -lcrypt  -lnl 

Use IPVS Framework       : Yes

IPVS sync daemon support : Yes

IPVS use libnl           : Yes

fwmark socket support    : Yes

Use VRRP Framework       : Yes

Use VRRP VMAC            : Yes

Use VRRP authentication  : Yes

SNMP keepalived support  : No

SNMP checker support     : No

SNMP RFCv2 support       : No

SNMP RFCv3 support       : No

SHA1 support             : No

Use Debug flags          : No

libnl version            : 1

Use IPv4 devconf         : No

Use libiptc              : No

Use libipset             : No

[root@wjb10000-master keepalived-1.2.20]# make && make install


[root@wjb10000-master keepalived-1.2.20]# ln -s /usr/local/keepalived/sbin/keepalived /usr/bin/keepalived


[root@wjb10000-master keepalived-1.2.20]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived


[root@wjb10000-master keepalived-1.2.20]# chmod 755 /etc/init.d/keepalived


[root@wjb10000-master keepalived-1.2.20]# chkconfig keepalived on


[root@wjb10000-master keepalived-1.2.20]# vim /etc/init.d/keepalived


# Source configuration file (we set KEEPALIVED_OPTIONS there)

. /etc/sysconfig/keepalived

改爲:

# Source configuration file (we set KEEPALIVED_OPTIONS there)

. /usr/local/keepalived/etc/sysconfig/keepalived

[root@wjb10000-master keepalived-1.2.20]# mkdir /etc/keepalived


[root@wjb10000-master keepalived-1.2.20]# ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf


[root@wjb10000-master keepalived-1.2.20]# service keepalived restart

2.主服務器配置文件

[root@wjb10000-master /]# vim /etc/keepalived/keepalived.conf


! Configuration File for keepalived

 

global_defs {

   notification_email {

     [email protected]#收件人郵箱

   }

   notification_email_from keepalived@localhost#發件人郵箱

   smtp_server 127.0.0.1#郵箱服務器地址

   smtp_connect_timeout 30#連接時間

   router_id LVS_DEVEL#標識,唯一性

}

 

vrrp_instance VI_1 {#實例

    state MASTER#主MASTER備BACKUP

    interface eno16777736#網卡

    virtual_router_id 51#路由標識

    priority 100#優先級

    advert_int 1#檢查間隔

    authentication {

        auth_type PASS#驗證類型

        auth_pass 1111#驗證密碼

    }

    virtual_ipaddress {

        192.168.1.100#vip

    }

}

 

virtual_server 192.168.1.100 80 {

    delay_loop 6#輪詢間隔時間

    lb_algo rr#調度算法,rr|wrr|lc|wlc|lblc|sh|dh

    lb_kind DR#集羣模式,NAT|DR|TUN

    persistence_timeout 50#會話保持時間

    protocol TCP#使用協議

 

    real_server 192.168.1.30 80 {

        weight 1#權重

        TCP_CHECK {#檢查方式,有HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK

            connect_port    80#連接端口

            connect_timeout 3#連接超時時間

            nb_get_retry 3#重試次數

            delay_before_retry 3#重連時間間隔

        } 

    } 

 

    real_server 192.168.1.40 80 {

        weight 1

        TCP_CHECK {

            connect_port    80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        } 

    } 

}

3.配置 IPVS

3.1.開啓IP轉發功能

[root@wjb10000-master /]# echo 1 >/proc/sys/net/ipv4/ip_forward   默認是0,關閉ip轉發;這裏需要開啓,所以設置值爲1

3.2.配置重定向

[root@wjb10000-master /]# echo "0" >/proc/sys/net/ipv4/ip_forward


[root@wjb10000-master /]# echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects


[root@wjb10000-master /]# echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects


[root@wjb10000-master /]# echo "1" >/proc/sys/net/ipv4/conf/eth0/send_redirects

3.3.清除ipvsadm表

[root@wjb10000-master /]# ipvsadm –C

3.4.使用ipvsadm安裝web服務

[root@wjb10000-master /]# ipvsadm -A -t 192.168.1.100:80-s rr

3.5.增加二臺web服務器

[root@wjb10000-master /]# ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.30:80 -m -w 1


[root@wjb10000-master /]# ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.40:80 -m -w 1

3.6查看結果

[root@wjb10000-master ~]#  ipvsadm -L  -n


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.1.100:80 rr

  -> 192.168.1.30:80              Route   1      0          1       

  -> 192.168.1.40:80              Route   1      0          0

[root@wjb10000-master keepalived-1.2.20]# service keepalived restart

4.主服務器IP查看

[root@wjb10000-master /]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:74:1f:4e brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.10/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet 192.168.1.100/32 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe74:1f4e/64 scope link

       valid_lft forever preferred_lft forever

5.備服務器的安裝

[root@wjb10000-backup ~]# wget http://www.keepalived.org/software/keepalived-1.2.20.tar.gz


[root@wjb10000-backup ~]# tar -zxvf keepalived-1.2.20.tar.gz


[root@wjb10000-backup ~]# cd keepalived-1.2.20/


[root@wjb10000-backup keepalived-1.2.20]# ./configure --prefix=/usr/local/keepalived


configure: error:

  !!! OpenSSL is not properly installed on your system. !!!

  !!! Can not include OpenSSL headers files.            !!!

[root@wjb10000-backup keepalived-1.2.20]# yum -y install openssl-devel libnl-devel libnfnetlink-devel ipvsadm


[root@wjb10000-backup keepalived-1.2.20]# ./configure --prefix=/usr/local/keepalived


Keepalived configuration

------------------------

Keepalived version       : 1.2.20

Compiler                 : gcc

Compiler flags           : -g -O2 -DFALLBACK_LIBNL1

Extra Lib                : -lssl -lcrypto -lcrypt  -lnl 

Use IPVS Framework       : Yes

IPVS sync daemon support : Yes

IPVS use libnl           : Yes

fwmark socket support    : Yes

Use VRRP Framework       : Yes

Use VRRP VMAC            : Yes

Use VRRP authentication  : Yes

SNMP keepalived support  : No

SNMP checker support     : No

SNMP RFCv2 support       : No

SNMP RFCv3 support       : No

SHA1 support             : No

Use Debug flags          : No

libnl version            : 1

Use IPv4 devconf         : No

Use libiptc              : No

Use libipset             : No

[root@wjb10000-backup keepalived-1.2.20]# make && make install


[root@wjb10000-backup keepalived-1.2.20]# ln -s /usr/local/keepalived/sbin/keepalived /usr/bin/keepalived


[root@wjb10000-backup keepalived-1.2.20]# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived


[root@wjb10000-backup keepalived-1.2.20]# chmod 755 /etc/init.d/keepalived


[root@wjb10000-backup keepalived-1.2.20]# chkconfig keepalived on


[root@wjb10000-backup keepalived-1.2.20]# vim /etc/init.d/keepalived


[root@wjb10000-backup keepalived-1.2.20]# mkdir /etc/keepalived


[root@wjb10000-backup keepalived-1.2.20]# ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf


[root@wjb10000-backup keepalived-1.2.20]# service keepalived restart

 

6.備服務器配置文件

[root@wjb10000-backup /]# vim /etc/keepalived/keepalived.conf


! Configuration File for keepalived

 

global_defs {

   notification_email {

     [email protected]

   }

   notification_email_from keepalived@localhost

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface eno16777736

    virtual_router_id 51

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        192.168.1.100

    }

}

 

virtual_server 192.168.1.100 80 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

    persistence_timeout 50

    protocol TCP

 

    real_server 192.168.1.30 80 {

        weight 1

        TCP_CHECK {

            connect_port    80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        } 

    } 

 

    real_server 192.168.1.40 80 {

        weight 1

        TCP_CHECK {

            connect_port    80

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        } 

    } 

}

7.配置 IPVS

7.1.開啓IP轉發功能

[root@wjb10000-backup /]# echo 1 >/proc/sys/net/ipv4/ip_forward  

7.2.配置重定向

[root@wjb10000-backup /]# echo "0" >/proc/sys/net/ipv4/ip_forward


[root@wjb10000-backup /]# echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects


[root@wjb10000-backup /]# echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects


[root@wjb10000-backup /]# echo "1" >/proc/sys/net/ipv4/conf/eth0/send_redirects

7.3.清除ipvsadm表

[root@wjb10000-backup /]# ipvsadm –C

7.4.使用ipvsadm安裝web服務

[root@wjb10000-backup /]# ipvsadm -A -t 192.168.1.100:80-s rr

7.5.增加二臺web服務器

[root@wjb10000-backup /]# ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.30:80 -m -w 1


[root@wjb10000-backup /]# ipvsadm -a -t 192.168.1.100:80 -r 192.168.1.40:80 -m -w 1

7.6查看結果

[root@wjb10000-backup /]#  ipvsadm -L  -n


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.1.100:80 rr

  -> 192.168.1.30:80              Route   1      0          1       

  -> 192.168.1.40:80              Route   1      0          0

[root@wjb10000-backup /]# service keepalived restart

8.備服務器IP查看

[root@wjb10000-backup /]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:85:b8:f8 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.20/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe85:b8f8/64 scope link

       valid_lft forever preferred_lft forever

備註:如果主備服務器都有VIP存在,請關閉selinux和防火牆

# setenforce 0

# systemctl stop firewalld.service

三.web服務器的配置

1.此處以默認安裝好的web服務,如Nginx和Apache等等。只進行節點的配置步驟。簡單安裝apache服務,方便後面的測試。

[root@wjb10000-web1 /]# yum -y install httpd


[root@wjb10000-web1 /]# vim /var/www/html/index.html

 

<h1>web1 server</h1>


[root@wjb10000-web1 /]# systemctl start httpd.service

wKiom1dhDMOB8fi8AABrn6NR5sE431.png-wh_50


[root@wjb10000-web2 /]# yum -y install httpd


[root@wjb10000-web2 /]# vim /var/www/html/index.html


<h1>web2 server</h1>


[root@wjb10000-web2 /]# systemctl start httpd.service

wKioL1dhDeXz-RhtAABtNZipBx4615.png-wh_50

2.web1服務器LVS配置:

[root@wjb10000-web1 /]# vim /etc/init.d/lvsrs


#!/bin/bash

#description:start realserver

vip=192.168.1.100

source /etc/rc.d/init.d/functions

case $1 in

start)

        echo "Start Realserver"

        /sbin/ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up

         /sbin/route add -host $VIP dev lo:0

        echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce

        echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore

        echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

          sysctl -p >/dev/null 2>&1

;;

stop)

        echo "Stop Realserver"

        /sbin/ifconfig lo:0 down

         /sbin/route del $VIP >/dev/null 2>&1

        echo "0" > /proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "0" > /proc/sys/net/ipv4/conf/lo/arp_announce

        echo "0" > /proc/sys/net/ipv4/conf/all/arp_ignore

        echo "0" > /proc/sys/net/ipv4/conf/all/arp_announce

;;

*)

        echo "Usage: $0 (start | stop)"

exit 1

esac

[root@wjb10000-web1 /]# chmod 755 /etc/init.d/lvsrs


[root@wjb10000-web1 /]# service lvsrs start

3.web2服務器LVS配置:

[root@wjb10000-web2 ~]# vim /etc/init.d/lvsrs


#!/bin/bash

#description:start realserver

vip=192.168.1.100

source /etc/rc.d/init.d/functions

case $1 in

start)

        echo "Start Realserver"

        /sbin/ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up

         /sbin/route add -host $VIP dev lo:0

        echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce

        echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore

        echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce

          sysctl -p >/dev/null 2>&1

;;

stop)

        echo "Stop Realserver"

        /sbin/ifconfig lo:0 down

         /sbin/route del $VIP >/dev/null 2>&1

        echo "0" > /proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "0" > /proc/sys/net/ipv4/conf/lo/arp_announce

        echo "0" > /proc/sys/net/ipv4/conf/all/arp_ignore

        echo "0" > /proc/sys/net/ipv4/conf/all/arp_announce

;;

*)

        echo "Usage: $0 (start | stop)"

exit 1

esac

[root@wjb10000-web2 ~]# chmod 755 /etc/init.d/lvsrs


[root@wjb10000-web2 ~]# service lvsrs start


到此爲止,Keepalived+LVS的高可用集羣系統搭建完成,下面進行測試。

四.高可用keepalived功能測試

1.先停掉主服務器上的keepalived服務,看看備服務器是否接管vip

[root@wjb10000-master ~]# service keepalived stop


[root@wjb10000-master ~]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:74:1f:4e brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.10/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe74:1f4e/64 scope link

       valid_lft forever preferred_lft forever

2.查看備服務器的ip

[root@wjb10000-backup ~]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:85:b8:f8 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.20/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet 192.168.1.100/32 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe85:b8f8/64 scope link

       valid_lft forever preferred_lft forever

上面信息可以看到備服務器已經接管vip地址。

3.恢復主服務器keepalived服務。

[root@wjb10000-master ~]# service keepalived start


[root@wjb10000-master ~]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:74:1f:4e brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.10/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet 192.168.1.100/32 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe74:1f4e/64 scope link

       valid_lft forever preferred_lft forever

上面信息可以看到主服務器已經重新接管vip地址。

4.再查看備服務器ip地址

[root@wjb10000-backup ~]# ip add


1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

       valid_lft forever preferred_lft forever

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:85:b8:f8 brd ff:ff:ff:ff:ff:ff

    inet 192.168.1.20/24 brd 192.168.1.255 scope global eno16777736

       valid_lft forever preferred_lft forever

    inet6 fe80::20c:29ff:fe85:b8f8/64 scope link

       valid_lft forever preferred_lft forever

上面信息可以看到主服務器已經釋放了vip地址。

五.負載均衡LVS功能測試

打開瀏覽器輸入http://192.168.1.100.不斷刷新頁面分別看到web1 serverweb2 server表示LVS已經負載均衡了。

wKiom1dhDODyOKwsAABrL2_Td9E939.png-wh_50

 

wKioL1dhDf3S1n7oAABqhzCzh8E458.png-wh_50

六.故障切換測試

1.停掉web1服務器的httpd服務

[root@wjb10000-web1 ~]# systemctl stop httpd.service

2. 查看主服務器狀態

[root@wjb10000-master ~]# ipvsadm


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  wjb10000-master:http rr

  -> 192.168.1.40:http            Route   1      0          0

3.查看備服務器狀態

[root@wjb10000-backup ~]# ipvsadm


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.1.100:http rr

  -> 192.168.1.40:http            Route   1      0          0

以上內容可以看出Keepalived檢測web1服務器出現故障,將此服務器從集羣中移除。

4.恢復web1服務器

[root@wjb10000-web1 ~]# systemctl start httpd.service

5.再查看主服務器狀態

[root@wjb10000-master ~]# ipvsadm


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  wjb10000-master:http rr

  -> 192.168.1.30:http            Route   1      0          0       

  -> 192.168.1.40:http            Route   1      0          0

6.再查看備服務器狀態

[root@wjb10000-backup ~]# ipvsadm


IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.1.100:http rr

  -> 192.168.1.30:http            Route   1      0          0        

  -> 192.168.1.40:http            Route   1      0          0 

節點恢復重新加入集羣當中,瀏覽器刷新頁面,可以重新看到web1 serverweb2 server內容交替顯示。

結束語:到此Keepalived+LVS的高可用集羣系統實驗完全結束。大家哪裏不明白的歡迎留言。

 


 

 

 

 

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章