LVS+Keepalived負載均衡及高可用WEB服務器集羣

原創 fslc2004 2018-09-12 02:31

環境：
操作系統Centos 6.5 X86_64(final)
LVS-Master:192.168.2.32
LVS-Backup:192.168.3.31
VIP:192.168.2.33
Web2:192.168.2.29
Web4:192.168.2.30


1、安裝LVS軟件（分別在LVS-Master和LVS-Backup中）
1.1.查看是否有IPVS模塊，LVS是Linux標準內核的一部分,直接被編譯在內核中!
[root@LVS-Master ~]# modprobe -l | grep ipvs    #使用modprobe命令查看
kernel/net/netfilter/ipvs/ip_vs.ko
kernel/net/netfilter/ipvs/ip_vs_rr.ko
kernel/net/netfilter/ipvs/ip_vs_wrr.ko
kernel/net/netfilter/ipvs/ip_vs_lc.ko
kernel/net/netfilter/ipvs/ip_vs_wlc.ko
kernel/net/netfilter/ipvs/ip_vs_lblc.ko
kernel/net/netfilter/ipvs/ip_vs_lblcr.ko
kernel/net/netfilter/ipvs/ip_vs_dh.ko
kernel/net/netfilter/ipvs/ip_vs_sh.ko
kernel/net/netfilter/ipvs/ip_vs_sed.ko
kernel/net/netfilter/ipvs/ip_vs_nq.ko
kernel/net/netfilter/ipvs/ip_vs_ftp.ko
kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko


1.2.安裝IPVS管理工具
[root@LVS-Master ~]# yum install ipvsadm

2、安裝keepalived（分別在LVS-Master和LVS-Backup中）
2.1.安裝keepalived並配置。
[root@LVS-Master ~]# yum install keepalived
[root@LVS-Master ~]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]       
     [email protected]         #設置報警地址，可以設置多個，每行1個
     [email protected]         #需安裝sendmail，並開啓郵件報警
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1           #設置SMTP Server地址
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER                   #指定Keepalived的角色，MASTER爲主，BACKUP爲備
    interface eth0                 #綁定真實的物理網卡
    virtual_router_id 51           #虛擬路由編號，主備要一致
    priority 100           #定義優先級，數字越大，優先級越高，主DR必須大於備用DR    
    advert_int 1           #檢查間隔，默認爲1s
    authentication {
        auth_type PASS            #認證方式，PASS或AH兩種
        auth_pass 1111           #認證密碼，主備服務器要一致
    }
    virtual_ipaddress {
        192.168.2.33             #定義虛擬IP(VIP)爲192.168.2.33，可多設，每行一個
    }
}


virtual_server 192.168.2.33 80 {   #定義對外提供服務的LVS的VIP以及port
    delay_loop 6            #設置健康檢查時間，單位是秒
    lb_algo wlc               #設置負載調度的算法爲wlc
    lb_kind DR               #設置LVS實現負載的機制，有NAT、TUN、DR三個模式
    persistence_timeout 50         #會話保持時間，單位爲秒
    protocol TCP             #指定轉發協議類型，有TCP和UDP兩種

    real_server 192.168.2.29 80 {  #指定real server1的IP地址
        weight 100             #配置節點權值，數字越大
        TCP_CHECK {
            connect_timeout 3      #表示3秒無響應，則超時
            nb_get_retry 3        #表示重試次數
            delay_before_retry 3   #表示重試間隔      
        }
    }

    real_server 192.168.2.30 80 {  #指定real server2的IP地址
        weight 100            #配置節點權值，數字越大
        TCP_CHECK {
            connect_timeout 3      #表示3秒無響應，則超時
            nb_get_retry 3       #表示重試次數
            delay_before_retry 3   #表示重試間隔
        }
    }
}

2.2.啓動keepalived、設置keepalived服務自啓動，檢查是否生效。

[root@LVS-Master ~]# /etc/init.d/keepalived start
Starting keepalived:                     [  OK  ]

[root@LVS-Master ~]# chkconfig keepalived on

[root@LVS-Master keepalived]# ipvsadm -L -n           #查看ipvs規則是否生效
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.2.33:80 wlc persistent 50
  -> 192.168.2.29:80              Route   100    0          0         
  -> 192.168.2.30:80              Route   100    0          0


3、在LVS-Backup上安裝LVS、keepalived並配置
3.1.安裝LVS、keepalived
[root@LVS-Backup ~]# yum install ipvsadm
[root@LVS-Backup ~]# yum install keepalived

3.2.配置LVS-Backup的keepalived
先從LVS－Master把修改過的keepalived配置文件複製過來。
[root@LVS-Master keepalived]# scp keepalived.conf [email protected]:/etc/keepalived/

[root@LVS-Backup ~]# vi /etc/keepalived/keepalived.conf
只修改state爲BACKUP和priority爲90，MASTER爲主機服務器，BACKUP爲備用服務器。
    state BACKUP
    priority 90

3.3.啓動LVS-Backup的keeplived服務，並檢查是否生效。
[root@LVS-Backup ~]# /etc/init.d/keepalived start

[root@LVS-Backup ~]# chkconfig keepalived on

[root@LVS-Backup keepalived]# ipvsadm -L -n         #查看ipvs規則是否生效
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.2.33:80 wlc persistent 50
  -> 192.168.2.29:80              Route   100    0          0         
  -> 192.168.2.30:80              Route   100    0          0

4、測試VIP，檢查是否能主備切換。
4.1.啓動keeplived後，先用ip addr顯示主備服務器的VIP情況，當主備服務器同時啓用keepalived時，只有主服務器擁有VIP地址，備服務器沒有。
[root@LVS-Master ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:a6:00:13 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.32/24 brd 192.168.2.255 scope global eth0
    inet 192.168.2.33/32 scope global eth0                      #可以看到主服務器擁有192.168.2.33這個VIP地址
    inet6 fe80::250:56ff:fea6:13/64 scope link
       valid_lft forever preferred_lft forever

[root@LVS-Backup ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 08:00:27:77:d3:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.31/24 brd 192.168.2.255 scope global eth0   #可以到看備服務器沒有192.168.2.33這個VIP地址
    inet6 fe80::a00:27ff:fe77:d382/64 scope link
       valid_lft forever preferred_lft forever

4.2.當停止主服務器的keepalived服務，再查看下主備服務器VIP地址，發現VIP地址已從主服務器轉移到了備服務器
[root@LVS-Master ~]# service keepalived stop
Stopping keepalived:                [  OK  ]

[root@LVS-Master keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:a6:00:13 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.32/24 brd 192.168.2.255 scope global eth0         #主服務器keepalived停止後，VIP地址將移除
    inet6 fe80::250:56ff:fea6:13/64 scope link
       valid_lft forever preferred_lft forever

[root@LVS-Backup ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 08:00:27:77:d3:82 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.31/24 brd 192.168.2.255 scope global eth0        
    inet 192.168.2.33/32 scope global eth0                  #可以看到VIP地址已轉移到備服務器，繼續提供服務
    inet6 fe80::a00:27ff:fe77:d382/64 scope link
       valid_lft forever preferred_lft forever


4.3.日誌查看主備切換
當主服務器停止keepalived後，主備服務器相關日誌
[root@LVS-Master ~]# service keepalived stop
Stopping keepalived:                [  OK  ]
主服務器日誌
Sep  2 17:39:13 LVS-Master kernel: IPVS: __ip_vs_del_service: enter   
Sep  2 17:39:13 LVS-Master Keepalived[13192]: Stopping Keepalived v1.2.7 (02/21,2013)              #主服務器停掉後            
Sep  2 17:39:13 LVS-Master Keepalived_vrrp[13195]: VRRP_Instance(VI_1) sending 0 priority     
Sep  2 17:39:13 LVS-Master Keepalived_vrrp[13195]: VRRP_Instance(VI_1) removing protocol VIPs.

備服務器日誌

Sep  2 17:39:14 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  2 17:39:15 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Entering MASTER STATE         #備轉爲MASTER，並設置VIP地址爲192.168.2.33
Sep  2 17:39:15 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep  2 17:39:15 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33
Sep  2 17:39:15 LVS-Backup Keepalived_healthcheckers[978]: Netlink reflector reports IP 192.168.2.33 added
Sep  2 17:39:20 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33



當主服務器重啓keepalived後，主備服務器相關日誌
[root@LVS-Master ~]# service keepalived start
Starting keepalived:                 [  OK  ]
主服務器日誌
Sep  2 17:41:28 LVS-Master Keepalived_vrrp[13231]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep  2 17:41:29 LVS-Master Keepalived_vrrp[13231]: VRRP_Instance(VI_1) Entering MASTER STATE       #主服務器重啓後，從新接手MASTER角色，並拿回VIP地址192.168.2.33        
Sep  2 17:41:29 LVS-Master Keepalived_vrrp[13231]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep  2 17:41:29 LVS-Master Keepalived_healthcheckers[13230]: Netlink reflector reports IP 192.168.2.33 added
Sep  2 17:41:29 LVS-Master Keepalived_vrrp[13231]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.2.33

備服務器日誌
Sep  2 17:41:28 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Received higher prio advert
Sep  2 17:41:28 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) Entering BACKUP STATE         #備服務器從MASTER又變回了BACKUP，並移除VIP地址192.168.2.33
Sep  2 17:41:28 LVS-Backup Keepalived_vrrp[979]: VRRP_Instance(VI_1) removing protocol VIPs.
Sep  2 17:41:28 LVS-Backup Keepalived_healthcheckers[978]: Netlink reflector reports IP 192.168.2.33 removed

5、配置WEB服務器，即real server節點1和2

由於採用的是DR方式調度，Real_Server會以LVS的VIP來直接回復Client，所以需要在Real_Server的lo上開啓LVS的VIP來與Client建立通信
配置web2
[root@web2 ~]# vi /etc/init.d/lvsrs                ＃編輯一個lvsrs腳本，並放到/etc/init.d/下

#!/bin/bash

#description : start Real Server

VIP=192.168.2.33

./etc/rc.d/init.d/functions

case "$1" in

start)

echo " Start LVS of Real Server "

/sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up

/sbin/route add -host $VIP dev lo:0

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore                 

echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

;;

stop)

/sbin/ifconfig lo:0 down

echo "close LVS Director server"

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

;;

*)

echo "Usage: $0 {start|stop}"

exit 1

esac


[root@web2 ~]# chmod 755 /etc/init.d/lvsrs                #給新增lvsrs腳本一個755權限

[root@web2 ~]# service lvsrs start              #啓動lvsrs服務，我環境啓動時出錯，給一個functions 755權限，按各自環境操作    
/etc/init.d/lvsrs: line 7: ./etc/rc.d/init.d/functions: Permission denied
 Start LVS of Real Server

[root@web2 init.d]# chmod 755 /etc/rc.d/init.d/functions  #給functions 755權限

[root@web2 ~]# service lvsrs start                      #啓動lvsrs服務
 Start LVS of Real Server

[root@web2 ~]# ifconfig                      #查看已有一個VIP地址
lo:0      Link encap:Local Loopback  
          inet addr:192.168.2.33  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

[root@web2 ~]# echo "service lvsrs start" >> /etc/rc.local   #把lvsrs服務添加到開機啓動，否則下次啓動電腦後，還需手動啓動。


[root@web2 ~]# yum install httpd
[root@web2 ~]# vi /var/www/html/index.html
<h1>Welcome to web2(192.168.2.29)</h1>

[root@web2 ~]# service httpd start


配置web4


配置同上
[root@web4 ~]# scp [email protected]:/etc/init.d/lvsrs /etc/init.d/
[root@web4 ~]# chmod 755 /etc/rc.d/init.d/functions
[root@web4 ~]# service lvsrs start
 Start LVS of Real Server

[root@web4 ~]# echo "service lvsrs start" >> /etc/rc.local


[root@web4 ~]# ifconfig
lo:0      Link encap:Local Loopback  
          inet addr:192.168.2.33  Mask:255.255.255.255
          UP LOOPBACK RUNNING  MTU:16436  Metric:1

[root@web4 ~]# yum install httpd

[root@web4 ~]# vi /var/www/html/index.html
<h1>Welcome to web4(192.168.2.30)</h1>

[root@web4 ~]# service httpd start

6、負載均衡以及高可用測試
6.1.負載均衡測試
（1）、兩臺WEB服務器都工作正常，訪問VIP地址時，可以被負載到兩臺WEB服務器上。
（2）、停掉一臺WEB服務器的http服務後，訪問VIP地址時，被負載到正常的WEB服務器上。
6.2.高可用測試
（1）、兩臺DR的keepalived服務器正常時，VIP地址在主服務器，提供正常服務。
（2）、當關閉主服務器的keepalived後，VIP地址自動轉移到備服務器上，當主服務器重啓keepalived服務後，VIP自動轉移回主服務器，備服務器繼續備用，實現了高可用負載均衡。






參考如下：
http://www.jizhuomi.com/software/351.html
http://nmshuishui.blog.51cto.com/1850554/1405745
http://czybl.blog.51cto.com/4283444/1536474
http://zhumeng8337797.blog.163.com/blog/static/1007689142011101392553489/












發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Linux 開機(腳本)啓動順序

Uchen 2019-02-24 13:53:20

Linux基本操作命令

wbzjacky 2019-02-24 13:12:38

tar Command Daily Work Summary

海膽階段 2019-02-24 12:58:10

SSH and SCP Daily Work Summary

海膽階段 2019-02-24 12:58:10

Linux積累-安裝與配置Hadoop

lftong 2019-02-23 14:00:30

centos ***

samplelife 2019-02-23 13:57:36

新手學習Linux系統的一點見解

jackieban 2019-02-23 13:57:24

CentOS 6.4下PXE+Kickstart無人值守安裝操作系統

paul8339 2019-02-23 13:56:09

Linux學習之旅 - 第一天

lichen_zt 2019-02-23 13:55:58

Linux I/O重定向以及正則表達式

zhongqijian916 2019-02-23 13:54:50

Linux---facl以及終端

zhongqijian916 2019-02-23 13:54:50

Easy ***實驗

wbzjacky 2019-02-24 13:12:37

Spring Cloud Alibaba基礎教程：支持的幾種服務消費方式（RestTemplate、WebClient、Feign）

程序猿dd 2019-02-24 10:41:27

tcpdump查看Nginx長連接還是短連接

ouyangbro 2019-02-23 15:52:48

50元打造雙網卡負載均衡服務器

138web 2019-02-23 13:55:58