lvs dr keepalive

      lvs dr keepalive

Lvs linux virtual server linux虛擬服務器，是個網絡第四層的負載均衡羣集的調度器，是個軟件實現。lvs實現在linux內核中，其模塊名爲ipvs，用戶管理命令爲ipvsadm(rpm軟件包ipvsadm提供），ipvs工作在netfiler的input鏈上，有經過input鏈的且符合定義羣集服務的流量，強行改變其方向，根據調度算法調度到響應real server上去。

Lvs類型有四種：

1.nat 2.dr 3.tun 4.fullnat

Lvs-Nat主要特點：1.vip爲公網地址，dip、rip爲私網地址，且dip、rip必須在同一個物理網絡裏，且每個real server節點的網關必須指向dip. 2. Cip的請求報文及rip的響應報文都經過director,所以director很容易成爲網絡瓶頸。3.nat 其實就是多目標的dnat,所以支持端口映射。4.director主機的操作系統一般爲linux,而real server 主機操作系統，可以爲提供羣集服務的任何操作系統。

Lvs-Dr的主要特點：1. Vip爲公網的地址，dip、rip可以是公網地址，也可以是私網地址。2.cip的請求報文經過director調度，但rip的響應報文不經過director,real server的網關一定不能指向dir,所以此種調度方式，效率極高。

3.由於調度時通過修改mac地址進行，不修改第三，四層信息，所以不支持端口映射。4.dip與rip必須在同一個物理網絡裏。5.director的操作系統一般爲linux,而real server 的操作系統，必須支持lo網卡別名，管理arp響應級別和通告級別的機制。

Tun的主要特點：1. Vip dip rip都是公網地址2.不修改報文的ip首部，而是在client 的請求報文之外再封裝一個ip首部 ，源ip是dip,目標ip是rip 3.rs的網關一定不能指向dip 4.不支持端口映射5.rs必須支持隧道功能。

Lvs-Fullnat 的主要特點：1.同時修改請求報文的源ip,目標ip cip改爲dip, vip改爲rip,因此director與rs可以用路由器分割跨網，其他特點以lvs-nat一樣。

Lvs scheduler十種調度方法：

根據其調度時是否考慮個各RS當前的負載狀態，可分爲靜態和動態調度方法：

根據算法本身靜態調度有四種：

1. rr---簡單的輪循調度，起點公平

2. Wrr-----加權重的輪循調度，根據權重比例來輪循調度

3. Sh----原地址hash 根據原地址生成一個原地址hash表進行調度，在記錄的有效的時間內，相同原地址始終發往同一個real server ,可以在第四層進行會話綁定。

4. Dh ----目標地址hash，一般用在正向代理，根據相同目標地址始終發往同一個real server.

根據RS的實時的負載進行動態調度有六種：

1.lc------最少連接 誰連接最少，最先被選中。計算方法：Overhead=activeconns*256 +inactivconns

2.Wlc--加權重的最少連接數,計算方法：Overhead=(activeconns*256+inactivconns)/weighted

3.Sed--最短的期望延遲，讓性能最好的real server 來響應。計算方法：Overhead=(activeconns+1)*256/weighted

4.Nq:nerver queue 不能出現有real server 有隊列，而有的real server 無請求。

5.LbLC：動態的dh算法。Locality-based least connections 基於本地的最少連接

6.LBLCR：LBLC with Replication

十種算法的默認算法是wlc

四種lvs類型的默認類型是dr

 

 

 

Lvs dr keepalive  實現一個高可用的lvs 的基於web服務的羣集實驗

實驗環境：

四臺主機: 兩臺主機centos7.2A centos7.2B 做基於keepAlived 高可用的Lvs dr

另兩臺主機：centos6.7A centos6.7B 各做一個網站

在網段172.16.0.0/16網絡已有dhcp server  172.16.0.1/16（172.16.0.0網段的）

Ntp server 172.16.0.1

Centos7.2A 一塊網卡 橋接模式 ip 172.16.254.79/16

Centos7.2B  一塊網卡 橋接模式 ip 172.16.250.205/16

Cenots6.7A 一塊網卡 橋接模式 ip 172.16.251.237/16

Centos 6.7B 一塊網卡 橋接模式 ip 172.16.251.178/16

爲了做實驗的方便，關閉掉所有主機的防火牆及selinux：

Centos7.2A centos7.2B systemctl stop firewalld  setenforce 0

Centoos6.7A centos6.7B service iptables stop    setenforce 0

 

Vim /etc/keepalived/keepalived.conf

global_defs {

   notification_email {

          root@localhost

   }

   notification_email_from keepalived@localhost

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id router1

   vrrp_mcast_group4 224.10.10.10

}

 

vrrp_instance VI_1 {

    state MASTER

    interface eno16777728

    virtual_router_id 22

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 11112222

    }

    virtual_ipaddress {

        172.16.100.100/32 dev eno16777728 brd 172.16.100.100 label eno16777728:0

}

Notify_master  “/scripts/vrrpstatuschange.sh master”

Notify_backup  “/scripts/vrrpstatuschange.sh backup”

Notify_fault  “/scripts/vrrpstatuschange.sh fault”

 

}

virtual_server 172.16.100.100 80 {

    delay_loop 3

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.0.0

#    persistence_timeout 50

    protocol TCP

    sorry_server 127.0.0.1 80

 

    real_server 172.16.251.237 80 {

        weight 1

        HTTP_GET {

            url {

              path /index.html

            #  digest ff20ad2481f97b1754ef3e12ecd3a9cc

                status_code 200

            }

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

     }

    real_server 172.16.251.178 80 {

        weight 1

        HTTP_GET {

            url {

              path /index.html

 #  digest ff20ad2481f97b1754ef3e12ecd3a9cc

                status_code 200

            }

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

Sorry server 網頁

Vim /var/www/html/index.html

<h1> sorry server centos6.7A </h1>

Vim /scripts/vrrpstatuschange.sh

#!/bin/bash

recv='root@localhost'

notify() {

      mailsubject="$(hostname) to be $1,vip floating"

      mailbody="$(date +'%F %T'):vrrp virtual address 172.16.100.100 trans , $(hostname) change to be $1"

      echo "$mailbody" | mail -s "$mailsubject"  $recv

}

case $1 in

master)

     notify master

;;

backup)

      notify backup

;;

fault)

    notify fault

;;

*)

echo "error Usage: $(basename $0) master|backup|fault"

esac

Systemctl start keepalived

 

Ipvsadm -L -n

[root@localhost Desktop]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  172.16.100.100:80 rr

  -> 172.16.251.178:80            Route   1      0          0         

  -> 172.16.251.237:80            Route   1      0          0  

把後面兩個網站都關閉：

Service httpd stop

 

Ipvsadm -L -n [root@localhost Desktop]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  172.16.100.100:80 rr

  -> 127.0.0.1:80                 Route   1      0          0    ----- 調度sorry server

Curl http://172.16.100.100

[root@localhost Desktop]# curl http://172.16.100.100

<h1> say sorry 7.2A </h1>

 

 

 

 

Cenots7.2A  yum -y install ipvsadm keepalived httpd

   ntpdate 172.16.0.1

Sorry server

Vim /var/www/html/index.html

<h1> sorry server centos7.2B</h1>

Vim /etc/keepalive/keepalived.conf

! Configuration File for keepalived

 

global_defs {

   notification_email {

          root@localhost

   }

   notification_email_from keepalived@localhost

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id router2

   vrrp_mcast_group4 224.10.10.10

}

 

vrrp_instance VI_1 {

    state BACKUP

    interface eno16777728

    virtual_router_id 22

    priority 90

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 11112222

    }

    virtual_ipaddress {

        172.16.100.100/32 dev eno16777728 brd 172.16.100.100 label eno16777728:0

}

Notify_master  “/scripts/vrrpstatuschange.sh master”

Notify_backup  “/scripts/vrrpstatuschange.sh backup”

Notify_fault  “/scripts/vrrpstatuschange.sh fault”

 

}

virtual_server 172.16.100.100 80 {

    delay_loop 3

    lb_algo rr

    lb_kind DR

    nat_mask 255.255.0.0

#    persistence_timeout 50

    protocol TCP

    sorry_server 127.0.0.1 80

 

    real_server 172.16.251.237 80 {

        weight 1

        HTTP_GET {

            url {

              path /index.html

            #  digest ff20ad2481f97b1754ef3e12ecd3a9cc

                status_code 200

            }

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

   }

    real_server 172.16.251.178 80 {

        weight 1

                         HTTP_GET {

            url {

              path /index.html

            #  digest ff20ad2481f97b1754ef3e12ecd3a9cc

                status_code 200

            }

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

                           

Vim /scripts/vrrpstatuschange.sh

 

             #!/bin/bash

recv='root@localhost'

notify() {

      mailsubject="$(hostname) to be $1,vip floating"

      mailbody="$(date +'%F %T'):vrrp virtual address 172.16.100.100 trans , $(hostname) change to be $1"

      echo "$mailbody" | mail -s "$mailsubject"  $recv

}

case $1 in

master)

     notify master

;;

backup)

      notify backup

;;

fault)

    notify fault

;;

*)

echo "error Usage: $(basename $0) master|backup|fault"

esac

                                  

 

 

 

Cenots6.7A 配置：

                [root@www ~]# cat /scripts/lvsdr.sh    

#!/bin/bash

vip=172.16.100.100

netmask=255.255.255.255

a=`ifconfig | grep lo:0 | awk '{print $1}'`

case $1 in

enable)

     if [ "$a" == lo:0 ];then

      echo "lvsdr is already enable"

    else

     echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

     echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

     echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

     echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

     ifconfig lo:0 $vip netmask $netmask  broadcast $vip up

     route add -host $vip dev lo:0

   fi

;;

disable)

     if [  "$a" == lo:0 ] ;then

     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

     echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

     echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce

     ifconfig lo:0 down

   else

      echo "lvsdr is already disable"

fi

;;

*)

  echo "error:Usage:$(basename $0)  enable|disable"

;;

Esac

/scirpts/lvsdr.sh enable

顯示結果：

                       

Ifconfig

lo:0      Link encap:Local Loopback  

          inet addr:172.16.100.100  Mask:255.255.255.255

          UP LOOPBACK RUNNING  MTU:65536  Metric:1

Route -n

[root@www ~]# route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

172.16.100.100  0.0.0.0         255.255.255.255 UH    0      0        0 lo

172.16.0.0      0.0.0.0         255.255.0.0     U     1      0        0 eth0

0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0

網站首頁：

Vim /var/www/html/index.html

<h1> centos6.7A</h1>

 

 

Centos6.7B 配置：               [root@www ~]# cat /scripts/lvsdr.sh    

#!/bin/bash

vip=172.16.100.100

netmask=255.255.255.255

a=`ifconfig | grep lo:0 | awk '{print $1}'`

case $1 in

enable)

     if [ "$a" == lo:0 ];then

      echo "lvsdr is already enable"

    else

     echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

     echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

     echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

     echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

     ifconfig lo:0 $vip netmask $netmask  broadcast $vip up

     route add -host $vip dev lo:0

   fi

;;

disable)

     if [  "$a" == lo:0 ] ;then

     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore

     echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore

     echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce

     echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce

     ifconfig lo:0 down

   else

      echo "lvsdr is already disable"

fi

;;

*)

  echo "error:Usage:$(basename $0)  enable|disable"

;;

Esac

/scirpts/lvsdr.sh enable

顯示結果：

                       

Ifconfig

lo:0      Link encap:Local Loopback  

          inet addr:172.16.100.100  Mask:255.255.255.255

          UP LOOPBACK RUNNING  MTU:65536  Metric:1

Route -n

[root@www ~]# route -n

Kernel IP routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface

172.16.100.100  0.0.0.0         255.255.255.255 UH    0      0        0 lo

172.16.0.0      0.0.0.0         255.255.0.0     U     1      0        0 eth0

0.0.0.0         172.16.0.1      0.0.0.0         UG    0      0        0 eth0

網站首頁：

Vim /var/www/html/index.html

<h1> centos6.7B</h1>

 

 

訪問四個網站：

ocalhost Desktop]# curl http://172.16.254.79

<h1> say sorry 7.2A </h1>

[root@localhost Desktop]# curl http://172.16.250.205

<h1> say sorry 7.2B </h1>

[root@localhost Desktop]# curl http://172.16.251.237

<h1> centos6.7A</h1>

 

[root@localhost Desktop]# curl http://172.16.251.178

<h1> centos6.7B</h1>

 

當centos7.2A 作爲lvs master時：

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7A</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7A</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7A</h1>

兩個站點輪循訪問

 

關閉centos6.7A 站點：

Service httpd stop

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

只能訪問centos6.7B 站點了

 

把centos7.2A 的keepalvied 關閉：

Systemctl keepalived stop[root@localhost Desktop]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  172.16.100.100:80 rr

  -> 172.16.251.178:80            Route   1      0          0   

 

Centos 7.2B keepalived 開始變成master[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

 

[root@www Desktop]# curl http://172.16.100.100

<h1> centos6.7B</h1>

Centos 6.7B 站點照樣可以訪問：

Keepalived服務達到了lvs dr 高可用。

 

在關閉centos6.7B 的站點

Service httpd stop

 

Centos7.2B

[root@localhost Desktop]# ipvsadm -L -n

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  172.16.100.100:80 rr

  -> 127.0.0.1:80                 Route   1      0          0  

 

 

[root@www Desktop]# curl http://172.16.100.100

<h1> say sorry 7.2B </h1>----------------sorry server 訪問正常