keepalive+nginx集羣架構文檔

1.準備兩臺機器

A機器：192.168.232.132

B機器：192.168.232.131

A級器作爲master，B機器作爲backup

 

2.兩臺機器都安裝：keepalived

A機器
[root@aaa-01 ~]# yum install -y keepalived
B機器
[root@bbb-01 ~]# yum install -y keepalived

 

3.兩臺機器上都安裝nginx

若是A、B機器沒有裝nginx服務，可以直接 yum安裝

因爲我A機器上已經源碼包安裝編譯過nginx了所以就不用再安裝了

B機器安裝完之後啓動nginx

[root@bbb-01 ~]# yum install -y nginx
[root@bbb-01 ~]# service nginx start
Redirecting to /bin/systemctl start nginx.service
[root@bbb-01 ~]# ps aux|grep nginx
root     16687  0.0  0.0  46364   964 ?        Ss   11:51   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
nginx    16688  0.0  0.1  46764  1924 ?        S    11:51   0:00 nginx: worker process
root     16690  0.0  0.0 112676   980 pts/0    R+   11:51   0:00 grep --color=auto nginx

注意：有時直接yum安裝不了，需要安裝yum擴展源：yum install -y epel-release

 

下面是在A機器上操作的

1.更改A機器上的更改keepalived配置文件

默認的配置文件路徑在/etc/keepalived/keepalived.conf

清空文件內容

> /etc/keepalived/keepalived.conf

編輯配置文件

vim /etc/keepalived/keepalived.conf

添加加以下內容：

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"
    interval 3
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass aminglinux>com
    }
    virtual_ipaddress {
        192.168.232.100
    }
    track_script {
        chk_nginx
    }
}

這裏需要注意的是："virtual_ipaddress"也就是所謂的vip我們設置爲192.168.232.100

 

2.定義監控腳本

腳本路徑在keepalived配置文件中有定義，路徑爲/usr/local/sbin/check_ng.sh

編輯配置文件：

vim /usr/local/sbin/check_ng.sh

增加以下內容：

#!/bin/bash
#時間變量，用於記錄日誌
d=`date --date today +%Y%m%d_%H:%M:%S`
#計算nginx進程數量
n=`ps -C nginx --no-heading|wc -l`
#如果進程爲0，則啓動nginx，並且再次檢測nginx進程數量，
#如果還爲0，說明nginx無法啓動，此時需要關閉keepalived
if [ $n -eq "0" ]; then
        /etc/init.d/nginx start
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
fi

 

3.腳本創建完之後，還需要改變腳本的權限（不更改權限，就無法自動加載腳本，那就無法啓動keepalived服務）

[root@aaa-01 ~]# chmod 755 /usr/local/sbin/check_ng.sh

 

4.啓動keepalived服務，並查看是否啓動成功

[root@aaa-01 ~]# systemctl start keepalived
[root@aaa-01 ~]# ps aux |grep keepalived
root     34653  0.0  0.1 118652  1400 ?        Ss   12:16   0:00 /usr/sbin/keepalived -D
root     34654  0.0  0.2 122852  2392 ?        S    12:16   0:00 /usr/sbin/keepalived -D
root     34655  0.0  0.2 122852  2448 ?        S    12:16   0:00 /usr/sbin/keepalived -D
root     34661  0.0  0.0 112720   988 pts/1    S+   12:16   0:00 grep --color=auto keepalived

啓動不成功，有可能是防火牆未關閉或者規則限制導致的

systemctl stop firewalld 關閉firewalld
iptables -nvL查看防火牆
setenforce 0 臨時關閉selinux
getenforce命令查看是否爲Permissive

這時再來啓動keepalived，就會看到keepalived進程服務了

 

5.這時停止nginx服務

/etc/init.d/nginx stop

再來查看nginx服務進程，會看到自動加載了

[root@aaa-01 ~]# /etc/init.d/nginx stop
Stopping nginx (via systemctl):                            [  確定  ]
[root@aaa-01 ~]# ps aux |grep nginx
root     34813  0.0  0.0  20548   628 ?        Ss   12:17   0:00 nginx: master process /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
nobody   34817  0.0  0.3  22992  3216 ?        S    12:17   0:00 nginx: worker process
nobody   34818  0.0  0.3  22992  3216 ?        S    12:17   0:00 nginx: worker process
root     34832  0.0  0.0 112720   984 pts/1    R+   12:17   0:00 grep --color=auto nginx

 

6.查看ip地址，使用 ip add 命令,可以查看到vip192.168.232.100

[root@aaa-01 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c2:05:5a brd ff:ff:ff:ff:ff:ff
    inet 192.168.232.132/24 brd 192.168.232.255 scope global dynamic ens33
       valid_lft 1559sec preferred_lft 1559sec
    inet 192.168.232.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fec2:55a/64 scope link 
       valid_lft forever preferred_lft forever

 

以下是B機器上的配置

1.自定義B機器keepalived配置文件，更改虛擬IP和主一樣的，首先清空B機器keepalived裏面自帶的配置文件

 

配置文件路徑：/etc/keepalived/keepalived.conf
清空：> /etc/keepalived/keepalived.conf

編輯配置文件：

[root@bbb-01 ~]# vim /etc/keepalived/keepalived.conf

增加以下內容：

global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_script chk_nginx {
    script "/usr/local/sbin/check_ng.sh"
    interval 3
}
vrrp_instance VI_1 {
    state BACKUP        //這裏 和master不一樣的名字
    interface eno16777736        //網卡和當前機器一致，否則無法啓動keepalived服務
    virtual_router_id 51        //和主機器 保持一致
    priority 90            //權重，要比主機器小的數值
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass aminglinux>com
    }
    virtual_ipaddress {
        192.168.74.100        //這裏更改爲192.168.232.100
    }
    track_script {
        chk_nginx
    }
}

 

2.定義監控腳本，路徑再keepalived裏面已定義過

[root@bbb-01 ~]# vim /usr/local/sbin/check_ng.sh

增加以下內容：

#時間變量，用於記錄日誌
d=`date --date today +%Y%m%d_%H:%M:%S`
#計算nginx進程數量
n=`ps -C nginx --no-heading|wc -l`
#如果進程爲0，則啓動nginx，並且再次檢測nginx進程數量，
#如果還爲0，說明nginx無法啓動，此時需要關閉keepalived
if [ $n -eq "0" ]; then
        systemctl start nginx
        n2=`ps -C nginx --no-heading|wc -l`
        if [ $n2 -eq "0"  ]; then
                echo "$d nginx down,keepalived will stop" >> /var/log/check_ng.log
                systemctl stop keepalived
        fi
fi

 

 

3.改動腳本的權限，設置爲755權限

[root@bbb-01 ~]# chmod 755 /usr/local/sbin/check_ng.sh

 

4.啓動keepalived服務：

[root@bbb-01 ~]# systemctl start keepalived
[root@bbb-01 ~]# ps aux |grep keep
root     19134  0.0  0.1 118608  1384 ?        Ss   12:33   0:00 /usr/sbin/keepalived -D
root     19135  0.0  0.2 122804  2384 ?        S    12:33   0:00 /usr/sbin/keepalived -D
root     19136  0.0  0.2 122804  2424 ?        S    12:33   0:00 /usr/sbin/keepalived -D
root     19143  0.0  0.0 112676   984 pts/0    R+   12:33   0:00 grep --color=auto keep

 

區分主和從上的nginx

1.A機器，是源碼包安裝的nginx（PS：這是lnmp配置好的環境虛擬主機內容）

[root@aaa-01]# cat /usr/local/nginx/conf/vhost/aaa.com.conf
server
{
    listen 80 default_server; 
    server_name aaa.com;
    index index.html index.htm index.php;
    root /data/wwwroot/default;
}

2.索引頁：

[root@aaa-01~]# cat /data/wwwroot/default/index.html 
This is the default sete.
[root@aaa-01~]# vim /data/wwwroot/default/index.html   #增加內容 
master This is the default sete.

3.用網頁查看

B機器是yum安裝的nginx

默認的索引頁在 /usr/share/nginx/html/index.html

[root@bbb-01 ~]# vim /usr/share/nginx/html/index.html    #增加內容

backup backup.

網頁查看

訪問192.168.132.100這個VIP會看到和主機器（即A機器相同的內容），說明現在訪問到的是機器master，VIP在master上

 

測試高可用

1.模擬線上生產環境，主機器宕機環境，最簡單直接的方法，就是直接關閉keepalived服務

關閉master機器（即A機器）上的keepalived服務關閉

查看A機器上的VIP被已經釋放掉了

[root@aaa-01 ~]# systemctl start keepalived
[root@aaa-01 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:c2:05:5a brd ff:ff:ff:ff:ff:ff
    inet 192.168.232.132/24 brd 192.168.232.255 scope global dynamic ens33
       valid_lft 1532sec preferred_lft 1532sec
    inet6 fe80::20c:29ff:fec2:55a/64 scope link 
       valid_lft forever preferred_lft forever

 

2.查看backup機器（即B機器）發現它在監聽VIP

[root@bbb-01 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:fb:fc:6f brd ff:ff:ff:ff:ff:ff
    inet 192.168.232.131/24 brd 192.168.232.255 scope global dynamic ens33
       valid_lft 1179sec preferred_lft 1179sec
    inet 192.168.232.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fefb:fc6f/64 scope link 
       valid_lft forever preferred_lft forever

 

3.瀏覽器訪問vip，會看到已經變成backup機器上的了

這證明成功了，當A機器出現問題B機器就頂了上去