項目需求
當我們訪問某個網站的時候可以在瀏覽器中輸入IP或者域名鏈接到Web Server進行訪問,如果這個Web Server掛了,那麼整個系統都無法使用,用戶也就不能進行正常的訪問,這種情況將對公司產生一定的影響。這就是我們常說的系統中的單點故障。這部分的單點故障可以通過引入負載均衡器和至少另一個Web Server來緩解。同時由於有多臺服務器同時提供服務,也加大了系統的負載能力提高了性能。
因此我們採用LVS的負載均衡技術,將前端請求按照設定規則調度到後端服務器,並與keepalived相結合實現高可用負載均衡。
項目拓撲
項目環境
主機名 | 主機IP | 主機角色 |
---|---|---|
K1 | 192.168.36.110 | Keepalived-Master |
K2 | 192.168.36.111 | Keepalived-Backup |
WEB1 | 192.168.36.112 | Nginx、PHP |
WEB2 | 192.168.36.113 | Nginx、PHP |
NFS | 192.168.36.114 | NFS |
Mariadb-M | 192.168.36.115 | Mariadb-Master |
Mariadb-S | 192.168.36.116 | Mariadb-Slave |
開始部署:Keepalived服務器配置
安裝Keepalived
[root@k1 ~]#yum install -y keepalived
修改Keepalived配置文件,開啓郵件通知功能
[root@k1 ~]#vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs { # 全局配置
notification_email {
[email protected] # keepalived 發生故障切換時郵件發送的對象,可以按行區分寫多個
}
notification_email_from [email protected] # 設置郵件發送地址
smtp_server 127.0.0.1 # smtp服務器地址
smtp_connect_timeout 30 # 指定smtp連接超時時間
router_id K1.mylinuxops.com # 運行keepalived服務器標識,發送郵件時顯示在郵件標題中的信息
vrrp_skip_check_adv_addr # 所有報文都檢查比較消耗性能,此配置爲如果收到的報文和上一個報文是同一個路由器則跳過檢查報文中的源地址
#vrrp_strict # 嚴格遵守VRRP協議,不允許狀況:1,沒有VIP地址,2.單播鄰居,3.在VRRP版本2中有IPv6地址
vrrp_iptables # 嚴格遵守VRRP防火牆規則
vrrp_garp_interval 0 # ARP保溫發送延遲
vrrp_gna_interval 0 # 消息發送延遲
}
vrrp_instance VI_1 { # vrrp實例定義
state MASTER # 在此虛擬路由器上節點的初始狀態:其中所有服務器裏只有一個可以是MASTER節點,其餘的是BACKUP節點
interface ens33 # 指定HA檢測網絡的接口,即網卡名稱
virtual_router_id 27 # 當前虛擬路由器的惟一標識,範圍是0-255
priority 100 # 當前主機在此虛擬路徑器中的優先級;範圍1-254。主服務器一定要高於備用服務器,且兩者之間的數值差越小越好
advert_int 1 # vrrp通告間隔
authentication { # 存儲的驗證類型和密碼以進行驗證
auth_type PASS # 進行驗證類型:類型僅可以設置成PASS和AH兩種
auth_pass 1111 # 進行驗證的密碼:在同一個vrrp_instance中,使用相同的密碼才能進行正確的通信
}
unicast_src_ip 192.168.36.110 # 單播配置
unicast_peer {
192.168.36.111 # 目標主機IP
}
virtual_ipaddress { # 虛擬IP的網絡地址,即VIP地址
192.168.36.100 dev ens33 label ens33:0
192.168.36.200 dev ens33 label ens33:1
}
# 定義郵件通知腳本
notify_master "/etc/keepalived/notify.sh master" # 當前節點成爲主節點時觸發的腳本
notify_backup "/etc/keepalived/notify.sh backup" # 當前節點轉爲備節點時觸發的腳本
notify_fault "/etc/keepalived/notify.sh fault" # 當前節點轉爲“失敗”狀態時觸發的腳本
}}
# 編寫郵件通知腳本
[root@k1 ~]#vim /etc/keepalived/notify.sh
#!/bin/bash
contact='[email protected]' # 通知的郵箱(首先需要確保能連通外網,否則通知不過去)
notify() {
mailsubject="$(hostname) to be $1, vip轉移" # 主題
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1" # 郵件內容
echo "$mailbody" | mail -s "$mailsubject" $contact # 郵件發送的內容
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
# 添加執行權限
[root@k1 ~]#chmod a+x /etc/keepalived/notify.sh
# 郵箱配置
[root@k1 ~]#yum install -y mailx
[root@k1 ~]#vim /etc/mail.rc
set bsdcompat
set [email protected] # 接收郵件的郵箱
set smtp=smtp.qq.com
set [email protected]
set smtp-auth-password=kosulaxbbhxrgaci # 郵箱授權碼(我的郵箱-->設置-->賬戶-->開啓POP3/SMTP等服務,生成授權碼)
[root@k1 ~]#chmod a+x /etc/mail.rc # 文件添加執行權限
# 重啓keepalived服務
[root@k1 ~]#systemctl restart keepalived
# 生成VIP地址(注:Master存活時,VIP在Master上,Slave上不會存在VIP。當Master宕機,VIP將調到Slave上)
[root@k1 ~]#ifconfig
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.36.100 netmask 255.255.255.255 broadcast 0.0.0.0
ether 00:0c:29:56:39:e8 txqueuelen 1000 (Ethernet)
ens33:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.36.200 netmask 255.255.255.255 broadcast 0.0.0.0
ether 00:0c:29:56:39:e8 txqueuelen 1000 (Ethernet)
# 將keepalived配置文件 scp 到BACKUP服務器中
[root@k1 ~]#scp /etc/keepalived/keepalived.conf 192.168.36.104:/etc/keepalived/keepalived.conf
[email protected]'s password:
keepalived.conf 100% 1374 1.4MB/s 00:00
# BACKUP服務器配置,其餘配置與MASTER相同
[root@k1 ~]#vim /etc/keepalived/keepalived.conf
....
state BACKUP # 修改爲BACKUP節點
interface ens33
virtual_router_id 37 # 修改標識爲37
priority 90 # 修改優先級,需要比 MASTER 節點低
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
unicast_src_ip 192.168.36.111
unicast_peer {
192.168.36.110
}
....
# 編寫郵件通知腳本
[root@k1 ~]#vim /etc/keepalived/notify.sh
#!/bin/bash
contact='[email protected]' # 通知的郵箱(首先需要確保能連通外網,否則通知不過去)
notify() {
mailsubject="$(hostname) to be $1, vip轉移" # 主題
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1" # 郵件內容
echo "$mailbody" | mail -s "$mailsubject" $contact # 郵件發送的內容
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
# 添加執行權限
[root@k1 ~]#chmod a+x /etc/keepalived/notify.sh
# 郵箱配置
[root@k1 ~]#yum install -y mailx
[root@k1 ~]#vim /etc/mail.rc
set bsdcompat
set [email protected] # 接收郵件的郵箱
set smtp=smtp.qq.com
set [email protected]
set smtp-auth-password=kosulaxbbhxrgaci # 郵箱授權碼(我的郵箱-->設置-->賬戶-->開啓POP3/SMTP等服務,生成授權碼)
[root@k1 ~]#chmod a+x /etc/mail.rc # 文件添加執行權限
# 重啓keepalived服務,並進行宕機測驗,查看VIP跳轉到K2服務器上
[root@k2 ~]#ifconfig
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.36.100 netmask 255.255.255.255 broadcast 0.0.0.0
ether 00:0c:29:56:39:e8 txqueuelen 1000 (Ethernet)
ens33:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.36.200 netmask 255.255.255.255 broadcast 0.0.0.0
ether 00:0c:29:56:39:e8 txqueuelen 1000 (Ethernet)
VIP跳轉郵箱接收郵件
搭建WEB站點(兩個WEB站點執行相同操作)
# 編寫Nginx編譯安裝腳本
[root@WEB1 ~]#vim nginx.sh
#!/bin/bash
yum install -y vim lrzsz tree screen psmisc lsof tcpdump wget ntpdate gcc gcc-c++ glibc glibc-devel pcre pcre-devel openssl openssl-devel systemd-devel net-tools iotop bc zip unzip zlib-devel bash-completion nfs-utils automake libxml2 libxml2-devel libxslt libxslt-devel perl perl-ExtUtils-Embed &>/dev/null
wget https://nginx.org/download/nginx-1.14.2.tar.gz &>/dev/null
cd nginx-1.14.2/
./configure --prefix=/apps/nginx --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre --with-stream --with-stream_ssl_module --with-stream_realip_module &>/dev/null
make && make install
useradd nginx -s /sbin/nologin -u 2000
chown nginx.nginx -R /apps/nginx/
echo "\n=======================版本==================================\n"
/apps/nginx/sbin/nginx -V
# 給腳本添加執行權限
[root@WEB1 ~]#chmod a+x nginx.sh
# 啓動安裝腳本
[root@WEB1 ~]#./nginx.sh
# 成功安裝Nginx,做nginx命令軟鏈接
[root@WEB1 ~]#ln -sv /apps/nginx/sbin/nginx /usr/sbin/
‘/usr/sbin/nginx’ -> ‘/apps/nginx/sbin/nginx’
# 啓動Nginx
[root@WEB1 ~]#nginx
# 查看80端口
[root@WEB1 ~]#ss -ntl
# 修改Nginx配置文件,使其開啓php頁面訪問功能
[root@WEB1 ~]#vim /apps/nginx/conf/nginx.conf
2 user nginx nginx;
9 pid logs/nginx.pid;
39 charset utf-8;
42 location / {
43 root html;
44 index index.php index.html index.htm;
45 }
64 location ~ \.php$ {
65 root /apps/nginx/html;
66 fastcgi_pass 127.0.0.1:9000;
67 fastcgi_index index.php;
68 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
69 include fastcgi_params;
70 }
# 編寫php狀態頁面
[root@WEB1 ~]#vim /apps/nginx/html/index.php
<?php
phpinfo();
?>
# Nginx啓動前對配置文件進行檢查
[root@WEB1 ~]#nginx -t
# 重新加載Nginx配置文件
[root@WEB1 ~]#nginx -s reload
nginx: the configuration file /apps/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /apps/nginx/conf/nginx.conf test is successful
# 安裝php-fpm模塊
[root@WEB1 ~]#yum install -y php-fpm php-mysql
# 編寫php-fpm模塊配置文件
[root@WEB1 ~]#vim /etc/php-fpm.d/www.conf
12 listen = 127.0.0.1:9000
33 listen.mode = 0666
39 user = nginx
41 group = nginx
# 啓動php-fpm
[root@WEB1 ~]#systemctl restart php-fpm
# 9000端口查看
[root@WEB1 ~]#ss -ntl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 127.0.0.1:9000 *:*
兩個keepalived服務器中添加 web 的虛擬服務器
[root@k1 ~]#vim /etc/keepalived/keepalived.conf
....
virtual_server 192.168.36.100 80 { # 虛擬服務器的虛擬IP地址和服務的端口號
delay_loop 6 # 系統執行健康檢查的時間間隔
lb_algo wrr # lvs調度的算法:wrr輪詢算法
lb_kind DR # LVS的DR直接路由機制
protocol TCP # 指定轉發協議,TCP/UDP
real_server 192.168.36.112 80 { # 實際服務器IP地址和端口號
weight 1 # 權重值
TCP_CHECK { # 通過tcpcheck判斷RealServer的健康狀態
connect_port 80 # 檢測連接端口
connect_timeout 5 # 連接超時時間
nb_get_retry 3 # 重連次數
delay_before_retry 3 # 重連時間間隔
}
}
real_server 192.168.36.113 80 {
weight 1
TCP_CHECK {
connect_port 80
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
}
}
}
# 重新啓動keepalived服務
[root@k1 ~]#systemctl restart keepalived
# 查看生成的ipvsadm規則
[root@k1 ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.36.100:80 wrr
-> 192.168.36.112:80 Route 1 0 0
-> 192.168.36.113:80 Route 1 0 0
兩個WEB服務器創建lvs檢測腳本(步驟相同)
[root@WEB1 ~]#vim lvs_dr_rs.sh
#!/bin/bash
vip=192.168.36.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask #broadcast $vip up
#route add -host $vip dev $dev
echo "The RS Server is Ready!"
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "The RS Server is Canceled!"
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
# 添加執行權限並運行腳本
[root@WEB1 ~]#chmod a+x ./lvs_dr_rs.sh
[root@WEB1 ~]#./lvs_dr_rs.sh start
The RS Server is Ready!
# 生成檢測的虛擬IP
[root@WEB1 ~]#ifconfig lo:1
lo:1: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 192.168.36.100 netmask 255.255.255.255
loop txqueuelen 1000 (Local Loopback)
狀態頁面查看
VIP狀態頁
搭建數據庫主從複製服務器
Master服務器
# yum安裝mariadb服務
[root@Mariadb-M ~]#yum install -y mariadb-server
# 啓動數據庫服務
[root@Mariadb-M ~]#systemctl restart mariadb
# 修改mariadb配置文件
[root@Mariadb-M ~]#vim /etc/my.cnf
[mysqld]
server_id=1 # ID號
binlog_format=row # 基於行復制
log-bin=/data/bin/mysql-bin # 生成二進制文件的目錄與格式
# 創建二進制文件存放的目錄
[root@Mariadb-M ~]#mkdir /data/bin
# 授予目錄所屬關係
[root@Mariadb-M ~]#chown mysql.mysql /data/bin/ -R
# 重新啓動mariadb服務
[root@Mariadb-M ~]#systemctl restart mariadb
# 執行安全腳本
[root@Mariadb-M ~]#mysql_secure_installation
# 進入數據庫
[root@Mariadb-M ~]#mysql -uroot -p123456
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 19
Server version: 5.5.60-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> grant replication slave on *.* to repluser@'192.168.36.%' identified by 'centos'; # 添加slave複製權限
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges; # 刷新權限
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> reset master; # 重置master二進制文件大小
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> show master logs; # 查看並記錄
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 245 |
+------------------+-----------+
1 row in set (0.00 sec)
Slave服務器
# yum安裝mariadb服務
[root@Mariadb-M ~]#yum install -y mariadb-server
# 啓動數據庫服務
[root@Mariadb-M ~]#systemctl restart mariadb
# 修改mariadb配置文件
[root@Mariadb-M ~]#vim /etc/my.cnf
[mysqld]
server_id=2 # ID號
read_only # 只讀
# 重新啓動mariadb服務
[root@Mariadb-M ~]#systemctl restart mariadb
# 執行安全腳本
[root@Mariadb-M ~]#mysql_secure_installation
# 進入數據庫
[root@Mariadb-S ~]#mysql -uroot -p123456
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 15
Server version: 5.5.60-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> CHANGE MASTER TO # Slave節點添加同步Master數據庫的語句
-> MASTER_HOST='192.168.36.115',
-> MASTER_USER='repluser',
-> MASTER_PASSWORD='centos',
-> MASTER_PORT=3306,
-> MASTER_LOG_FILE='mysql-bin.000001',
-> MASTER_LOG_POS=245;
Query OK, 0 rows affected (0.01 sec)
MariaDB [(none)]> slave start; # 啓用從節點
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> show slave status\G; # 狀態查看
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.36.115
Master_User: repluser
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000001
Read_Master_Log_Pos: 245
Relay_Log_File: mariadb-relay-bin.000002
Relay_Log_Pos: 529
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: Yes # IO、SQL線程已經啓動,數據同步
Slave_SQL_Running: Yes
Master、Slave數據同步測試
MariaDB [(none)]> create database darius; # Master節點創建一個darius數據庫
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| darius |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)
MariaDB [(none)]> show databases; # Slave查看,如同步成功,則主從複製創建完成。
+--------------------+
| Database |
+--------------------+
| information_schema |
| darius |
| mysql |
| performance_schema |
+--------------------+
4 rows in set (0.00 sec)
兩個keepalived服務器添加mariad虛擬服務器
[root@k1 ~]#vim /etc/keepalived/keepalived.conf
....
virtual_server 192.168.36.200 3306 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
real_server 192.168.36.115 3306 {
weight 1
TCP_CHECK {
connect_port 3306
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.36.116 3306 {
weight 1
TCP_CHECK {
connect_port 3306
connect_timeout 5
nb_get_retry 3
delay_before_retry 3
}
}
}
....
# 生成ipvsadm規則
[root@k1 ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.36.100:80 wrr
-> 192.168.36.112:80 Route 1 0 0
-> 192.168.36.113:80 Route 1 0 0
TCP 192.168.36.200:3306 wrr
-> 192.168.36.115:3306 Route 1 0 0
-> 192.168.36.116:3306 Route 1 0 0
兩個mariadb服務器創建lvs檢測腳本(步驟相同)
[root@Mariadb-S ~]#vim lvs_dr_rs.sh
#!/bin/bash
vip=192.168.36.200
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask #broadcast $vip up
#route add -host $vip dev $dev
echo "The RS Server is Ready!"
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "The RS Server is Canceled!"
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
# 給腳本添加執行權限
[root@Mariadb-S ~]#chmod a+x lvs_dr_rs.sh
# 運行腳本
[root@Mariadb-S ~]#./lvs_dr_rs.sh start
The RS Server is Ready!
# 生成檢測的虛擬IP
[root@Mariadb-S ~]#ifconfig lo:1
lo:1: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 192.168.36.200 netmask 255.255.255.255
loop txqueuelen 1000 (Local Loopback)
搭建NFS服務器
# 修改NFS配置文件,設置將要共享的目錄
[root@NFS ~]#vim /etc/exports
/data *(rw,no_root_squash)
# 重新啓動NFS服務,並設置開機啓動
[root@NFS ~]#systemctl restart nfs
[root@NFS ~]#systemctl enable nfs
# 查看NFS共享目錄
[root@NFS ~]#exportfs -v
/data <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
# 解壓wordpress包
[root@NFS ~]#unzip wordpress-5.0-zh_CN.zip
# 將wordpress包內文件移動到共享目錄中,進行共享
[root@NFS ~]#mv wordpress/* /data/
[root@NFS ~]#cd /data/
# 生成wordpress配置文件
[root@NFS data]#mv wp-config-sample.php wp-config.php
# 修改wordpress配置文件
[root@NFS data]#vim wp-config.php
...
/** WordPress數據庫的名稱 */
define('DB_NAME', 'wordpress');
/** MySQL數據庫用戶名 */
define('DB_USER', 'wpuser');
/** MySQL數據庫密碼 */
define('DB_PASSWORD', 'centos');
/** MySQL主機 */
define('DB_HOST', '192.168.36.200');
/** 創建數據表時默認的文字編碼 */
define('DB_CHARSET', 'utf8');
...
將NFS共享的文件掛載到兩個WEB服務器中
# 寫入fstab文件中,開機自動掛載
[root@WEB1 ~]#vim /etc/fstab
192.168.36.114:/data /apps/nginx/html nfs _netdev,defaults 0 0
# 查看掛載情況
[root@WEB2 ~]#df -h
Filesystem Size Used Avail Use% Mounted on
192.168.36.114:/data 95G 3.9G 92G 5% /apps/nginx/html
訪問測試
多次宕機實驗依舊能訪問,起到web服務的高可用功能。