準備工作
準備三臺虛擬機, 均爲CentOS-7-x86_64最小化安裝, iptables與SELinux均處於關閉狀態, 配置好yum源(base和epel). 做好快照, 以便每次實驗後快速恢復.
HostA OS: CentOS-7-x86_64 hostname: 80e54d87.twoyang.com eno16777736: 172.18.71.101/16 gateway: 172.18.0.1 HostB OS: CentOS-7-x86_64 hostname: b9cf468b.twoyang.com eno16777736: 172.18.71.102/16 gateway: 172.18.0.1 HostC OS: CentOS-7-x86_64 hostname: 1f5fafa6.twoyang.com eno16777736: 172.18.71.103/16 gateway: 172.18.0.1
先不管Memcached和MSM, 按照nginx負載均衡tomcat部署好集羣.
集羣初始化
建立主機信任, 時間同步, 統一增加yum源. 這一步其實在哪個節點上做都可以, 我這裏選擇HostA.
# 寫了個腳本來做這個事情
[root@80e54d87 ~]# wget https://raw.githubusercontent.com/wqiang0917/LearnInMagedu/master/shell/sshtrust.sh
# 將集羣中所有節點IP地址(包括本機)都加入這個主機列表數組
[root@80e54d87 ~]# vim sshtrust.sh
HOSTS=("172.18.71.101" "172.18.71.102" "172.18.71.103")
# 分發密鑰, 建立主機信任.
[root@80e54d87 ~]# bash sshtrust.sh --key
# 同步時間
[root@80e54d87 ~]# bash sshtrust.sh "ntpdate 172.18.0.1"
[root@80e54d87 ~]# bash sshtrust.sh "hwclock --systohc"
[root@80e54d87 ~]# bash sshtrust.sh date
Tue Jun 7 21:54:22 CST 2016
Tue Jun 7 21:54:22 CST 2016
Tue Jun 7 21:54:22 CST 2016HostA
# 安裝`OpenJDK`和`Tomcat`
[root@80e54d87 ~]# yum install -y java-1.7.0-openjdk java-1.7.0-openjdk-devel tomcat tomcat-lib tomcat-admin-webapps tomcat-webapps
[root@80e54d87 ~]# echo "export JAVA_HOME=/usr" > /etc/profile.d/java.sh
[root@80e54d87 ~]# exec bash
# 創建sessapp應用, 並提供session測試主頁.
[root@80e54d87 ~]# mkdir -p /var/lib/tomcat/webapps/sessapp/{classes,lib,WEB-INF,META-INF}
[root@80e54d87 ~]# cat << EOF > /var/lib/tomcat/webapps/sessapp/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatA</title></head>
<body>
<h1><font color="red">TomcatA.magedu.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("magedu.com","magedu.com"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
EOF
# 啓動服務, 並檢查監聽端口.
[root@80e54d87 ~]# systemctl start tomcat.service
[root@80e54d87 ~]# ss -tnl | grep "\(4000\|8009\|8080\)"
LISTEN 0 100 :::8009 :::*
LISTEN 0 100 :::8080 :::*
LISTEN 0 50 ::ffff:172.18.71.101:4000 :::*HostB
# 安裝`OpenJDK`和`Tomcat`
[root@b9cf468b ~]# yum install -y java-1.7.0-openjdk java-1.7.0-openjdk-devel tomcat tomcat-lib tomcat-admin-webapps tomcat-webapps
[root@b9cf468b ~]# echo "export JAVA_HOME=/usr" > /etc/profile.d/java.sh
[root@b9cf468b ~]# exec bash
[root@b9cf468b ~]# mkdir -p /var/lib/tomcat/webapps/sessapp/{classes,lib,WEB-INF,META-INF}
# 創建sessapp應用, 並提供session測試主頁.
[root@b9cf468b ~]# cat << EOF > /var/lib/tomcat/webapps/sessapp/index.jsp
<%@ page language="java" %>
<html>
<head><title>TomcatB</title></head>
<body>
<h1><font color="blue">TomcatB.magedu.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("magedu.com","magedu.com"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
EOF
# 啓動服務, 並檢查監聽端口.
[root@b9cf468b ~]# systemctl start tomcat.service
[root@b9cf468b ~]# ss -tnl | grep "\(4000\|8009\|8080\)"
LISTEN 0 100 :::8009 :::*
LISTEN 0 100 :::8080 :::*
LISTEN 0 50 ::ffff:172.18.71.101:4000 :::*HostC
# 首先測試使用curl直接訪問HostA與HostB的sessapp.
[root@1f5fafa6 ~]# curl http://172.18.71.101:8080/sessapp/
<html>
<head><title>TomcatA</title></head>
<body>
<h1><font color="red">TomcatA.magedu.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<td>4D5C16AFE6FDA767E506729B0781A0B7</td>
</tr>
<tr>
<td>Created on</td>
<td>1465284908776</td>
</tr>
</table>
</body>
</html>
[root@1f5fafa6 ~]# curl http://172.18.71.102:8080/sessapp/
<html>
<head><title>TomcatB</title></head>
<body>
<h1><font color="blue">TomcatB.magedu.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<td>185E2BDA227CB166CA75B7FDC1A07BE4</td>
</tr>
<tr>
<td>Created on</td>
<td>1465284924384</td>
</tr>
</table>
</body>
</html>
# 安裝nginx作爲前端調度器反代.
[root@1f5fafa6 ~]# yum install -y nginx
[root@1f5fafa6 ~]# vim /etc/nginx/nginx.conf
...
upstream tcsrvs {
server 172.18.71.101:8080 weight=1;
server 172.18.71.102:8080 weight=2;
}
server {
...
location / {
proxy_pass http://tcsrvs;
}
}
...
# 測試語法
[root@1f5fafa6 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# 啓動服務
[root@1f5fafa6 ~]# systemctl start nginx.service
[root@1f5fafa6 ~]# ss -tnl | grep 80
LISTEN 0 128 *:80 *:*
LISTEN 0 128 :::80 :::*使用瀏覽器訪問http://172.18.71.103/sessapp/, 持續刷新頁面. 可以看到通過前端調度器nginx在按照權重輪調後端服務器, 並且SessionID一直在變化. 此時使用Memcached和MSM存儲會話.
注意: 不要使用curl來測試, curl不是daemon進程, 執行完就退出了, 沒有辦法保存會話.
Memcached和MSM
在HostA和HostB兩個後端tomcat主機上均要執行以下操作, 以HostA爲例.
# 安裝memcached [root@80e54d87 ~]# yum install -y memcached [root@80e54d87 ~]# systemctl start memcached.service [root@80e54d87 ~]# ss -tnl | grep 11211 LISTEN 0 128 *:11211 *:* LISTEN 0 128 :::11211 :::* # 下載MSM的類庫放置在tomcat的公共類庫目錄中 [root@80e54d87 ~]# ls msm/ javolution-5.4.3.1.jar memcached-session-manager-tc7-1.8.3.jar spymemcached-2.11.1.jar memcached-session-manager-1.8.3.jar msm-javolution-serializer-1.8.3.jar [root@80e54d87 ~]# cp msm/*.jar /usr/share/java/tomcat/ # 增加配置 [root@80e54d87 ~]# vim /etc/tomcat/server.xml ... <Host ...> <Context path="/sessapp" docBase="/var/lib/tomcat/webapps/sessapp" reloadable="true"> <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:172.18.71.101:11211,n2:172.18.71.102:11211" # 使用non-sticky sessions模式, 請參照參考資料中關於非粘性session的解釋. sticky="false" # session備份使用同步模式 sessionBackupAsync="false" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory" /> </Context> </Host> ... # 重啓服務 [root@80e54d87 ~]# systemctl restart tomcat.service
測試驗證
使用瀏覽器訪問http://172.18.71.103/sessapp/, 持續刷新頁面.
![tomcat_msm]()
可以觀察到調度器一直在按照權重輪調後端服務器, 但是SessionID一直沒有變化. 此時使用的是標識爲n2的memcached作爲主session server. 說明session可以全局共享.
注意: 不要使用curl來測試, curl不是daemon進程, 執行完就退出了, 沒有辦法保存會話.
關閉當前作爲主session server的HostB上的memcached, 再來刷新頁面.
[root@b9cf468b ~]# systemctl stop memcached.service
可以觀察到SessionID沒有變化, 而session server已經切換到了是標識爲n1的memcached. 說明單個session server發生故障, 會話也不會丟失.