apache分別基於mod_proxy_ajp, mod_proxy_http, mod_jk三種方案實現代理、負載均衡、會話綁定及Tomcat session cluster
1、nginx, haproxy, apache(mod_proxy_ajp, mod_proxy_http, mod_jk)反代用戶請求至tomcat;
2、nginx, haproxy, apache(mod_proxy_ajp, mod_proxy_http, mod_jk)負載用戶請求至tomcat servers;額外實現session sticky;
3、tomcat session cluster;
4、tomcat session server(msm);
規劃:
172.16.1.2 apache反代服務器
172.16.1.3 tomcat A
172.16.1.10 tomcat B
說明:
如題,可以選擇使用一個去反代用戶請求至tomcat,因爲apache和Tomcat都是Apache 軟件基金會的產品,兩者有更好的兼容性,所以選擇用apache,並且會以mod_proxy_ajp, mod_proxy_http, mod_jk三種方式分別實現題目要求。
1.方案一:mod_proxy_http
vim /etc/httpd/conf.d/lb_tomcat.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy balancer://tcsrvs>
BalancerMember http://172.16.1.10:8080/testapp route=TomcatA loadfactor=1
BalancerMember http://172.16.1.3:8080/testapp route=TomcatB loadfactor=2
ProxySet lbmethod=byrequests
ProxySet stickysession=ROUTEID
</Proxy>
<VirtualHost :80>
ServerName lb.zrs.com
ProxyVia On
ProxyRequests Off
<Proxy >
Require all granted
</Proxy>
ProxyPass / balancer://tcsrvs/
<Location />
Require all granted
</Location>
</VirtualHost>
分別爲兩個Tomcat的server.xml配置文件中的引擎中添加jvmRoute
1
2
3
4
5
第一個tomcat配置爲TomcatA:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatA">
第二個tomcat配置爲TomcatB:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatB">
分別爲兩個Tomcat Server的testapp提供測試頁面:
tomcatA:
mkdir -pv /usr/share/tomcat/webapps/testapp/{WEB-INF,classes,lib}
vim /usr/share/tomcat/webapps/testapp/index.jsp
添加如下內容:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<%@ page language="java" %>
<html>
<head><title>TomcatA</title></head>
<body>
<h1><font color="red">TomcatA.zrs.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("zrs.com","zrs.com"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
tomcatB:
mkdir -pv /usr/share/tomcat/webapps/testapp/{WEB-INF,classes,lib}
vim /usr/share/tomcat/webapps/testapp/index.jsp
添加如下內容:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<%@ page language="java" %>
<html>
<head><title>TomcatB</title></head>
<body>
<h1><font color="blue">TomcatB.zrs.com</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("zrs.com","zrs.com"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
客戶端測試,因爲做了session sticky,會一直調度到後端一臺服務器上,如我的一直是TomcatA
A.PNG
關掉TomcatA,便會調度到TomcatB上。
B.PNG
2.方案二:mod_proxy_ajp
使用mod_proxy_ajp,僅需要把mod_proxy_http的http改爲ajp,端口8080改爲8009即可,需要注意的是前端用了ajp,可以把後端的tomcat的8080端口註釋掉,不讓其監聽,從而使安全性更高。
vim /etc/httpd/conf.d/lb_tomcat.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy balancer://tcsrvs>
BalancerMember ajp://172.16.1.10:8009/testapp route=TomcatA loadfactor=1
BalancerMember ajp://172.16.1.3:8009/testapp route=TomcatB loadfactor=2
ProxySet lbmethod=byrequests
ProxySet stickysession=ROUTEID
</Proxy>
<VirtualHost :80>
ServerName lb.qhdlink.com
ProxyVia On
ProxyRequests Off
<Proxy >
Require all granted
</Proxy>
ProxyPass / balancer://tcsrvs/
<Location />
Require all granted
</Location>
</VirtualHost>
關閉session sticky,客戶端查看
ajp A.PNG
ajp B.PNG
3.方案三:mod_jk
此方案需要mod_jk模塊
需要安裝這個文件tomcat-connectors-VERSION.tar.gz
配置文件:
vim /etc/httpd/conf.d/mod_jk_lb.conf
1
2
3
4
5
6
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* tcsrvs
JkMount /jk-status Stat
vim /etc/httpd/conf.d/workers.properties
1
2
3
4
5
6
7
8
9
10
11
12
13
worker.list=tcsrvs,Stat
worker.TomcatA.host=172.16.1.10
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.16.1.3
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.tcsrvs.type=lb
worker.tcsrvs.balance_workers=TomcatA,TomcatB
worker.tcsrvs.sticky_session=1
worker.Stat.type=status
4.tomcat session cluster
在兩個tomcat的server.xml配置啓用集羣
添加如下配置內容,並且註釋掉8080端口
需要注意的是下面的內容中,將address="auto"的auto改爲該主機的IP地址
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.100.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
並且在Host中添加一個Context組件
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<Context path="/test" docBase="/myapps/webapps" reloadable="true"/>
創建目錄
[root@zrs tomcat]# mkdir -pv /myapps/webapps/{classes,lib,WEB-INF}
創建測試頁面
跟上面的一樣,創建AB兩個不同的頁面
[root@zrs tomcat]# vim /myapps/webapps/index.jsp
複製web.xml到指定目錄
[root@zrs tomcat]# cp web.xml /myapps/webapps/WEB-INF/
編輯WEB-INF/web.xml
在任意非段落外添加<distributable/>這個是在 <web 裏面
[root@zrs tomcat]# vim /myapps/webapps/WEB-INF/web.xml
反代服務器配置
[root@director conf.d]# vim mod_jk_proxy.conf
1
2
3
4
5
6
LoadModule jk_module modules/mod_jk.so
JkWorkersFile /etc/httpd/conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel debug
JkMount /* tcsrvs
JkMount /jk-status Stat
[root@director conf.d]# vim workers.properties
1
2
3
4
5
6
7
8
9
10
11
12
13
worker.list=tcsrvs,Stat
worker.TomcatA.host=172.16.1.10
worker.TomcatA.port=8009
worker.TomcatA.type=ajp13
worker.TomcatA.lbfactor=1
worker.TomcatB.host=172.16.1.3
worker.TomcatB.port=8009
worker.TomcatB.type=ajp13
worker.TomcatB.lbfactor=2
worker.tcsrvs.type=lb
worker.tcsrvs.balance_workers=TomcatA,TomcatB
worker.tcsrvs.sticky_session=0
worker.Stat.type=status
分別開啓三臺主機的httpd服務和tomcat服務
客戶端測試
可以看到反代服務器以ABB的次序輪詢後端兩臺tomcat服務器,但是ID不變,從而實現session cluster。
ID不變A.PNG
ID 不變 B.PNG
5.tomcat session server(msm)
接着剛纔的配置環境,這個構建方案的原理是將所有會話放在同一臺後端tomcat服務器上,當後端這臺tomcat服務器宕機時,整個搭建便無法正常提供服務,所以也要給tomcat服務器做高可用。
這個構建方案需要memcached作爲旁掛式緩存服務器,所以需要提前下載並啓動memcached服務。
停止兩臺tomcat服務,還原server.xml配置文件
註釋掉8080端口,在Engine中添加jvmRoute,如下
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatA" >
或
<Engine name="Catalina" defaultHost="localhost" jvmRoute="TomcatB" >
在Host段添加配置內容
<Context path="/test" docBase="/myapps/webapps" reloadable="true">
<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes="n1:172.16.1.10:11211,n2:172.16.1.3:11211"
failoverNodes="n1"
requestUriIgnorePattern=".*.(ico|png|gif|jpg|css|js|html|htm)$"
transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
/>
</Context>
導入類文件
memcached-session-manager項目地址,http://code.google.com/p/memcached-session-manager/
下載如下jar文件至各tomcat節點的tomcat安裝目錄下的lib目錄中,其中的${version}要換成你所需要的版本號,tc${6,7,8}要換成與tomcat版本相同的版本號。
memcached-session-manager-${version}.jar
memcached-session-manager-tc${6,7,8}-${version}.jar
spymemcached-${version}.jar
msm-javolution-serializer-${version}.jar
javolution-${version}.jar
將所有的類放在/usr/share/java/tomcat目錄中。
啓動tomcat服務即可看到反代服務器以AB的次序輪詢後端兩臺tomcat服務器,但是session ID不變。