1. 安裝環境準備
1.1 主機環境準備
1.1.1. 關閉selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
setenforce 0
1.1.2. 軟件下載
apache-zookeeper-3.6.1-bin.tar.gz:下載地址
1.1.3. 部署規劃
軟件安裝路徑 /usr/local/zookeeper
端口規劃 2192
1.1.4. 系統主機時間、時區、系統語言
本節視實際情況需要操作
修改時區
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
修改系統語言環境
echo 'LANG="en_US.UTF-8"' >> /etc/profile && source /etc/profile
配置主機NTP時間同步
yum -y install ntp
systemctl enable ntpd && systemctl start ntpd
echo 'server ntp1.aliyun.com' >> /etc/ntp.conf
echo 'server ntp2.aliyun.com' >> /etc/ntp.conf
2. Zookeeper安裝部署
2.1 Zookeeper依賴安裝及部署
添加用戶與用戶組(用戶名請自行定義)
groupadd -r middleware && useradd -s /sbin/nologin -r -M -g middleware middleware
JDK安裝部署
tar -zxvf jdk-8u231-linux-x64.tar.gz -C /usr/local/
cat >>/etc/profile<<EOF
export JAVA_HOME=/usr/local/jdk1.8.0_231
export JRE_HOME=\${JAVA_HOME}/jre
export CLASSPATH=.:\${JAVA_HOME}/lib:\${JRE_HOME}/lib
export PATH=\${JAVA_HOME}/bin:\$PATH
EOF
source /etc/profile
java -version
下載apache-zookeeper-3.6.1-bin.tar.gz安裝包,並解壓安裝
yum -y install gcc gcc-c++ automake autoconf libevent-devel libevent make wget net-tools
cd /opt
wget https://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.6.1/apache-zookeeper-3.6.1-bin.tar.gz
tar -zxvf apache-zookeeper-3.6.1-bin.tar.gz -C /usr/local/
cd /usr/local/
mv apache-zookeeper-3.6.1-bin zookeeper
mkdir -p zookeeper/data/zookeeper
mkdir zookeeper/dataLog
cd zookeeper/conf
cp zoo_sample.cfg zoo.cfg
修改zookeeper數據存儲路徑與連接端口
vi zoo.cfg
dataDir=/usr/local/zookeeper/data/zookeeper
dataLogDir=/usr/local/zookeeper/dataLog
clientPort=2192
chown -R middleware:middleware /usr/local/zookeeper
配置Zookeeper環境變量
cat >>/etc/profile<< EOF
export PATH="\$PATH:/usr/local/zookeeper/bin"
EOF
source /etc/profile
2.2 配置zookeeper系統服務
2.2.1. 針對6系統添加系統服務
1、添加防火牆策略
(1)所有機器可訪問
iptables -A INPUT -p tcp --dport 2192 -j ACCEPT
service iptables save
(2)特定IP192.168.31.130可訪問本機2192端口
iptables -A INPUT -p tcp -s 192.168.31.130 --dport 2192 -j ACCEPT
service iptables save
2、添加zookeeper系統服務啓動腳本
cd /usr/local/zookeeper/bin/
sed -i '77aJAVA_HOME="/usr/local/jdk1.8.0_231"' zkEnv.sh
vi /etc/init.d/zookeeper
#!/bin/bash
#
# zookeeper start/stop the zookeeper daemon
#
# chkconfig: 345 80 20
# description: zookeeper is a message server.
#
ZOOKEEPER_HOME=/usr/local/zookeeper
PIDFILE=/usr/local/zookeeper/data/zookeeper/zookeeper_server.pid
case $1 in
start)
if [ -f $PIDFILE ]
then
echo "$PIDFILE exists, process is already running"
else
echo "Starting zookeeper server..."
sudo -u middleware $ZOOKEEPER_HOME/bin/zkServer.sh start
fi
;;
stop)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
sudo -u middleware $ZOOKEEPER_HOME/bin/zkServer.sh stop
fi
;;
status)
if [ ! -f $PIDFILE ]
then
echo "$PIDFILE does not exist, process is not running"
else
sudo -u middleware $ZOOKEEPER_HOME/bin/zkServer.sh status
echo "Zookeeper service is running..."
fi
;;
restart)
sudo -u middleware $ZOOKEEPER_HOME/bin/zkServer.sh restart
;;
*)
echo "Please use start|stop|status|restart as first argument"
;;
esac
3、配置zookeeper系統服務及自啓動
chmod +x /etc/init.d/zookeeper
chkconfig --add zookeeper && chkconfig zookeeper on
chkconfig --list zookeeper
4、啓動與停止zookeeper服務
service zookeeper start
ps -ef|grep zookeeper
service zookeeper stop
2.2.2. 針對7系統添加系統服務
1、添加防火牆策略
(1)所有機器可訪問
firewall-cmd --permanent --zone=public --add-port=2192/tcp
firewall-cmd --reload
(2)特定IP192.168.31.130可訪問本機2192端口
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.31.130" port protocol="tcp" port="2192" accept"
firewall-cmd --reload
(3)特定IP段192.168.142.0/24可訪問本機2192端口
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="192.168.142.0/24" port protocol="tcp" port="2192" accept"
firewall-cmd --reload
2、添加zookeeper系統服務啓動腳本
獲取當前服務器PATH路徑信息,並將此信息添加到zookeeper系統服務中
echo $PATH
/usr/local/jdk1.8.0_231/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
cat >/usr/lib/systemd/system/zookeeper.service<<EOF
[Unit]
Description=Zookeeper
After=network.target
[Service]
Type=forking
Environment=ZOO_LOG_DIR=/usr/local/zookeeper/logs
Environment=PATH=/usr/local/jdk1.8.0_231/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
PIDFile=/usr/local/zookeeper/data/zookeeper/zookeeper_server.pid
ExecStart=/usr/local/zookeeper/bin/zkServer.sh start
ExecStop=/usr/local/zookeeper/bin/zkServer.sh stop
ExecRestart=/usr/local/zookeeper/bin/zkServer.sh restart
User=middleware
Group=middleware
[Install]
WantedBy=multi-user.target
EOF
3、配置zookeeper系統服務及自啓動
systemctl daemon-reload
systemctl enable zookeeper.service
4、啓動與停止zookeeper服務
systemctl start zookeeper
ps -ef|grep zookeeper
systemctl stop zookeeper
3. Zookeeper加固
3.1 最小化權限用戶啓動
用戶名請自行定義
groupadd -r middleware && useradd -s /sbin/nologin -r -M -g middleware middleware
3.2 預防DOS***
限制zookeeper客戶端的最大連接數。
vi /usr/local/zookeeper/conf/zoo.cfg
maxClientCnxns=60
3.3 修改默認2181端口
默認情況下,zookeeper默認使用2181端口,請修改默認監聽端口,如本文檔使用的是2192
vi /usr/local/zookeeper/conf/zoo.cfg
clientPort=2192
3.4 禁用管理控制檯
如不需要使用zookeeper的管理控制檯,建議禁用(zookeeper的管理控制檯是由jetty啓動的,默認爲http,存在一定的信息泄露及安全隱患。)
操作指導:
在bin/zkServer.sh文件中,將如下
vi /usr/local/zookeeper/bin/zkServer.sh
start)
echo -n "Starting zookeeper ... "
if [ -f "$ZOOPIDFILE" ]; then
if kill -0 `cat "$ZOOPIDFILE"` > /dev/null 2>&1; then
echo $command already running as process `cat "$ZOOPIDFILE"`.
exit 1
fi
fi
nohup "$JAVA" $ZOO_DATADIR_AUTOCREATE "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" \
"-Dzookeeper.log.file=${ZOO_LOG_FILE}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
修改爲(即在nohup這一行,添加 "-Dzookeeper.admin.enableServer=false")
start)
echo -n "Starting zookeeper ... "
if [ -f "$ZOOPIDFILE" ]; then
if kill -0 `cat "$ZOOPIDFILE"` > /dev/null 2>&1; then
echo $command already running as process `cat "$ZOOPIDFILE"`.
exit 1
fi
fi
nohup "$JAVA" $ZOO_DATADIR_AUTOCREATE "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" \
"-Dzookeeper.log.file=${ZOO_LOG_FILE}" "-Dzookeeper.admin.enableServer=false" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \
3.5 日誌清理
建議設置對zookeeper日誌的定期清理功能,在配置文件中清理日誌策略,如下所示:
vi /usr/local/zookeeper/conf/zoo.cfg
autopurge.snapRetainCount=10
autopurge.purgeInterval=24
參數說明:
autopurge.snapRetainCount=10 //保留多少個快照
autopurge.purgeInterval=24 //多少小時清理一次
3.6 配置事務日誌與快照日誌分離
vi /usr/local/zookeeper/conf/zoo.cfg
dataDir=/usr/local/zookeeper/data/zookeeper
dataLogDir=/usr/local/zookeeper/dataLog
3.7 添加對zookeeper的指定IP授權訪問
zookeeper在默認情況下,是允許任意客戶端未經授權訪問,存在很大的安全隱患。具體連接指令如下:
/usr/local/zookeeper/bin/zkCli.sh -server 127.0.0.1:2192
WatchedEvent state:SyncConnected type:None path:null //敲回車
等待輸入操作指令,如創建用戶、授權等
[zk: 127.0.0.1:2192(CONNECTED) 0]
getAcl / 表示查看當前權限 quit 表示退出客戶端連接
[zk: 127.0.0.1:2192(CONNECTED) 3] getAcl /
'world,'anyone
: cdrwa
添加可訪問IP,一組可訪問ip間以符號,隔開,格式如下
[zk: 127.0.0.1:2192(CONNECTED) 3]
setAcl / ip:192.168.31.130:cdrwa,ip:127.0.0.1:cdrwa
查看權限是否添加成功
[zk: 127.0.0.1:2192(CONNECTED) 3] getAcl /
'ip,'192.168.31.130
: cdrwa
'ip,'127.0.0.1
: cdrwa
回退方法
[zk: 127.0.0.1:2192(CONNECTED) 3] setAcl / world:anyone:cdrwa
zookeeper身份的認證有4種方式:
(1)world:默認方式,相當於全世界都能訪問
(2)auth:代表已經認證通過的用戶(cli中可以通過addauth digest user:pwd 來添加當前上下文中的授權用戶)
(3)digest:即用戶名:密碼這種方式認證,這也是業務系統中最常用的,用username:password 字符串來產生一個MD5串,然後該串被用來作爲ACL ID,認證是通過明文發送username:password 來進行的,當用在ACL時,表達式爲username:base64 ,base64是password的SHA1摘要的編碼;
(4)ip:使用Ip地址認證
ID授權對象ID是指,權限賦予的用戶或者一個實體,例如:IP 地址或者機器,授權模式 授權對象有:
(1)IP:通常是一個IP地址或IP段,例如“192.168.29.100”或“192.168.29.100/110”
(2)Digest:自定義,通常是“username:BASE64(SHA-1(username:password))”,例如"foo:kWN6aNsbjcKWpqjiV7cg0N24raU="
(3)Word 只有一個ID:“anyone”
(4)Super:與Digest模式一致
zookeeper支持的權限有5種分別是(其中delete是指對子節點的刪除權限,其它4種權限指對自身節點的操作權限)
cdrwa:
create: 可以創建子節點;
read: 可以獲取節點數據以及當前節點的子節點列表;
write: 可以爲節點設置數據;
delete: 可以刪除子節點;
admin: 可以爲節點設置權限。
3.8 賬號與認證
1、通過zkCli.cmd 進入zookeeper客戶端
/usr/local/zookeeper/bin/zkCli.sh -server 127.0.0.1:2192
WatchedEvent state:SyncConnected type:None path:null //敲回車
2、使用auth方式加密,添加用戶名crm和密碼pwd
addauth digest crm:crm#pwd
3、授予/dubbo auth權限
setAcl /dubbo auth:crm:crm#pwd:rwadc
4、查看目錄加密後的權限
getAcl /dubbo
3.9 配置防火牆策略
根據操作系統的不同,參考2.2章節(注意如果是配置特定IP地址訪問時,也要添加3.7章節中添加的指定IP)
3.10 定期升級
使用官方最新穩定版本
4. Zookeeper優化
4.1 優化內核參數
cat >>/etc/sysctl.conf<<EOF
fs.file-max = 6815744
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_max_tw_buckets = 10000
net.core.somaxconn=4000
net.ipv4.tcp_syncookies = 1
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 262144
EOF
sysctl -p
4.2 系統資源限制
cat >>/etc/security/limits.conf<<EOF
* soft nofile 65525
* hard nofile 65525
* soft nproc 65525
* hard nproc 65525
EOF