【Java高級工程師蛻變之路】079 Zookeeper 的環境搭建

:::tip 文章更新歷史

2022/05/25 feat:初稿。

:::

Zookeeper 的環境搭建

Zookeeper 的安裝方式有三種：單機模式、集羣模式和僞集羣模式。

• 單機模式：Zookeeper 只運行在一臺服務器上，適合測試環境。
• 集羣模式：Zookeeper 運行於一個集羣上，適合生產環境，這個計算機集羣被稱爲一個“集合體”。
• 僞集羣模式：就是在一臺服務器上運行多個 Zookeeper 實例。

單機模式搭建

普通搭建

zookeeper 官網

https://zookeeper.apache.org/

下載 Zookeeper，最新穩定版是3.7.1

cd /app
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz

解壓

tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz

運行

cd apache-zookeeper-3.7.1-bin
mkdir data

cd conf
cp zoo_sample.cfg zoo.cfg
vim zoo.cfg

修改配置如下

# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/app/apache-zookeeper-3.7.1-bin/data
# the port at which the clients will connect
clientPort=2181

啓動

cd ..
cd bin
./zkServer.sh start

效果

➜  bin ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /app/apache-zookeeper-3.7.1-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

查看狀態

./zkServer.sh status

➜  bin ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /app/apache-zookeeper-3.7.1-bin/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone

docker 方式搭建

官方鏡像地址：

https://hub.docker.com/_/zookeeper

Dockerfile

FROM zookeeper:3.8

docker-compose.yml

version: '3'

services:
  zoo1:
    image: terwer/zookeeper
    container_name: portable-zookeeper
    build:
      context: .
      dockerfile: Dockerfile
    restart: always
    hostname: zoo1
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181

僞集羣模式搭建

僞集羣模式是在同一機器上運行多個 Zookeeper 實例來模擬多個 Zookeeper 集羣的情況，操作簡單，便於開發和測試階段調試。

僞集羣模式需要注意的問題：

1、要保證端口不能衝突。

2、dataDir 的路徑不能相同。

3、dataDir 需要創建 myid 文件夾來標識對應的 Zookeeper 服務器實例。

• clientPort 端口

  監聽的端口要獨立

• dataDir 和 dataLogDir

  數據文件和日誌文件要分別存儲

• Server.X 和 myid

Server.X 中的 X 對應 data/myid 中的 id 。

下載

參考普通模式

cd /media/terwer/data/app
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz

解壓，創建集羣目錄

mkdir zkcluster

tar -zxvf apache-zookeeper-3.7.1-bin.tar.gz -C /media/terwer/data/app/zkcluster

複製並改名

cd zkcluster
mv apache-zookeeper-3.7.1-bin zookeeper01

cp -r zookeeper01/ zookeeper02
cp -r zookeeper01/ zookeeper03

分別在 zookeeper01 、zookeeper02 、zookeeper03 目錄下創建 data 和 logs 目錄

mkdir data
mkdir logs

修改配置文件名稱

cd conf
cp zoo_sample.cfg zoo.cfg

修改每一個 Zookeeper 的 DataDir、DataLogDir 和 ClientPort，端口分別爲2181、2182、2183

dataDir=/media/terwer/data/app/zkcluster/zookeeper01/data
dataLogDir=/media/terwer/data/app/zkcluster/zookeeper01/logs
clientPort=2181

dataDir=/media/terwer/data/app/zkcluster/zookeeper02/data
dataLogDir=/media/terwer/data/app/zkcluster/zookeeper02/logs
clientPort=2182

dataDir=/media/terwer/data/app/zkcluster/zookeeper03/data
dataLogDir=/media/terwer/data/app/zkcluster/zookeeper03/logs
clientPort=2183

配置集羣

• 在每個集羣的 data 目錄下都新建一個 myid 文件，值分別爲1、2、3

  cd data
  touch myid
  vim myid
  

• 在每個 Zookeeper 的 zoo.cfg 中配置訪問端口和集羣列表

  server.1=192.168.3.46:2181:2181
  server.2=192.168.3.46:2182:2182
  server.3=192.168.3.46:2183:2183
  # server.服務器 ID=服務器 IP 地址:服務器之間通信端口:服務器之間投票選舉端口
  

啓動集羣

依次啓動三個集羣

cd bin
./zkServer.sh start
./zkServer.sh status

報錯解決

錯誤一

org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Client and election port must be different! Please update the configuration file on server.3
        at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.initializeWithAddressString(QuorumPeer.java:362)
        at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.<init>(QuorumPeer.java:279)
        at org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer.<init>(QuorumPeer.java:274)
        at org.apache.zookeeper.server.quorum.flexible.QuorumMaj.<init>(QuorumMaj.java:92)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.createQuorumVerifier(QuorumPeerConfig.java:658)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseDynamicConfig(QuorumPeerConfig.java:689)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.setupQuorumPeerConfig(QuorumPeerConfig.java:663)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:491)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:194)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:125)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:91)
Invalid config, exiting abnormally
2022-06-01 22:17:55,296 [myid:] - INFO  [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.
2022-06-01 22:17:55,298 [myid:] - ERROR [main:ServiceUtils@48] - Exiting JVM with code 2

客戶端端口和選舉端口不能一樣，換一下端口就行了。

server.1=192.168.3.46:2181:3181
server.2=192.168.3.46:2182:3182
server.3=192.168.3.46:2183:3183

錯誤二

QuorumCnxManager$Listener@7631Exception while listening java.net.BindException

解決方法：在三個 zoo.cfg 中加入

quorumListenOnAllIPs=true

錯誤三

Exception when following the leader java.io.EOFException

實際上，第一個問題沒解決，服務端端口也不能和客戶端一樣。

server.1=192.168.3.46:2888:3888
server.2=192.168.3.46:2889:3889
server.3=192.168.3.46:2890:3890

查看效果

啓動第一個之後

➜  bin ./zkServer.sh start 
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper01/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
➜  bin ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper01/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Error contacting service. It is probably not running.

啓動第二個之後

➜  bin ./zkServer.sh start 
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper02/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
➜  bin ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper02/bin/../conf/zoo.cfg
Client port found: 2182. Client address: localhost. Client SSL: false.
Mode: leader

啓動第三個節點之後

➜  bin ./zkServer.sh start 
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper03/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
➜  bin ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /media/terwer/data/app/zkcluster/zookeeper03/bin/../conf/zoo.cfg
Client port found: 2183. Client address: localhost. Client SSL: false.
Mode: follower

docker 版僞集羣模式搭建

docker-compose.yml

version: '3.1'

services:
  zoo1:
    image: zookeeper
    restart: "no"
    hostname: zoo1
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

  zoo2:
    image: zookeeper
    restart: "no"
    hostname: zoo2
    ports:
      - 2182:2181
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

  zoo3:
    image: zookeeper
    restart: "no"
    hostname: zoo3
    ports:
      - 2183:2181
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

檢測

先 Attach Shell ，然後

/apache-zookeeper-3.8.0-bin/bin
/zkServer.sh status

結果

root@zoo3:/apache-zookeeper-3.8.0-bin/bin# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader