ZooKeeper-3.3.4集羣安裝配置

原創 千与 2018-08-30 20:13

ZooKeeper是一個分佈式開源框架，提供了協調分佈式應用的基本服務，它向外部應用暴露一組通用服務——分佈式同步（Distributed Synchronization）、命名服務（Naming Service）、集羣維護（Group Maintenance）等，簡化分佈式應用協調及其管理的難度，提供高性能的分佈式服務。ZooKeeper本身可以以Standalone模式安裝運行，不過它的長處在於通過分佈式ZooKeeper集羣（一個Leader，多個Follower），基於一定的策略來保證ZooKeeper集羣的穩定性和可用性，從而實現分佈式應用的可靠性。

有關ZooKeeper的介紹，網上很多，也可以參考文章後面，我整理的一些相關鏈接。

下面，我們簡單說明一下ZooKeeper的配置。


ZooKeeper Standalone模式


從Apache網站上（zookeeper.apache.org）下載ZooKeeper軟件包，我選擇了3.3.4版本的（zookeeper-3.3.4.tar.gz），在一臺Linux機器上安裝非常容易，只需要解壓縮後，簡單配置一下即可以啓動ZooKeeper服務器進程。

將zookeeper-3.3.4/conf目錄下面的 zoo_sample.cfg修改爲zoo.cfg，配置文件內容如下所示：

tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
上面各個配置參數的含義也非常簡單，引用如下所示： 

tickTime —— the basic time unit in milliseconds used by ZooKeeper. It is used to do heartbeats and the minimum session timeout will be twice the tickTime.
dataDir —— the location to store the in-memory database snapshots and, unless specified otherwise, the transaction log of updates to the database.
clientPort —— the port to listen for client connections
下面啓動ZooKeeper服務器進程： 

cd zookeeper-3.3.4/
bin/zkServer.sh start
通過jps命令可以查看ZooKeeper服務器進程，名稱爲QuorumPeerMain。

在客戶端連接ZooKeeper服務器，執行如下命令：

bin/zkCli.sh -server dynamic:2181
上面dynamic是我的主機名，如果在本機執行，則執行如下命令即可：
bin/zkCli.sh
客戶端連接信息如下所示： 

hadoop@master:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server dynamic:2181
Connecting to dynamic:2181
2012-01-08 21:30:06,178 - INFO  [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 21:30:06,188 - INFO  [main:Environment@97] - Client environment:host.name=master
2012-01-08 21:30:06,191 - INFO  [main:Environment@97] - Client environment:java.version=1.6.0_30
2012-01-08 21:30:06,194 - INFO  [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2012-01-08 21:30:06,200 - INFO  [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 21:30:06,203 - INFO  [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 21:30:06,206 - INFO  [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 21:30:06,213 - INFO  [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2012-01-08 21:30:06,216 - INFO  [main:Environment@97] - Client environment:java.compiler=<NA>
2012-01-08 21:30:06,235 - INFO  [main:Environment@97] - Client environment:os.name=Linux
2012-01-08 21:30:06,244 - INFO  [main:Environment@97] - Client environment:os.arch=i386
2012-01-08 21:30:06,246 - INFO  [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2012-01-08 21:30:06,251 - INFO  [main:Environment@97] - Client environment:user.name=hadoop
2012-01-08 21:30:06,254 - INFO  [main:Environment@97] - Client environment:user.home=/home/hadoop
2012-01-08 21:30:06,255 - INFO  [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 21:30:06,264 - INFO  [main:ZooKeeper@379] - Initiating client connection, connectString=dynamic:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2012-01-08 21:30:06,339 - INFO  [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server dynamic/192.168.0.107:2181
Welcome to ZooKeeper!
2012-01-08 21:30:06,397 - INFO  [main-SendThread(dynamic:2181):ClientCnxn$SendThread@950] - Socket connection established to dynamic/192.168.0.107:2181, initiating session
JLine support is enabled
2012-01-08 21:30:06,492 - INFO  [main-SendThread(dynamic:2181):ClientCnxn$SendThread@739] - Session establishment complete on server dynamic/192.168.0.107:2181, sessionid = 0x134b9b714f9000c, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: dynamic:2181(CONNECTED) 0]
接着，可以使用help查看Zookeeper客戶端可以使用的基本操作命令。

ZooKeeper Distributed模式


ZooKeeper分佈式模式安裝（ZooKeeper集羣）也比較容易，這裏說明一下基本要點。

首先要明確的是，ZooKeeper集羣是一個獨立的分佈式協調服務集羣，“獨立”的含義就是說，如果想使用ZooKeeper實現分佈式應用的協調與管理，簡化協調與管理，任何分佈式應用都可以使用，這就要歸功於Zookeeper的數據模型（Data Model）和層次命名空間（Hierarchical Namespace）結構，詳細可以參考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html。在設計你的分佈式應用協調服務時，首要的就是考慮如何組織層次命名空間。

下面說明分佈式模式的安裝配置，過程如下所示：

第一步：主機名稱到IP地址映射配置

ZooKeeper集羣中具有兩個關鍵的角色：Leader和Follower。集羣中所有的結點作爲一個整體對分佈式應用提供服務，集羣中每個結點之間都互相連接，所以，在配置的ZooKeeper集羣的時候，每一個結點的host到IP地址的映射都要配置上集羣中其它結點的映射信息。

例如，我的ZooKeeper集羣中每個結點的配置，以slave-01爲例，/etc/hosts內容如下所示：

192.168.0.179   slave-01
192.168.0.178   slave-02
192.168.0.177   slave-03

ZooKeeper採用一種稱爲Leader election的選舉算法。在整個集羣運行過程中，只有一個Leader，其他的都是Follower，如果ZooKeeper集羣在運行過程中Leader出了問題，系統會採用該算法重新選出一個Leader。因此，各個結點之間要能夠保證互相連接，必須配置上述映射。

ZooKeeper集羣啓動的時候，會首先選出一個Leader，在Leader election過程中，某一個滿足選舉算的結點就能成爲Leader。整個集羣的架構可以參考http://zookeeper.apache.org/doc/trunk/zookeeperOver.html#sc_designGoals。

第二步：修改ZooKeeper配置文件

在其中一臺機器（slave-01）上，解壓縮zookeeper-3.3.4.tar.gz，修改配置文件conf/zoo.cfg，內容如下所示：

tickTime=2000
dataDir=/home/hadoop/storage/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=slave-01:2888:3888
server.2=slave-02:2888:3888
server.3=slave-03:2888:3888
上述配置內容說明，可以參考http://zookeeper.apache.org/doc/trunk/zookeeperStarted.html#sc_RunningReplicatedZooKeeper。

第三步：遠程複製分發安裝文件

上面已經在一臺機器slave-01上配置完成ZooKeeper，現在可以將該配置好的安裝文件遠程拷貝到集羣中的各個結點對應的目錄下：

cd /home/hadoop/installation/
scp -r zookeeper-3.3.4/ hadoop@slave-02:/home/hadoop/installation/
scp -r zookeeper-3.3.4/ hadoop@slave-03:/home/hadoop/installation/
第四步：設置myid

在我們配置的dataDir指定的目錄下面，創建一個myid文件，裏面內容爲一個數字，用來標識當前主機，conf/zoo.cfg文件中配置的server.X中X爲什麼數字，則myid文件中就輸入這個數字，例如：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ echo "1" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-02:~/installation/zookeeper-3.3.4$ echo "2" > /home/hadoop/storage/zookeeper/myid
hadoop@slave-03:~/installation/zookeeper-3.3.4$ echo "3" > /home/hadoop/storage/zookeeper/myid
按照上述進行配置即可。

第五步：啓動ZooKeeper集羣

在ZooKeeper集羣的每個結點上，執行啓動ZooKeeper服務的腳本，如下所示：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-02:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkServer.sh start
以結點slave-01爲例，日誌如下所示：
hadoop@slave-01:~/installation/zookeeper-3.3.4$ tail -500f zookeeper.out 
2012-01-08 06:51:19,117 - INFO  [main:QuorumPeerConfig@90] - Reading configuration from: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
2012-01-08 06:51:19,133 - INFO  [main:QuorumPeerConfig@310] - Defaulting to majority quorums
2012-01-08 06:51:19,167 - INFO  [main:QuorumPeerMain@119] - Starting quorum peer
2012-01-08 06:51:19,227 - INFO  [main:NIOServerCnxn$Factory@143] - binding to port 0.0.0.0/0.0.0.0:2181
2012-01-08 06:51:19,277 - INFO  [main:QuorumPeer@819] - tickTime set to 2000
2012-01-08 06:51:19,278 - INFO  [main:QuorumPeer@830] - minSessionTimeout set to -1
2012-01-08 06:51:19,279 - INFO  [main:QuorumPeer@841] - maxSessionTimeout set to -1
2012-01-08 06:51:19,281 - INFO  [main:QuorumPeer@856] - initLimit set to 5
2012-01-08 06:51:19,347 - INFO  [Thread-1:QuorumCnxManager$Listener@473] - My election bind port: 3888
2012-01-08 06:51:19,393 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@621] - LOOKING
2012-01-08 06:51:19,396 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@663] - New election. My id =  1, Proposed zxid = 0
2012-01-08 06:51:19,400 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 1 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2012-01-08 06:51:19,416 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:19,420 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:19,612 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:19,615 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:19,616 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 400
2012-01-08 06:51:20,019 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,021 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,022 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 800
2012-01-08 06:51:20,825 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,827 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:20,828 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 1600
2012-01-08 06:51:22,435 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/192.168.0.178:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:22,439 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:51:22,441 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 3200
2012-01-08 06:51:22,945 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 2 (n.sid), LOOKING (my state)
2012-01-08 06:51:22,946 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@721] - Updating proposal
2012-01-08 06:51:22,949 - WARN  [WorkerSender Thread:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/192.168.0.177:3888
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:340)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:360)
        at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:333)
        at java.lang.Thread.run(Thread.java:662)
2012-01-08 06:51:22,951 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 2 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 1 (n.sid), LOOKING (my state)
2012-01-08 06:51:23,156 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumPeer@643] - FOLLOWING
2012-01-08 06:51:23,170 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@80] - TCP NoDelay set to: true
2012-01-08 06:51:23,206 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:host.name=slave-01
2012-01-08 06:51:23,207 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.version=1.6.0_30
2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.vendor=Sun Microsystems Inc.
2012-01-08 06:51:23,208 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 06:51:23,209 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 06:51:23,210 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.io.tmpdir=/tmp
2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:java.compiler=<NA>
2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.name=Linux
2012-01-08 06:51:23,212 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.arch=i386
2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:os.version=3.0.0-14-generic
2012-01-08 06:51:23,213 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.name=hadoop
2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.home=/home/hadoop
2012-01-08 06:51:23,214 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Environment@97] - Server environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 06:51:23,223 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:ZooKeeperServer@151] - Created server with tickTime 2000 minSessionTimeout 4000 maxSessionTimeout 40000 datadir /home/hadoop/storage/zookeeper/version-2 snapdir /home/hadoop/storage/zookeeper/version-2
2012-01-08 06:51:23,339 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@294] - Getting a snapshot from leader
2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:Learner@325] - Setting leader epoch 1
2012-01-08 06:51:23,358 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FileTxnSnapLog@254] - Snapshotting: 0
2012-01-08 06:51:25,511 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 1 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
2012-01-08 06:51:42,584 - INFO  [WorkerReceiver Thread:FastLeaderElection@496] - Notification: 3 (n.leader), 0 (n.zxid), 2 (n.round), LOOKING (n.state), 3 (n.sid), FOLLOWING (my state)
我啓動的順序是slave-01>slave-02>slave-03，由於ZooKeeper集羣啓動的時候，每個結點都試圖去連接集羣中的其它結點，先啓動的肯定連不上後面還沒啓動的，所以上面日誌前面部分的異常是可以忽略的。通過後面部分可以看到，集羣在選出一個Leader後，最後穩定了。

其他結點可能也出現類似問題，屬於正常。

第六步：安裝驗證

可以通過ZooKeeper的腳本來查看啓動狀態，包括集羣中各個結點的角色（或是Leader，或是Follower），如下所示，是在ZooKeeper集羣中的每個結點上查詢的結果：

hadoop@slave-01:~/installation/zookeeper-3.3.4$ bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower

hadoop@slave-02:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: leader

hadoop@slave-03:~/installation/zookeeper-3.3.4$  bin/zkServer.sh status
JMX enabled by default
Using config: /home/hadoop/installation/zookeeper-3.3.4/bin/../conf/zoo.cfg
Mode: follower
通過上面狀態查詢結果可見，slave-02是集羣的Leader，其餘的兩個結點是Follower。

另外，可以通過客戶端腳本，連接到ZooKeeper集羣上。對於客戶端來說，ZooKeeper是一個整體（ensemble），連接到ZooKeeper集羣實際上感覺在獨享整個集羣的服務，所以，你可以在任何一個結點上建立到服務集羣的連接，例如：

hadoop@slave-03:~/installation/zookeeper-3.3.4$ bin/zkCli.sh -server slave-01:2181
Connecting to slave-01:2181
2012-01-08 07:14:21,068 - INFO  [main:Environment@97] - Client environment:zookeeper.version=3.3.3-1203054, built on 11/17/2011 05:47 GMT
2012-01-08 07:14:21,080 - INFO  [main:Environment@97] - Client environment:host.name=slave-03
2012-01-08 07:14:21,085 - INFO  [main:Environment@97] - Client environment:java.version=1.6.0_30
2012-01-08 07:14:21,089 - INFO  [main:Environment@97] - Client environment:java.vendor=Sun Microsystems Inc.
2012-01-08 07:14:21,095 - INFO  [main:Environment@97] - Client environment:java.home=/home/hadoop/installation/jdk1.6.0_30/jre
2012-01-08 07:14:21,104 - INFO  [main:Environment@97] - Client environment:java.class.path=/home/hadoop/installation/zookeeper-3.3.4/bin/../build/classes:/home/hadoop/installation/zookeeper-3.3.4/bin/../build/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../zookeeper-3.3.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/log4j-1.2.15.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/jline-0.9.94.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-lang-2.4.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-collections-3.2.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/commons-cli-1.1.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-tasks-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../lib/apache-rat-core-0.6.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../src/java/lib/*.jar:/home/hadoop/installation/zookeeper-3.3.4/bin/../conf:/home/hadoop/installation/jdk1.6.0_30/lib/*.jar:/home/hadoop/installation/jdk1.6.0_30/jre/lib/*.jar
2012-01-08 07:14:21,111 - INFO  [main:Environment@97] - Client environment:java.library.path=/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
2012-01-08 07:14:21,116 - INFO  [main:Environment@97] - Client environment:java.io.tmpdir=/tmp
2012-01-08 07:14:21,124 - INFO  [main:Environment@97] - Client environment:java.compiler=<NA>
2012-01-08 07:14:21,169 - INFO  [main:Environment@97] - Client environment:os.name=Linux
2012-01-08 07:14:21,175 - INFO  [main:Environment@97] - Client environment:os.arch=i386
2012-01-08 07:14:21,177 - INFO  [main:Environment@97] - Client environment:os.version=3.0.0-14-generic
2012-01-08 07:14:21,185 - INFO  [main:Environment@97] - Client environment:user.name=hadoop
2012-01-08 07:14:21,188 - INFO  [main:Environment@97] - Client environment:user.home=/home/hadoop
2012-01-08 07:14:21,190 - INFO  [main:Environment@97] - Client environment:user.dir=/home/hadoop/installation/zookeeper-3.3.4
2012-01-08 07:14:21,197 - INFO  [main:ZooKeeper@379] - Initiating client connection, connectString=slave-01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@bf32c
2012-01-08 07:14:21,305 - INFO  [main-SendThread():ClientCnxn$SendThread@1061] - Opening socket connection to server slave-01/192.168.0.179:2181
Welcome to ZooKeeper!
2012-01-08 07:14:21,376 - INFO  [main-SendThread(slave-01:2181):ClientCnxn$SendThread@950] - Socket connection established to slave-01/192.168.0.179:2181, initiating session
JLine support is enabled
[zk: slave-01:2181(CONNECTING) 0] 2012-01-08 07:14:21,872 - INFO  [main-SendThread(slave-01:2181):ClientCnxn$SendThread@739] - Session establishment complete on server slave-01/192.168.0.179:2181, sessionid = 0x134bdcd6b730000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null

[zk: slave-01:2181(CONNECTED) 0] ls /
[zookeeper]
當前根路徑爲/zookeeper。

總結說明


主機名與IP地址映射配置問題

啓動ZooKeeper集羣時，如果ZooKeeper集羣中slave-01結點的日誌出現如下錯誤：

java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:37:46,026 - INFO  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:FastLeaderElection@697] - Notification time out: 6400
2012-01-08 06:37:57,431 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 2 at election address slave-02/202.106.199.35:3888
java.net.SocketTimeoutException
        at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:371)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:404)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:688)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:622)
2012-01-08 06:38:02,442 - WARN  [QuorumPeer:/0:0:0:0:0:0:0:0:2181:QuorumCnxManager@384] - Cannot open channel to 3 at election address slave-03/202.106.199.35:3888

很顯然，slave-01在啓動時連接集羣中其他結點（slave-02、slave-03）時，主機名映射的IP與我們實際配置的不一致，所以集羣中各個結點之間無法建立鏈路，整個ZooKeeper集羣啓動是失敗的。

上面錯誤日誌中slave-02/202.106.199.35:3888實際應該是slave-02/202.192.168.0.178:3888就對了，但是在進行域名解析的時候映射有問題，修改每個結點的/etc/hosts文件，將ZooKeeper集羣中所有結點主機名到IP地址的映射配置上。


參考鏈接


下面是我整理蒐集的有關ZooKeeper相關內容的網址，可以學習參考。

中文鏈接：

英文鏈接：




發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Apache ZooKeeper Watcher 機制源碼解釋

分佈式系統從根本上來說就是不同節點上的進程併發執行，並且相互之間對進程的行爲進行協調處理的過程。不同節點上的進程互相協調行爲的過程叫做分佈式同步。許多分佈式系統需要一個進程作爲任務的協調者，執行一些其他進程並不執行的特殊的操作，一般情況下

zmycoco2 2020-07-06 06:45:55

用 sqlRest 將數據庫轉換爲 REST 風格的 Web 服務

本文介紹 sqlRest 框架，它是一種高效的輕量級數據庫 REST 服務解決方案，您可以通過簡單的配置直接將數據庫中的數據暴露成 REST 風格的 Web 服務，並將數據庫的 CRUD 操作和 REST 服務的 GET， DELETE

ibmjournal 2020-07-04 17:38:25

編寫 Firefox 擴展，用於旋轉在線地圖應用程序中的圖像

學習如何使用 JavaScript 和 Imager Perl 模塊連接 Firefox 擴展，從而在 Google Maps 中旋轉圖像塊（tile）。 許多在線地圖應用程序認爲理想的視圖應該始終處於圖像頂部的北端。本文提供的工具和代

ibmjournal 2020-07-04 17:38:25

PHP V5.3 中的新特性，第 1 部分：對象接口的變化

PHP V5.3 計劃在 2008 年底發行，這個版本包含了大量在過去一直處於計劃階段的新特性。PHP V5.3 在最初被譽爲 “只缺少原生 Unicode 支持的 PHP V6”，現在它已經是 PHP V5 系列的全面升級。PHP V

ibmjournal 2020-07-04 17:38:25

搭建world wind WMS server(For world wind c# with LandSAT&SRTM30)之client配置

Client配置 1.       For LandSAT 你可以參考WorldWind/bin/Debug/Config/Earth/@images.tmp中landsat的結構和landsat下載數據中esta.xml中的相關參數，自

BIM极客 2020-06-29 16:24:31

VS2005sp1編譯VirtualPlanetBuilder

之前一篇文章說到編譯OSGGIS，要使用OSGGIS還得先使用VPB來處理數據，那麼下面就來說說編譯VPB吧。   VPB依賴於OSG和gdal，這裏分別是使用如下版本： VPB 0.9.8的源代碼 OSG 2.8.3的預編譯包 gdal

BIM极客 2020-06-29 16:24:31

Lucene 學習開發

package com.highcom.hcgip.lucenesearch;import javax.servlet.*;import javax.servlet.http.*;import java.io.*;import java.

sinbo 2020-06-28 16:00:29

ssh不用密碼登陸

出處：http://linuxproblem.org/art_9.html Your aim You want to use Linux and OpenSSH to automate your tasks. Therefore y

西凉阿土伯 2020-06-26 21:36:54

slf4j點滴

slf4j點滴 最小可用配置 slf4j-api-1.7.12.jar api包加空實現，必須有，如果僅有此jar，則不輸出任何log 其他slf4j-xxx-1.7.12.jar均爲不同的binder實現，實

西凉阿土伯 2020-06-26 21:36:53

cocos v2.2.6&cocos2d-x-3.6實練

1.安裝cocos v2.2.6 安裝好後啓動cocos,在商店裏下載 NDK JDK SDK 以及cocos2d-x framework(其實就是編譯後的cocos2d-x庫),不過這個framework包含了ant和pytho

西凉阿土伯 2020-06-26 21:36:53

Motorola 手機操作系統源代碼

      Motorola A1200 手機操作系統的源代碼，從摩托羅拉的官方網站 Download 下來的，     好東西不敢獨享，把鏈接放在這裏，有興趣的同學們去看看。      http://www.kudev.net/bbs/

MobileFox 2020-06-26 13:17:56

編譯skia靜態庫時，圖片解碼庫無法註冊的問題

轉載：http://www.cnblogs.com/imlucky/archive/2012/08/01/2617851.html 今天編譯skia庫，增加圖片解碼庫時總是無效。按照此博客的方法修改後成功，特此轉載。 androi

Redrain 2020-06-25 12:27:33

樣式佈局分享-基於frozen.js的移動OA

本文轉載於 石佳劼的博客，有問題請到原文諮詢，原文連接。 最近改造了公司移動OA的樣式，改善了顯示效果和用戶體驗，其中用到了騰訊的 frozen.js 框架，在此把佈局代碼分享給大家，希望大家可以用得上，具體效果如下圖。 De

劼哥stone 2020-06-25 06:47:57

Tomcat load balance and clustering - tomcat負載均衡和集羣

參考   http://hi.baidu.com/luodaijun/blog/item/5bbe4cfb5ffef864034f56a1.html   http://blog.csdn.net/zl5897/archive/2008/0

zealVampire 2020-06-21 10:37:04

再談Hibernate the owing session was closed

ERROR] - could not initialize proxy - the owning Session was closed - [org.hibernate.LazyInitializationException.<init>

zealVampire 2020-06-21 10:37:04