一、集羣組成說明
1. 搭建 Pulsar 集羣至少需要 3 個組件：ZooKeeper 集羣、BookKeeper 集羣和 broker 集羣（Broker 是 Pulsar 的自身實例)。這三個集羣組件如下：
ZooKeeper 集羣（3 個 ZooKeeper 節點組成）
bookie 集羣（也稱爲 BookKeeper 集羣，3 個 BookKeeper 節點組成）
broker 集羣（3 個 Pulsar 節點組成）
2. Pulsar 的安裝包已包含了搭建集羣所需的各個組件庫。無需單獨下載 ZooKeeper 安裝包和 BookKeeper 安裝包。
二、安裝JDK
在 3 臺 Linux 裸機服務器上安裝 JDK（要求版本不低於 JDK 8）安裝略過
下載最新版本pulsar,https://pulsar.apache.org/download/ 上傳至3臺服務器
解壓安裝包到指定目錄
$tar -zxvf apache-pulsar-2.3.2-bin.tar.gz -C /data
三、配置部署 ZooKeeper 集羣
/data/apache-pulsar-2.3.2$ cd /data/apache-pulsar-2.3.2/conf
/data/apache-pulsar-2.3.2$ vim zookeeper.conf
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/data/zookeeper/zookeeper dataLogDir=/data/zookeeper/logs # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients maxClientCnxns=300 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir autopurge.snapRetainCount=180 # Purge task interval in hours # Set to "0" to disable auto purge feature autopurge.purgeInterval=180 server.1=HOSTNAME:2888:3888 server.2=HOSTNAME:2888:3888 server.3=HOSTNAME:2888:3888 |
# 新建文件目錄
/data/apache-pulsar-2.3.2$ mkdir -PV /data/zookeeper/zookeeper
/data/apache-pulsar-2.3.2$ mkdir -PV /data/zookeeper/logs
# 新建文件 myid，分別在指定的sever上寫入配置文件中指定的編號
server.1服務器上
echo 1 > data/zookeeper/zookeeper/myid
server.2服務器上
/data/apache-pulsar-2.3.2$ echo 2 > data/zookeeper/zookeeper/myid
server.3服務器上
/data/apache-pulsar-2.3.2$ echo 3 > data/zookeeper/zookeeper/myid
# 執行後臺運行命令
/data/apache-pulsar-2.3.2$ bin/pulsar-daemon start zookeeper
# 執行 zookeeper 客戶端連接命令
/data/apache-pulsar-2.3.2$ bin/pulsar zookeeper-shell
#注意Enter 鍵進入命令行界面後，可完全使用 ZooKeeper 的各種命令，如ls、get等命令。使用quit命令退出
#在任一個 ZooKeeper 節點，初始化集羣元數據。
/data/apache-pulsar-2.3.2$bin/pulsar initialize-cluster-metadata \
--cluster pulsar-cluster-zk \
--zookeeper HOSTNAMEIP:2181 \
--configuration-store HOSTNAMEIP:2181 \
--web-service-url http://HOSTNAMEIP:8080 \
--web-service-url-tls https://HOSTNAMEIP:8443 \
--broker-service-url pulsar://HOSTNAMEIP:6650 \
--broker-service-url-tls pulsar+ssl://HOSTNAMEIP:6651
#查看集羣元數據是否初始化成功，看到如下圖所內容，表示初始化成功
$ bin/pulsar zookeeper-shell
四、配置部署 BookKeeper 集羣
編輯 bookkeeper.conf 文件
/data/apache-pulsar-2.3.2$ vim conf/bookkeeper.conf
advertisedAddress=HOSTNAMEIP zkServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 journalDirectory=/data/bookkeeper/journal ledgerDirectories=/data/bookkeeper/ledgers |
#創建bookie所需要目錄
/data/apache-pulsar-2.3.2$ mkdir -pv /data/bookkeeper/
#執行初始化元數據命令；若出現提示，輸入 Y，繼續（只需在一個bookie節點執行一次）
/data/apache-pulsar-2.3.2$ bin/bookkeeper shell metaformat
#以後臺進程啓動bookie
/data/apache-pulsar-2.3.2$ bin/pulsar-daemon start bookie
#按照以上步驟，啓動另外兩個 bookie 節點。
如果出現報錯
就是/data/bookkeeper/journal/current/VERSION /data/bookkeeper/ledgers/current/VERSION 這兩個文件與zk上的文件不一致造成的，將journal 和ledgers文件刪除，在把zk的註冊信息刪除，重新註冊一遍就好
驗證是否啓動成功
/data/apache-pulsar-2.3.2$ bin/bookkeeper shell bookiesanity
/data/apache-pulsar-2.3.2$ vim logs/pulsar-bookie-HOSTNAME.out.2
如果驗證出現以上錯誤。
#解決方案：查看8000端口是否被佔用
$egrep -v '^#|^$' conf/bookkeeper.conf
將prometheusStatsHttpPort=8000 端口更改掉
將bookie關閉
/data/apache-pulsar-2.3.2$ $bin/pulsar-daemon stop bookie
啓動bookie
/data/apache-pulsar-2.3.2$ $bin/pulsar-daemon start bookie
驗證是否啓動成功，出現如下顯示，表示啓動成功Bookie
/data/apache-pulsar-2.3.2$ bin/bookkeeper shell bookiesanity
五、部署配置 Broker 集羣
/data/apache-pulsar-2.3.2$ vim conf/broker.conf
zookeeperServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 configurationStoreServers=ZKSERVER1:2181,ZKSERVER2:2181,ZKSERVER3:2181 advertisedAddress=HOSTNAME clusterName=pulsar-cluster-zk |
#以後臺進程啓動 broker
/data/apache-pulsar-2.3.2$ bin/pulsar-daemon start broker
查看集羣 brokers 節點情況
/data/apache-pulsar-2.3.2$ bin/pulsar-admin brokers list pulsar-cluster
六、docker 安裝 apachepulsar/pulsar-dashboard
#docker安裝略過
#docker run --name pulsar-dashboard -dit -p 80:80 -e SERVICE_URL=http://PULSARSEVERIP:8080 apachepulsar/pulsar-dashboard
通過瀏覽器訪問docker 宿主機IP
