環境依賴
JDK: 1.8
操作系統: CentOS Linux release 7.4.1708 (Core)
安裝路徑: /home/install_package下
集羣主機: 192.168.11.136, 192.168.11.146
安裝包
zookeeper-3.4.14.tar.gz下載路徑
kafka_2.12-2.3.0.tgz下載路徑
Zookeeper集羣安裝
將Zookeeper安裝包上傳到安裝路徑，進行安裝:
$ mkdir zk
# 創建Zookeeper數據存儲路徑
$ mkdir zk/data
# 創建Zookeeper日誌存放路徑
$ mkdir zk/logs
# 解壓安裝包
$ tar -zxvf zookeeper-3.4.14.tar.gz
# 配置環境變量,添加下述內容
$ vi /etc/profile
export ZK_HOME=/home/install_package/zookeeper-3.4.14
export PATH=$ZK_HOME/bin:$PATH
$ source /etc/profile
# 生成Zookeeper配置文件
$ cd zookeeper-3.4.14/conf
$ cp zoo_sample.cfg zoo.cfg
修改Zookeeper配置
$ vi zoo.cfg
# 數據存放目錄
dataDir=/home/install_package/zk/data
# 日誌存放目錄
dataLogDir=/home/install_package/zk/logs
# 心跳間隔時間，時間單位爲毫秒值
tickTime=2000
# leader與客戶端連接超時時間，設爲5個心跳間隔
initLimit=5
# Leader與Follower之間的超時時間，設爲2個心跳間隔
syncLimit=2
# 客戶端通信端口
clientPort=2181
# 清理間隔，單位是小時，默認是0，表示不開啓
autopurge.purgeInterval=1
# 這個參數和上面的參數搭配使用，這個參數指定了需要保留的文件數目，默認是保留3個
autopurge.snapRetainCount=5
# server.NUM=IP:port1:port2 NUM表示本機爲第幾號服務器；IP爲本機ip地址；
# port1爲leader與follower通信端口；port2爲參與競選leader的通信端口
# 多個實例的端口配置不能重複，如下：
server.0=192.168.101.136:12888:13888
server.1=192.168.101.146:12888:13888
啓動Zookeeper集羣
$ cd /home/install_package/zk/data
# 此處的0是配置文件 server.x的x的值
$ echo '0' > myid
# 啓動Zookeeper
$ zKServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/install_package/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
# 查看Zookeeper狀態
$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/install_package/zookeeper-3.4.14/bin/../conf/zoo.cfg
Mode: follower
安裝Kafka集羣
將Kafka安裝包上傳到安裝路徑，進行安裝:
$ mkdir kafka
# 創建kafka日誌存放路徑
$ mkdir kafka/logs
# 解壓安裝包
$ tar -zxvf kafka_2.12-2.3.0.tgz
# 配置環境變量,添加下述內容
$ vi /etc/profile
export KAFKA_HOME=/home/install_package/kafka_2.12-2.3.0
export PATH=$KAFKA_HOME/bin:$PATH
$ source /etc/profile
# 修改kafka配置
$ cd kafka_2.12-2.3.0/config
$ vi server.properties
修改kafka配置
# broker.id每個實例的值不能重複
broker.id=0
# 配置主機的ip和端口
listeners=PLAINTEXT://192.168.101.136:9092
# 配置日誌存儲路徑
log.dirs=/home/install_package/kafka/logs
# 配置zookeeper集羣
zookeeper.connect=192.168.101.136:2181,192.168.101.146:2181
啓動kafka集羣
# 啓動kafka
$ kafka-server-start.sh -daemon /home/install_package/kafka_2.12-2.3.0/config/server.properties
# 創建一個名爲test-topic的topic，7個分區，複製因子爲2
# 複製因子數量無法超過broker數量
$ kafka-topics.sh --create --zookeeper 192.168.101.136:2181,192.168.101.146:2181 --replication-factor 2 --partitions 7 --topic test-topic
Created topic test-topic.
# 查看test-topic信息
$ kafka-topics.sh --describe --zookeeper 192.168.101.136:2181,192.168.101.146:2181 --topic test-topic
Topic:test-topic PartitionCount:7 ReplicationFactor:2 Configs:
Topic: test-topic Partition: 0 Leader: 0 Replicas: 0,1 Isr: 0,1
Topic: test-topic Partition: 1 Leader: 1 Replicas: 1,0 Isr: 1,0
Topic: test-topic Partition: 2 Leader: 0 Replicas: 0,1 Isr: 0,1
Topic: test-topic Partition: 3 Leader: 1 Replicas: 1,0 Isr: 1,0
Topic: test-topic Partition: 4 Leader: 0 Replicas: 0,1 Isr: 0,1
Topic: test-topic Partition: 5 Leader: 1 Replicas: 1,0 Isr: 1,0
Topic: test-topic Partition: 6 Leader: 0 Replicas: 0,1 Isr: 0,1
啓動kafka生產者和kafka啓動者
在136上啓動生產者
$ kafka-console-producer.sh --broker-list 192.168.101.136:9092,192.168.101.146:9092 --topic test-topic
在146上啓動消費者
$ kafka-console-consumer.sh --bootstrap-server 192.168.101.136:9092,192.168.101.146:9092 --topic test-topic --from-beginning
此時在生產者窗口輸入信息:
消費者窗口會顯示信息：
