Linux下安裝Kafka單機僞集羣

前提：已安裝zookeeper，參考：https://pandora.blog.csdn.net/article/details/106675933

kafka下載

鏈接：https://pan.baidu.com/s/1YbeWcNLKhabz-1dF9_XEMQ

提取碼：rhxw

解壓

> tar -xf kafka_2.12-2.3.0.tgz

修改配置config/server.properties

# 9090節點
broker.id=0

listeners=PLAINTEXT://*.*.*.*:9090

advertised.listeners=PLAINTEXT://127.0.0.1:9090

log.dirs=/owl/log/kafka-2.0.0/9090

zookeeper.connect=0.0.0.0:2180

複製2份，修改配置

# 9091節點
broker.id=1

listeners=PLAINTEXT://*.*.*.*:9091

advertised.listeners=PLAINTEXT://127.0.0.1:9091

log.dirs=/owl/log/kafka-2.0.0/9091

zookeeper.connect=0.0.0.0:2181

# 9092節點
broker.id=2

listeners=PLAINTEXT://*.*.*.*:9092

advertised.listeners=PLAINTEXT://127.0.0.1:9092

log.dirs=/owl/log/kafka-2.0.0/9092

zookeeper.connect=0.0.0.0:2182

編寫啓停腳本

# start_9090.sh
cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-server-start.sh ./config/server.properties &

# start_9091.sh
cd /owl/soft/kafka-2.3.0/9091
./bin/kafka-server-start.sh ./config/server.properties &

# start_9092.sh
cd /owl/soft/kafka-2.3.0/9092
./bin/kafka-server-start.sh ./config/server.properties &

# start_cluster.sh 
cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-server-start.sh ./config/server.properties &
cd /owl/soft/kafka-2.3.0/9091
./bin/kafka-server-start.sh ./config/server.properties &
cd /owl/soft/kafka-2.3.0/9092
./bin/kafka-server-start.sh ./config/server.properties &

# stop_9090.sh 
cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-server-stop.sh

# stop_9091.sh 
cd /owl/soft/kafka-2.3.0/9091
./bin/kafka-server-stop.sh

# stop_9092.sh 
cd /owl/soft/kafka-2.3.0/9092
./bin/kafka-server-stop.sh

#  stop_cluster.sh 
cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-server-stop.sh
cd /owl/soft/kafka-2.3.0/9091
./bin/kafka-server-stop.sh
cd /owl/soft/kafka-2.3.0/9092
./bin/kafka-server-stop.sh

啓動kafka集羣

> ./start_cluster.sh 

如果報錯：kafka.common.InconsistentBrokerIdException: Configured brokerId 3 doesn’t match stored brokerId 1 in meta.properties，則修改相應logDir目錄下的meta.properties

broker.id=相應節點的broker.id

進入zookeeper集羣，查看kafka集羣狀態

> ./conn_cluster.sh 

[zk: ****:2180,****:2181,****:2182(CONNECTED) 0] ls /brokers/ids

[0, 1, 2]   

# 節點啓動正常

編寫主題創建/主題描述/消息發佈/消息訂閱腳本

# create_topic_9090.sh 主題創建

cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-topics.sh  --create --zookeeper 127.0.0.1:2180 --replication-factor 1 --partitions 1 --topic $1

echo "獲取已發佈主題："
./bin/kafka-topics.sh --list --zookeeper 127.0.0.1:2180



# describe_topic_9090.sh 主題描述

cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-topics.sh  --describe --zookeeper 127.0.0.1:2180 --topic $1


# publish_message_9090_topic.sh 消息發佈

cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-console-producer.sh --broker-list 127.0.0.1:9090 --topic $1

# subscribe_message_9090_topic.sh 消息訂閱

cd /owl/soft/kafka-2.3.0/9090
./bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9090 --topic $1  --from-beginning

測試消息系統

創建主題

> ./create_topic_9090.sh test-topic
...
Created topic test-topic.
...
獲取已發佈主題：
...
test-topic

查看主題

> ./describe_topic_9090.sh test-topic

Topic:test-topic        PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: test-topic       Partition: 0    Leader: 1       Replicas: 1     Isr: 1

發佈消息

> ./publish_message_9090_topic.sh test-topic

>hello
>hi

訂閱消息

> ./subscribe_message_9090_topic.sh test-topic
hello
hi