Ubuntu18搭建kafka單機集羣

1. 配置zookeeper

https://www.cnblogs.com/convict/p/16438634.html

2. 下載kafka

http://archive.apache.org/dist/kafka/2.8.0/kafka_2.12-2.8.0.tgz

3. 解壓

tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz

放到opt目錄下
mv zookeeper-3.6.3/ /opt

4. 配置kafka數據目錄

mkdir -p /data/kafka/k1/log
mkdir -p /data/kafka/k2/log
mkdir -p /data/kafka/k3/log

5. 配置文件

cd /opt/kafka_2.12-2.8.0/config

config目錄下有個server.properties，這個是模板配置文件，以此爲模板複製一個server1.properties
cp server.properties server1.properties

修改server1.properties內容

broker.id=1

listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://192.168.123.201:9092  # 暴露給外部的地址

log.dirs=/data/kafka/k1/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

以server1.properties，複製server2.properties
cp server.properties server2.properties

修改server2.properties

broker.id=2

listeners=PLAINTEXT://:9093
advertised.listeners=PLAINTEXT://192.168.123.201:9093

log.dirs=/data/kafka/k2/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

以server1.properties，複製server3.properties
cp server.properties server3.properties

修改server3.properties

broker.id=3

listeners=PLAINTEXT://:9094
advertised.listeners=PLAINTEXT://192.168.123.201:9094

log.dirs=/data/kafka/k3/log
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183

6. 啓動kafka集羣

分開多個shell啓動

/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server1.properties
/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server2.properties
/opt/kafka_2.12-2.8.0/bin/kafka-server-start.sh /opt/kafka_2.12-2.8.0/config/server3.properties

7. 查看zookeeper上的brokers

/opt/zookeeper-3.6.3/bin/zkCli.sh
ls /
ls /brokers/ids

確認kafka註冊到zookeeper上了

8. 測試

cd /opt/kafka_2.12-2.8.0/bin

8.1 測試創建topic

./kafka-topics.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --replication-factor 1 --partitions 1 --create --topic MY_TEST_TOPIC

8.2 查看創建的topic

./kafka-topics.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --list

8.3 測試生產者

./kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic MY_TEST_TOPIC

8.4 測試消費者

./kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094 --topic MY_TEST_TOPIC --from-beginning