Kafka命令行操作

Kafka命令行操作

查看當前服務器中的所有topic

[root@hadoop141 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --list
[root@hadoop141 kafka]#

爲空 因爲還沒有創建topic

---

創建topic

[root@hadoop141 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --create --partitions 2 --replication-factor 2 --topic first
Created topic "first".

選項說明：

• –create：創建命令
• –partitions：定義分區數
• –replication-factor：定義副本數
• –topic：定義 topic 名

進入 logs 文件夾 可以看到創建得 topic

[root@hadoop141 kafka]# ll logs/
drwxr-xr-x 2 root root  4096 5月   2 17:00 first-0
drwxr-xr-x 2 root root  4096 5月   2 17:00 first-1

[root@hadoop142 kafka]# ll logs/
drwxr-xr-x. 2 root root  4096 5月   2 17:00 first-1

[root@hadoop143 kafka]# ll logs/
drwxr-xr-x. 2 root root  4096 5月   2 17:00 first-0

可以看出 first 有兩個分區 -0 -1，而且每個分區有兩個副本。

注意事項：

• 當你設置的副本數量大於你配置的服務器數量(我的是3個)時：–replication-factor 4 會報錯，所以 Kafka不像Hadoop那樣可以配置超過服務器的數量

[root@hadoop141 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --create --partitions 3 --replication-factor 4 --topic first
Error while executing topic command : replication factor: 4 larger than available brokers: 3
[2020-05-02 17:16:12,981] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 4 larger than available brokers: 3
 (kafka.admin.TopicCommand$)

---

刪除 topic

[root@hadoop141 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --delete --topic first
Topic first is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true

需要server.properties中設置delete.topic.enable=true否則只是標記刪除

#刪除topic功能使能
delete.topic.enable=true

這個選項設置爲 true 的話 topic 會被刪除掉

查看 logs/ 文件夾發現 first 已經被刪除。

---

發送消息

[root@hadoop141 kafka]# bin/kafka-console-producer.sh --broker-list hadoop141:9092 --topic first
>hello
>world

---

消費消息

[root@hadoop142 kafka]# bin/kafka-console-consumer.sh --bootstrap-server hadoop141:9092 --from-beginning --topic first
hello
world

–from-beginning：會把主題中以往所有的數據都讀取出來

---

查看某個Topic的詳情

[root@hadoop143 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --describe --topic first
Topic:first     PartitionCount:2        ReplicationFactor:3     Configs:
        Topic: first    Partition: 0    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: first    Partition: 1    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
You have new mail in /var/spool/mail/root

PartitionCount	ReplicationFactor	Configs
分區	副本	配置信息

---

修改分區數

[root@hadoop143 kafka]# bin/kafka-topics.sh --zookeeper hadoop141:2181 --alter --topic first --partitions 5
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!

查看 logs/ 文件夾:

drwxr-xr-x. 2 root root  4096 5月   2 17:33 first-0
drwxr-xr-x. 2 root root  4096 5月   2 17:31 first-1
drwxr-xr-x. 2 root root  4096 5月   2 17:40 first-2
drwxr-xr-x. 2 root root  4096 5月   2 17:40 first-3
drwxr-xr-x. 2 root root  4096 5月   2 17:40 first-4

---