CentOS7上,JDK 1.8.0_231-b11,zookeeper-3.4.6,kafka_2.12-2.3.0
测试multi-broker的例子https://kafka.apache.org/quickstart#quickstart_multibroker。
启动zookeeper
> bin
/zookeeper-server-start
.sh config
/zookeeper
.properties
[root@test_host zookeeper]# jps
9650 Jps
3098 QuorumPeerMain
启动第一台 Kafka server:
查看
config/server.properties:
broker.id=0
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-log
> bin
/kafka-server-start
.sh -daemon config
/server
.properties
启动第二第三台Kafka server:
1 2 |
|
Now edit these new files and set the following properties:
1 2 3 4 5 6 7 8 9 |
|
> bin
/kafka-server-start
.sh config
/server-1
.properties &
...
> bin
/kafka-server-start
.sh config
/server-2
.properties &
...
Now create a new topic with a replication factor of three:
> bin
/kafka-topics
.sh --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 1 --topic my-replicated-topic
[root@test_host kafka_2.12-2.3.0]# bin/kafka-topics.sh --describe --bootstrap-server localhost:9092 --topic my-replicated-topic
Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:segment.bytes=1073741824
Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 0,2,1 Isr: 1,2,0
[root@test_host kafka_2.12-2.3.0]#
bin/kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-topic
>my test message1
>my test message2
>^C
[root@test_host kafka_2.12-2.3.0]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic my-replicated-topic
my test message1
my test message2
目前一切正常。
为了测试高可靠性,现在杀掉第一台kafka server.
[root@test_host kafka_2.12-2.3.0]# ps aux | grep server.properties
找到kafka的进程id, kill -9 id。
查看topic状态
[root@test_host kafka_2.12-2.3.0]# bin/kafka-topics.sh --describe --bootstrap-server localhost:9093 --topic my-replicated-topic Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:segment.bytes=1073741824
Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 0,2,1 Isr: 1,2
发现broker.id=1,broker.id=2的broker活跃,id=0的broker已经不再Isr中。
这时测试生产和消费消息
[root@test_host kafka_2.12-2.3.0]# bin/kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-topic
>message3
>message4
(broker0已经杀掉了,此处连9092肯定失败,所以连broker1的端口9093。)
生产看来没有问题。但是消费端问题来了。
[root@test_host kafka_2.12-2.3.0]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9093 --from-beginning --topic my-replicd-topic
收不到任何消息。等多久都收不到。啊,啊,啊!Why? 换9094端口也是这样。哪里没有配置对吗?
如果此时把第一台broker启动,一切又恢复正常。