下載 wget ftp://ftp.kddilabs.jp/infosystems/apache/kafka/2.2.1/kafka_2.11-2.2.1.tgz
配置文件
broker.id=0
port=9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/opt/kafka/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.10.21:2181
zookeeper.connection.timeout.ms=6000
host.name=pmaster
group.initial.rebalance.delay.ms=0
nohup /home/devel/tz/kafka/kafka/bin/kafka-server-start.sh /home/devel/tz/kafka/kafka/config/server.properties >/dev/null 2>&1 &
查看當前服務器中的所有topic bin/kafka-topics.sh --list --zookeeper
zk01:2181
- 創建topic
bin/kafka-topics.sh --create --zookeeper 192.168.10.13:21812 --replication-factor 3 --partitions 3 --topic boray
- 刪除topic
bin/kafka-topics.sh --delete --zookeeper pmaster:2181 --topic test
需要 server.properties 中設置 delete.topic.enable=true 否則只是標記刪除或者直接重啓。
- 通過shell命令發送消息
bin/kafka-console-producer.sh --broker-list 192.168.10.11:9092 --topic haha
- 通過shell消費消息
bin/kafka-console-consumer.sh --zookeeper 192.168.10.13:21812 --from-beginning --topic boray
bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper pmaster:2181 --group
testGroup
shell生成
bin/kafka-console-producer.sh --broker-list 192.168.10.15:9092 --topic test
shell消費
bin/kafka-console-consumer.sh --bootstrap-server 192.168.10.13:9092 --topic boray
查看所有消費組
bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --list
查看偏移量
bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --describe --group test-consumer-group
查看某個Topic的詳情
bin/kafka-topics.sh --topic test --describe --zookeeper zk01:2181
對分區數進行修改
bin/kafka-topics.sh --zookeeper zk01 --alter --partitions 2 --topic test
scp -r 分發到所有的節點