【原創】kafka安裝與啓動

在spark001、spark002、spark003三臺機器上。

版本：kafka_2.10-0.8.2.1.zip

啓動命令：

nohup bin/kafka-server-start.sh start.sh config/server.properties > kafka.log 2>&1 &

在spark001上：

vim server.properties

```

broker.id=0

port=9092

#裏面存放的其實就是kafka數據

log.dirs=/kafka-logs

num.partitions=1 #topic的partition數量

log.retention.hours=168 #7天

zookeeper.connect=spark001:2191,spark002:2181,spark003:2181

```

分發配置文件。

每臺機器上啓動Zookeeper:

./bin/zkServer.sh start

每臺機器上啓動kafka：

nohup bin/kafka-server-start.sh config/server.properties > kafka.log 2>&1 &

# jps

Jps

QuorumPeerMain

Kafka

./bin/kafka-topic.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --list

./bin/kafka-topic.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --describe

./bin/kafka-topic.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --create --topic 20170920 --partitions 3 --replication-factor 2

./bin/kafka-topic.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --describe --topic 20170920

./bin/kafka-topic.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --delete --topic 20170920

./bin/kafka-console-consumer.sh --zookeeper spark001:2181,spark002:2181,spark003:2181 --topic 20170920

./bin/kafka-console-producer.sh --broker-list spark001:9092,spark002:9092,spark003:9092 --topic 20170920