原文地址:http://blog.csdn.net/honglei915/article/details/37564329
接下來一步一步搭建Kafka運行環境。
Step 1: 下載Kafka
點擊下載最新的版本並解壓.
> tar -xzf kafka_2.9.2-0.8.1.1.tgz > cd kafka_2.9.2-0.8.1.1
Step 2: 啓動服務
> bin/zookeeper-server-start.sh config/zookeeper.properties & [2013-04-22 15:01:37,495] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig) ...現在啓動Kafka:
> bin/kafka-server-start.sh config/server.properties [2013-04-22 15:01:47,028] INFO Verifying properties (kafka.utils.VerifiableProperties) [2013-04-22 15:01:47,051] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties) ...
Step 3: 創建 topic
> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test可以通過list命令查看創建的topic:
> bin/kafka-topics.sh --list --zookeeper localhost:2181 test
Step 4:發送消息.
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test This is a messageThis is another message
ctrl+c可以退出發送。
Step 5: 啓動consumer
Kafka also has a command line consumer that will dump out messages to standard output.> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning This is a message This is another message
你在一個終端中運行consumer命令行,另一個終端中運行producer命令行,就可以在一個終端輸入消息,另一個終端讀取消息。
Step 6: 搭建一個多個broker的集羣
> cp config/server.properties config/server-2.properties在拷貝出的新文件中添加以下參數:
config/server-1.properties: broker.id=1 port=9093 log.dir=/tmp/kafka-logs-1 config/server-2.properties: broker.id=2 port=9094 log.dir=/tmp/kafka-logs-2
> bin/kafka-server-start.sh config/server-1.properties & ... > bin/kafka-server-start.sh config/server-2.properties & ...創建一個擁有3個副本的topic:
> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic
> bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs: Topic: my-replicated-topic Partition: 0 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0
下面解釋一下這些輸出。第一行是對所有分區的一個描述,然後每個分區都會對應一行,因爲我們只有一個分區所以下面就只加了一行。
- leader:負責處理消息的讀和寫,leader是從所有節點中隨機選擇的.
- replicas:列出了所有的副本節點,不管節點是否在服務中.
- isr:是正在服務中的節點.
向topic發送消息:
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-topic ... my test message 1my test message 2^C消費這些消息:
> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic ... my test message 1 my test message 2 ^C
> ps | grep server-1.properties7564 ttys002 0:15.91 /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java... > kill -9 7564
> bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs: Topic: my-replicated-topic Partition: 0 Leader: 2 Replicas: 1,2,0 Isr: 2,0雖然最初負責續寫消息的leader down掉了,但之前的消息還是可以消費的:
> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic ... my test message 1 my test message 2 ^C
看來Kafka的容錯機制還是不錯的。