SparkStreaming On Kafka之kafka解析和安裝實戰
1、Kafka解析
2、Kafka安裝實戰
Kafka元數據被ZooKeeper管理
Kafka是Scala寫的，所以需要安裝Scala、Java
將slf4j-nop-1.7.6.jar拷貝到kafka的libs目錄下，slf4j用於nohup
配置集羣中每臺機器：
1、配置.bashrc
export KAFKA_HOME=/usr/local/kafka_2.10-0.9.0.10-0
PATH加入{$KAFKA_HOME}/bin2、配置屬性
kafka目錄下server.properties
broker.id=0(每臺機器不同Worker1設置爲1，Worker2設置爲2)
zookeeper.connect=Master:2181,Worker1:2182,Worker2:2181(zookeeper client默認端口)3、啓動（所有機器都要啓動）
nohup ./kafka-server-start.sh ../config/server.properties &創建topic
./kafka-topics.sh --create --zookeeper Master:2181,Worker1:2181,Worker2:2181 --replication-factor 3 --partitions 1 --topic HelloKafka查看topic
./kafka-topics.sh --discribe --zookeeper Master:2181,Worker1:2181,Worker2:2181 --topic HelloKafka配置生產者
./kafka-console-producer.sh --broker-list Master:9092,Worker1:9092,Worker2:9092 --topic HelloKafka在下面輸入
This is DT_Spark!
I’m Rocky!
Life is short, you need Spark!
消費者：
./kafka-console-consumer.sh --zookeeper Master:2181,Worker1:2181,Worker2:2181 --from-beginning --topic HelloKafka
