一、配置flume(apache-flume-1.6.0-src.tar.gz)
1.解壓
tar -xvf apache-flume-1.6.0-src.tar.gz
2.配置環境變量
vi ~/.bash_profile
export FLUME_HOME=/home/shaka/flume/flume-1.6.0
export PATH=$PATH:$FLUME_HOME/sbin
source ~/.bash_profile
3.啓動/關閉flume
1)在conf下編寫文件 如flume-source-conf.properties
2)啓動flume start source
3)關閉flume stop source
二、配置kafka(kafka_2.9.2-0.8.1.1)
1) 根據zookeeper的conf/zoo.cfg
server.1=10.1.73.8:4887:5887
2)配置config下的server.properties
zookeeper.connect=localhost:2181
3)相關操作
開啓：bin/kafka-server-start.sh config/server.properties &
查看：bin/kafka-topics.sh --list --zookeeper 127.0.0.1:2181
創建：bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic gaoyujie
刪除：bin/kafka-run-class.sh kafka.admin.DeleteTopicCommand --topic taoyujie --zookeeper 127.0.0.1:2181
生產者：bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic-003
消費者：bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic topic-003
生產者發送信息，消費者接收
開兩個終端！
三、配置storm
1.配置zookeeper(zookeeper-3.3.6)
1)獲取本地ip：ifconfig
2)修改conf/zoo.cfg
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
dataDir=/home/shaka/zookeeper/zookeeper-3.3.6/data
# the port at which the clients will connect
clientPort=2181
# set logs
dataLogDir=/home/shaka/zookeeper/zookeeper-3.3.6/logs
# set server
server.1=10.1.73.8:4887:5887
#server.2=10.162.219.52:4887:5887
#server.3=10.163.15.119:4887:5887
# add by shaka
# set max client connects
maxClientCnxns=300
3)啓動/關閉
啓動服務bin/zkServer.sh start
啓動客戶端bin/zkCli.sh -server 127.0.0.1:2181
bin/zkServer.sh status
2.配置依賴環境dependence(事先安裝gcc-c++)
1)zeromq-4.0.4
①解壓
②./configure
③make(root)
④make install(root)
2) jzmq
①解壓
②./configure
③make(root)
④make install(root)3.配置storm(apache-storm-0.9.2-incubating)
1)配置環境變量
vi ~/.bash_profile
export STORM_HOME=/home/shaka/storm/apache-storm-0.9.2-incubating
export PATH=$PATH:$STORM_HOME/bin
source ~/.bash_profile
2)配置文件
vi storm.yaml
storm.zookeeper.servers:
- "10.2.73.8"
# - "10.162.219.52"
#
nimbus.host: "10.2.73.8"
storm.local.dir: "/home/shaka/storm/apache-storm-0.9.2-incubating/topology"
#ui.port: "18080"
supervisor.slots.ports:
- 6700
- 6701
- 6702
4.啓動
strom nimbus &
strom supervisor &
strom ui &
5.操作
查看ui:http://localhost:8080
正在進行的任務:storm list
停止任務:strom kill word-count