文章目錄
環境
- flume使用
| 產品 | 版本 |
|---|---|
| Centos 7 | 3.10.0-1062.9.1.el7.x86_64 |
| java | jdk-8u211-linux-x64 |
| flume | apache-flume-1.9.0-bin |
flume安裝與配置
下載地址
- 清華鏡像
https://mirrors.tuna.tsinghua.edu.cn/apache/flume/
解壓
tar -zxvf apache-flume-1.9.0-bin.tar.gz
創建軟鏈接
ln -s apache-flume-1.8.0-bin/ flume
配置環境變量
vi ~/.bashrc
#FLUME
export FLUME_HOME=/home/hadoop/apache-flume-1.9.0-bin
export PATH=$PATH:$FLUME_HOME/bin
- 生效
source ~/.bashrc
查看版本
flume-ng version
telnet安裝
- 檢查是否安裝 telnet-server和xinetd
rpm -qa telnet-server
rpm -qa xinetd
- 如果沒有安裝過就安裝
yum -y install telnet-server.x86_64 telnet.x86_64 xinetd.x86_64
- 設置開機自啓：
systemctl enable xinetd.service
systemctl enable telnet.socket
- 開啓service：
systemctl start telnet.socket
systemctl start xinetd
測試flume
創建example.properties
cd conf
vim example.properties
# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
在這個配置文件中，·配置了source:a1 sink:k1 channel:c1。a1作爲信息源監聽本機44444端口，並將監聽到的信息轉換成event，channel緩存這些收集到的事件併發送給sink，sink將這些event傳輸到另一個終端上。
運行example.properties
bin/flume-ng agent --conf conf --conf-file conf/example.conf --name a1 -Dflume.root.logger=INFO,console
使用telnet
啓動另一個終端，模擬遠端：
telnet localhost 44444
有如下反饋。
使用後，關閉進程
netstat -lnpt|grep 44444
kill -9 [進程號]