Flume使用筆記

1、當一臺機器運行多個flume腳本，時執行:ps -aux|grep flume會輸出很多個進程出來，且當本地的環境變量配置很多時（如：Hadoop，Hbase...），此時會在控制檯打印很多JVM加載的依賴，沒辦法看哪個進程運行了哪個flume腳本。此時可以根據端口來找：

（1）netstat -nlpt|grep 44444

（2）第（1）步可以看到對應的進程號，只需要將該進程號kill掉即可。

2、當在採集端配置channel.type=file，source有多個的時候

nginx.channels.channel0.type = file
nginx.channels.channel0.capacity = 1000000
nginx.channels.channel0.batch-size=1000
nginx.channels.channel0.transactionCapacity = 100000
nginx.channels.channel0.keep-alive = 20
nginx.channels.channel0.request-timeout=50000
nginx.channels.channel0.connect-timeout=50000

會報如下錯誤：

23 Apr 2015 16:58:54,058 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception follows.
java.lang.IllegalStateException: Channel closed [channel=channel4]. Due to java.lang.IllegalArgumentException: CheckpointDir /root/.flume/file-channel/checkpoint could not be created
 at org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:352)
 at org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)

解決辦法：把file改成memoery的方式了(但這種方式會丟數據)

nginx.channels.memoryChannel.type = memory
nginx.channels.memoryChannel.capacity = 10000
nginx.channels.memoryChannel.transactionCapacity = 10000