Flume官網: https://flume.apache.org/
下載地址: https://archive.apache.org/dist/flume/
文檔地址: https://flume.apache.org/FlumeUserGuide.html
1.安裝部署
1.將二進制包上傳到服務器上[略]
2.解壓二進制包
mkdir /usr/flume
tar -zxvf apache-flume-1.11.0-bin.tar.gz -C /usr/flume/
3.將lib文件夾下的guava-11.0.2.jar刪除以兼容Hadoop 3.1.3+(依據Hadoop版本決定)
rm /usr/flume/apache-flume-1.11.0-bin/lib/guava-11.0.2.jar
注意:刪除 guava-11.0.2.jar 的服務器節點,一定要配置Hadoop的環境變量,否則會拋出 ClassNotFoundException 異常
4.修改配置調整Flume內存
mv /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh.template /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh
vi /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# If this file is placed at FLUME_CONF_DIR/flume-env.sh, it will be sourced
# during Flume startup.
# Enviroment variables can be set here.
# export JAVA_HOME=/usr/lib/jvm/java-8-oracle
# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
# export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote"
export JAVA_OPTS="-Xms2048m -Xmx2048m -Dcom.sun.management.jmxremote"
# Let Flume write raw event data and configuration information to its log files for debugging
# purposes. Enabling these flags is not recommended in production,
# as it may result in logging sensitive user information or encryption secrets.
# export JAVA_OPTS="$JAVA_OPTS -Dorg.apache.flume.log.rawdata=true -Dorg.apache.flume.log.printconfig=true "
# Note that the Flume conf directory is always included in the classpath.
#FLUME_CLASSPATH=""