Flume 1.11 部署

Flume官網: https://flume.apache.org/
下載地址: https://archive.apache.org/dist/flume/
文檔地址: https://flume.apache.org/FlumeUserGuide.html

1.安裝部署

1.將二進制包上傳到服務器上[略]

2.解壓二進制包

mkdir /usr/flume
tar -zxvf apache-flume-1.11.0-bin.tar.gz -C /usr/flume/

3.將lib文件夾下的guava-11.0.2.jar刪除以兼容Hadoop 3.1.3+(依據Hadoop版本決定)

rm /usr/flume/apache-flume-1.11.0-bin/lib/guava-11.0.2.jar

注意:刪除 guava-11.0.2.jar 的服務器節點,一定要配置Hadoop的環境變量,否則會拋出 ClassNotFoundException 異常

4.修改配置調整Flume內存

mv /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh.template /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh
vi /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# If this file is placed at FLUME_CONF_DIR/flume-env.sh, it will be sourced
# during Flume startup.

# Enviroment variables can be set here.

# export JAVA_HOME=/usr/lib/jvm/java-8-oracle

# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
# export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote"

export JAVA_OPTS="-Xms2048m -Xmx2048m -Dcom.sun.management.jmxremote"

# Let Flume write raw event data and configuration information to its log files for debugging
# purposes. Enabling these flags is not recommended in production,
# as it may result in logging sensitive user information or encryption secrets.
# export JAVA_OPTS="$JAVA_OPTS -Dorg.apache.flume.log.rawdata=true -Dorg.apache.flume.log.printconfig=true "

# Note that the Flume conf directory is always included in the classpath.
#FLUME_CLASSPATH=""

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章