Filebeat 安装

官方文档

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html

 

一、下载

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.15.2-linux-x86_64.tar.gz
tar xzvf filebeat-7.15.2-linux-x86_64.tar.gz

二、查看当前filebeat 模块

./filebeat modules list

三、启用模块

./filebeat modules enable system nginx kafka

四、配置日志规范

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-reference-yml.html

添加新的日志配置，输出到kafka中

filebeat.inputs:
- type: filestream
  enabled: true
  paths:
    - /data/logs/live/userevent.log
  fields:
    log_topic: notify-topic-prod


# ------------------------------ Kafka Output -------------------------------
output.kafka:
  hosts: ["10.61.153.47:9092"]
  topic: '%{[fields.log_topic]}'
  compression: gzip
  max_message_bytes: 1000000
  codec.format:
    string: '%{[message]}'

五、启动filebeat

./filebeat -c userevent.yml
#后台运行，写入日志到filebeat.out
./filebeat -c userevent.yml > filebeat.out &