下載docker鏡像
docker pull docker.elastic.co/logstash/logstash:6.6.2
創建文件夾 (用於容器文件的掛載)
mkdir /opt/sns/logstash/config
創建相關配置文件
將以下文件放到/opt/sns/logstash/config下
logstash.yml (空文件就行)
log4j2.properties
logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
logger.elasticsearchoutput.level = debug
pipelines.yml(那個小槓槓很重要)
- pipeline.id: my-logstash
path.config: "/usr/share/logstash/config/*.conf"
pipeline.workers: 3
*.conf 文件
input {
jdbc {
jdbc_connection_string => "jdbc:mysql://***.***.***.***:3306/***?useUnicode=true&characterEncoding=utf-8&autoReconnect=true&failOverReadOnly=false&serverTimezone=GMT%2B8"
jdbc_user => "****"
jdbc_password => "*****"
jdbc_driver_library => "/usr/share/logstash/config/mysql-connector-java.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "5000"
use_column_value => true
tracking_column => "sid"
record_last_run => true
lowercase_column_names => "false"
last_run_metadata_path => "/usr/share/logstash/config/*****_last.txt"
statement => "select *
from **** table
where table.sid > :sql_last_value
ORDER BY table.sid"
schedule => "* * * * *"
type => "TYPE_NAME"
}
}
output {
if[type] == "TYPE_NAME"{
elasticsearch {
hosts => ["***.***.***.***:9200"]
index => "***"
document_type => "TYPE_NAME"
document_id => "%{sid}"
}
}
}
如果conf文件中如上需要使用連接數據庫的話,需要jdbc包。
可以將jdbc包放在掛載的/opt/sns/logstash/config下
創建並啓動容器
docker run -d --name logstash --network host --restart=always -it -v /opt/sns/logstash/config:/usr/share/logstash/config docker.elastic.co/logstash/logstash:6.6.2