客户端配置rsyslog（centos自带rsyslog，不需要另外下载）
vim /etc/rsyslog.conf
#### MODULES ####
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
$ModLoad imfile #load the dimfile module
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging on
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.(内核)
kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!(记录的内核消息、各种服务的公共消息，报错信息等)
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.(包含验证和授权方面信息)
authpriv.* /var/log/secure
# Log all the mail messages in one place.(包含来着系统运行电子邮件服务器的日志信息)
mail.* -/var/log/maillog
# Log cron stuff(每当cron进程开始一个工作时，就会将相关信息记录在这个文件中)
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log(自定义的消息)
local7.* /var/log/boot.log
# 监视指定路径
$InputFileName /usr/local/nginx/logs/access.log
# 设置唯一标签（唯一，必须唯一）
$InputFileTag req_access
# 数据类型
$InputFileSeverity info
$InputFileStateFile /etc/rsyslog.d/stat-access
# 设置设备名为local5
$InputFileFacility local5
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
# 将此设备所有的数据全部发送到远程服务器中 @：UDP协议，@@：TCP协议
local5.* @@192.168.194.6:514
$InputFileName /usr/local/nginx/logs/error.log
$InputFileTag req_error
$InputFileSeverity info
$InputFileStateFile /etc/rsyslog.d/stat-error
$InputFileFacility local6
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
local6.* @@192.168.194.6:514
日志类型：
auth –pam产生的日志
authpriv –ssh,ftp等登录信息的验证信息
cron –时间任务相关
kern –内核
lpr –打印
mail –邮件
mark(syslog)–rsyslog服务内部的信息,时间标识
news –新闻组
user –用户程序产生的相关信息
uucp –unix to unix copy, unix主机之间相关的通讯
local 1~7 –自定义的日志设备
连接符号：
.xxx: 表示大于等于xxx级别的信息
.=xxx：表示等于xxx级别的信息
.!xxx：表示在xxx之外的等级的信息
日志级别：
级别从低到高，记录的信息越来越少
debug –有调式信息的，日志信息最多
info –一般信息的日志，最常用
notice –最具有重要性的普通条件的信息
warning –警告级别
err –错误级别，阻止某个功能或者模块不能正常工作的信息
crit –严重级别，阻止整个系统或者整个软件不能正常工作的信息
alert –需要立刻修改的信息
emerg –内核崩溃等严重信息
none –什么都不记录
处理方式：
/var/log/file 发送到日志文件
@@192.168.0.1 发送到TCP server
@192.168.0.1 发送到UDP server
user1,user2 发送到在线用户user1，user2
~ 丢弃该日志
^/path/script 执行的脚本，^后面跟可以执行的脚本，日志内容可以作为脚本的第一个参数，可以用来触发告警
在配置服务端之前可以在服务端抓包，查看数据是否传输过来
tcpdump -i ens33 port 514
显示两边有数据往来后，再配置logstash
服务端配置rsyslog
# rsyslog configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514
# Provides TCP syslog reception
#### GLOBAL DIRECTIVES ####
# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on
# File to store the position in the journal
$IMJournalStateFile imjournal.state
#### RULES ####
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local5.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList # run asynchronously
#$ActionResumeRetryCount -1 # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# end of the forwarding rule ###
# ********************Please copy the following text************************
# 配置远程来的数据格式和存放路径
$template SpiceTmpl1,"%msg%\n"
$template DynaFile1,"/var/log/remote_nginx/access-%$YEAR%-%$MONTH%-%$DAY%.log"
local5.* ?DynaFile1;SpiceTmpl1
$template SpiceTmpl,"%msg%\n"
$template DynaFile,"/var/log/remote_nginx/error-%$YEAR%-%$MONTH%-%$DAY%.log"
local6.* ?DynaFile;SpiceTmpl
查看路径中是否新建日志文件并产生实时数据：
tail -f /var/log/remote_nginx/access-2020-01-06.log
TIP:
两边都配置完rsyslog可以传输数据后，如果想将服务器端换成成logstash收集日志，配置完logstash后，直接启动会报错：
查看端口使用情况
netstat -an |grep 514
在服务端使用 systemctl stop rsyslog 命令关闭rsyslog后，就不会出现514端口被占用，address already in used 的情况。
再次启动logstash 收集日志就OK啦
