loganalyzer

在Linux中,服務、內核輸出的日誌信息都由rsyslog服務收集、展現。

一、ryslog 分爲兩部分:

        1. syslogd,用戶手機應用程序產生的日誌信息。
        2. klogd , 用於收集內核啓動時輸出的信息,通常保存爲二進制文件,可有dmesg命令查看。

二、rsyslog RPM包組成:

[auditor@node1 ~]$ rpm -ql rsyslog
/etc/logrotate.d/syslog
/etc/pki/rsyslog
/etc/rsyslog.conf
/etc/rsyslog.d
/etc/sysconfig/rsyslog
/usr/bin/rsyslog-recover-qi.pl
/usr/lib/systemd/system/rsyslog.service
/usr/lib64/rsyslog
/usr/lib64/rsyslog/imdiag.so
/usr/lib64/rsyslog/imfile.so
/usr/lib64/rsyslog/imjournal.so
/usr/lib64/rsyslog/imklog.so
/usr/lib64/rsyslog/immark.so
/usr/lib64/rsyslog/impstats.so
/usr/lib64/rsyslog/imptcp.so
/usr/lib64/rsyslog/imtcp.so
/usr/lib64/rsyslog/imudp.so
/usr/lib64/rsyslog/imuxsock.so
/usr/lib64/rsyslog/lmnet.so
/usr/lib64/rsyslog/lmnetstrms.so
/usr/lib64/rsyslog/lmnsd_ptcp.so
/usr/lib64/rsyslog/lmregexp.so
/usr/lib64/rsyslog/lmstrmsrv.so
/usr/lib64/rsyslog/lmtcpclt.so
/usr/lib64/rsyslog/lmtcpsrv.so
/usr/lib64/rsyslog/lmzlibw.so
/usr/lib64/rsyslog/mmanon.so
/usr/lib64/rsyslog/mmcount.so
/usr/lib64/rsyslog/mmutf8fix.so
/usr/lib64/rsyslog/omjournal.so
/usr/lib64/rsyslog/ommail.so
/usr/lib64/rsyslog/omprog.so
/usr/lib64/rsyslog/omruleset.so
/usr/lib64/rsyslog/omstdout.so
/usr/lib64/rsyslog/omtesting.so
/usr/lib64/rsyslog/omuxsock.so
/usr/lib64/rsyslog/pmaixforwardedfrom.so
/usr/lib64/rsyslog/pmcisconames.so
/usr/lib64/rsyslog/pmlastmsg.so
/usr/lib64/rsyslog/pmrfc3164sd.so
/usr/lib64/rsyslog/pmsnare.so
/usr/sbin/rsyslogd
/usr/share/doc/rsyslog-7.4.7
/usr/share/doc/rsyslog-7.4.7/AUTHORS
/usr/share/doc/rsyslog-7.4.7/COPYING
/usr/share/doc/rsyslog-7.4.7/COPYING.ASL20
/usr/share/doc/rsyslog-7.4.7/COPYING.LESSER
/usr/share/doc/rsyslog-7.4.7/ChangeLog
/usr/share/man/man5/rsyslog.conf.5.gz
/usr/share/man/man8/rsyslogd.8.gz
/var/lib/rsyslog

/etc/rsyslog.conf                                                   #配置文件
/usr/lib64/*.so                                                       #rsyslog提供的模塊,其中im開頭的用於收集日誌,om開頭的模塊用於輸出、存儲日誌

三、rsyslog 配置文件

    /etc/rsyslog.conf 
    #### MODULES ####   用於加載模塊

    # Provides UDP syslog reception  定義UDP/514端口接收日誌
    #$ModLoad imudp
    #$UDPServerRun 514

    # Provides TCP syslog reception  定義TCP/514端口接收日誌
    #$ModLoad imtcp
    #$InputTCPServerRun 514

    #### GLOBAL DIRECTIVES ####  定義全局選項

    #### RULES ####              定義收集服務、程序什麼級別的日誌以及存放何處
    格式:
    Facility.Priority                                       Target
    Facility:                                   定義設施,按功能對日誌進行分類
            a.  Auth                     認證相關類的日誌
            b.  AuthPriv               認證、授權相關類的日誌
            c.  cron                      計劃任務日誌
            d.  daemon                守護進程類日誌
            e.  local0-local7         允許用戶自定義日誌類
    Priority:
                debug                  調試日誌
                info                      信息日誌
                notice                 通知日誌
                warn                     警告日誌
                error                    錯誤日誌
                crit                        藍色警報日誌
                alert                     橙色警報日誌
                emerg                  紅色警報日誌
    Target:
                @Host                        將日誌發送至某一主機
                USER_NAME             將日誌發送至某在線用戶
                /PATH/TO/SOMEFILE                                                       將日誌發送至某一文件路徑,例如/var/log
                ommysql,host,db_name,user,password                            將日誌發送至MySQL中存儲

四、測試rsyslog

需求: 
   將node1作爲rsyslog Server,接受node2發送過來的日誌。
    node1:192.168.80.10 
    node2:192.168.80.11 

node1的配置:
    #開啓日誌接受功能 UDP/514
    [root@node1 ~]# vim /etc/rsyslog.conf 
    $ModLoad imudp
    $UDPServerRun 514

    [root@node1 ~]# systemctl restart rsyslog
    [root@node1 ~]# ss -unl | grep 514
    UNCONN     0      0            *:514                      *:*                  
    UNCONN     0      0           :::514                     :::*                  

 node2的配置:
    [root@node2 ~]# vim /etc/rsyslog.conf 
    *.*                                                     @192.168.80.10:514
    [root@node2 ~]# systemctl restart rsyslog
    [root@node2 ~]# systemctl restart vsftpd

 驗證:其中有不少node2的vsftpd日誌
        [root@node1 ~]# tailf /var/log/messages 
        Jul 14 02:15:12 node2 systemd: Starting Vsftpd ftp daemon...
        Jul 14 02:15:12 node2 systemd: Started Vsftpd ftp daemon.
        Jul 14 02:15:46 node2 systemd: Stopping Vsftpd ftp daemon...
        Jul 14 02:15:46 node2 systemd: Starting Vsftpd ftp daemon...
        Jul 14 02:15:46 node2 systemd: Started Vsftpd ftp daemon.
        Jul 14 02:15:51 node2 systemd: Starting System Logging Service...
        Jul 14 02:15:51 node2 systemd: Started System Logging Service.
        Jul 14 02:15:58 node2 systemd: Stopping Vsftpd ftp daemon...
        Jul 14 02:15:58 node2 systemd: Starting Vsftpd ftp daemon...
        Jul 14 02:15:58 node2 systemd: Started Vsftpd ftp daemon.
        Jul 14 02:19:49 node2 kernel: perf: interrupt took too long (23735 > 23313), lowering kernel.perf_event_max_sample_rate to 8000
        Jul 27 07:00:01 node1 systemd: Started Session 194 of user root.
        Jul 27 07:00:01 node1 systemd: Starting Session 194 of user root.
        Jul 27 07:01:01 node1 systemd: Started Session 195 of user root.
        Jul 27 07:01:01 node1 systemd: Starting Session 195 of user root.
        Jul 14 02:21:08 node2 systemd: Starting Cleanup of Temporary Directories...
        Jul 14 02:21:08 node2 systemd: Started Cleanup of Temporary Directories.
        Jul 14 02:21:37 node2 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6564" x-info="http://www.rsyslog.com"] exiting on signal 15.
        Jul 14 02:21:37 node2 rsyslogd: [origin software="rsyslogd" swVersion="7.4.7" x-pid="6636" x-info="http://www.rsyslog.com"] start
        Jul 14 02:21:37 node2 systemd: Stopping System Logging Service...
        Jul 14 02:21:37 node2 systemd: Starting System Logging Service...
        Jul 14 02:21:37 node2 systemd: Started System Logging Service.

五、rsyslog + loganalyzer

loganazer 是一個PHP寫的日誌分析、展現程序,運行需要LAMP環境。
由ryslog負責收集日誌、loganazer負責分析、展現、MySQL負責存儲日誌。

loganalyzer 官網: http://loganalyzer.adiscon.com/
 
下面我們搭建個rsyslog+loganalyzer試試水:
node1 : 192.168.80.10     LAMP、Loganalyzer、RsyslogServer、RsyslogClient
node2 : 192.168.80.11     RsyslogClient

1. 安裝LAMP運行環境
    [root@node1 ~]# yum -y install httpd php php-mysql mariadb mariadb-server
2. 安裝Loganalyzer
    #安裝mysql模塊,用於rsyslog驅動MySQL
    [root@node1 ~]# yum -y install rsyslog-mysql

    [root@node1 ~]# vim /etc/rsyslog.conf
            #### MODULES ####  加載MySQL模塊,必須要在MODULES段中
            $ModLoad ommysql

            #開啓TCP/514 、UDP/514端口用於收集日誌
            # Provides UDP syslog reception
            $ModLoad imudp
            $UDPServerRun 514

            # Provides TCP syslog reception
            $ModLoad imtcp
            $InputTCPServerRun 514

            #將收集到的所有日誌都發往MySQL
            *.*                     :ommysql:192.168.80.10,RsyslogDB,rsyslog,123

            #創建用戶、數據庫
    MariaDB [(none)]> CREATE DATABASE RsyslogDB;
    MariaDB [(none)]> GRANT ALL ON RsyslogDB.*  TO 'rsyslog'@'%' IDENTIFIED BY '123';

    #查看MySQL模塊中的文件,並導入sql腳本
    [root@node1 ~]# rpm -ql rsyslog-mysql
            /usr/lib64/rsyslog/ommysql.so
            /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql
    #注意:這個腳本會創建數據庫,根據自己情況修改此腳本,我上面已經創建過數據RsyslogDB了所以改動內容如下:

  [root@node1 ~]# vim rsyslog-mysql.sql 
  USE RsyslogDB;
  CREATE TABLE SystemEvents
  (
        ID int unsigned not null auto_increment primary key,
        CustomerID bigint,
        ReceivedAt datetime NULL,
        DeviceReportedTime datetime NULL,
        Facility smallint NULL,
        Priority smallint NULL,
        FromHost varchar(60) NULL,
        Message text,
        NTSeverity int NULL,
        Importance int NULL,
        EventSource varchar(60),
        EventUser varchar(60) NULL,
        EventCategory int NULL,
        EventID int NULL,
        EventBinaryData text NULL,
        MaxAvailable int NULL,
        CurrUsage int NULL,
        MinUsage int NULL,
        MaxUsage int NULL,
        InfoUnitID int NULL ,
        SysLogTag varchar(60),
        EventLogType varchar(60),
        GenericFileName VarChar(60),
        SystemID int NULL
  );

  CREATE TABLE SystemEventsProperties
  (
        ID int unsigned not null auto_increment primary key,
        SystemEventID int NULL ,
        ParamName varchar(255) NULL ,
        ParamValue text NULL
  );
  [root@node1 ~]# mysql -ursyslog -p123  -D RsyslogDB <rsyslog-mysql.sql 

    #安裝程序自己去官網下
    [root@node1 ~]# tar -xzf loganalyzer-4.1.6.tar.gz  -C /var/www/html/
    [root@node1 html]# ln -sv loganalyzer-4.1.6 loganalyzer
            ‘loganalyzer’ -> ‘loganalyzer-4.1.6’
    [root@node1 html]# chown -R apache loganalyzer
    [root@node1 ~]# touch  /var/www/html/loganalyzer/config.php 
    [root@node1 html]#  chmod 666  /var/www/html/loganalyzer/config.php

    #重啓服務 
    [root@node1 html]# systemctl restart  mariadb httpd rsyslog

    3. 客戶端配置
    [root@node2 ~]# vim /etc/rsyslog.conf 
            *.*                                                     @192.168.80.10:514
    [root@node2 ~]# systemctl restart rsyslog

訪問: http://192.168.80.10/loganalyzer/src
loganalyzer

loganalyzer
loganalyzer
loganalyzer

loganalyzer
loganalyzer
loganalyzer
loganalyzer

loganalyzer

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章