ELK日誌分析系統部署

軟件版本列表：

• OS: centos 6.6

• redis: redis 2.4.10

• jdk: jdk-8u51-linux-x64

• elasticsearch: elasticsearch

• kibana: kibana-4.1.1

• logstash: logstash-1.5.3

安裝elk

1.redis軟件安裝

#yum -y install epel-release
#yum makecache
#yum -y install redis
#chkconfig redis on

2.jdk安裝 jdk從官網下載，版本新一點比較好

#rpm -ivh jdk-8u51-linux-x64.rpm

3.安裝logstash 從官網下載源碼包

#mkdir -p /usr/local/elk/
#tar zxvf logstash-1.5.3.tar.gz
#mv logstash-1.5.3 /usr/local/elk/logstash

4.安裝es 從官網下載源碼包

#unzip elasticsearch-1.7.0.zip
#mv elasticsearch-1.7.0 /usr/local/elk/es

5.安裝kibana 從官網下載源碼包

#tar zxvf kibana-4.1.1-linux-x64.tar.gz
#mv kibana-4.1.1-linux-x64 /usr/local/elk/kibana

配置elk

1.啓動es

#/usr/local/elk/es/bin/elasticsearch -d

2.配置並啓動logstash 創建配置文件目錄

#mkdir /usr/local/elk/logstash/etc

創建一個從日誌文件讀取，並寫入redis的配置文件(本文件採用默認方式進行輸入，輸出)

#cat agent.conf
input {
    file {
        path => "/var/log/httpd/access_log"    //設置讀取的日誌路徑
        sincedb_path => "../.sincedb"
        type => "httpd"
        start_position => "beginning"
    }
}
output {
    redis {
        host => ["127.0.0.1"]
        port => 6379
        batch => true
            batch_events => 5
            data_type => "list"
            key => "logstash:redis"
    }
}

配置一個從redis讀取日誌並輸出到es的配置文件

#cat index.conf
input {
  redis {
    host => ["127.0.0.1"]
    port => 6379
    data_type => "list"
    key => "logstash:redis"
  }
}

output {
        elasticsearch {
        host => "127.0.0.1"
        protocol => "http"
        index => "logstash-%{type}-%{+YYYY.MM.dd}"
        index_type => "%{type}"
        }
}

啓動logstash

#/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/etc/agent.conf -l /var/log/elk/logstash/logstash.log &

#/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/etc/index.conf -l /var/log/elk/logstash/logstash.log &

啓動kibana

/usr/local/elk/kibana/bin/kibana &

寫入開機啓動

#cat /etc/rc.d/rc.local
/usr/sbin/ntpdate ntp.sjtu.edu.cn

/usr/local/elk/es/bin/elasticsearch -d

/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/etc/agent.conf -l /var/log/elk/logstash/logstash.log &

/usr/local/elk/logstash/bin/logstash -f /usr/local/elk/logstash/etc/index.conf -l /var/log/elk/logstash/logstash.log &

/usr/local/elk/kibana/bin/kibana &

測試訪問elk

http://ip:5601