ELK集羣搭建及logstash+kibana展示--logstash

第一篇：logstash安裝加簡單配置

https://blog.csdn.net/mayancheng7/article/details/84754571

第二篇：elasticsearch安裝加簡單配置

https://blog.csdn.net/mayancheng7/article/details/84767669

第三篇：kibana安裝加簡單配置

https://blog.csdn.net/mayancheng7/article/details/84771319

官網地址：https://www.elastic.co/cn/downloads

下載鏈接，最新版的6.5.1。

Elasticsearch  Kibana Logstash 記得版本保持一致

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.1.tar.gz
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.5.1-linux-x86_64.tar.gz

機器list

ip	logstash	elasticsearch	Kibana
192.168.3.17	√	√	√
192.168.3.16	×	√	×
192.168.3.18	×	√	×

解壓logstash

tar zxvf src/logstash-6.5.1.tar.gz
vim logstash-6.5.1/config/std_test.conf            //新建測試conf
input {
    stdin{
    }
} 

output {
    stdout{
    }
}

測試

./logstash-6.5.1/bin/logstash -f logstash-6.5.1/config/std_test.conf        //啓動測試，如果出現下列字符就表示安裝安裝成功

Sending Logstash logs to /home/yx/ma/logstash-6.5.1/logs which is now configured via log4j2.properties
[2018-11-28T17:38:43,938][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/home/yx/ma/logstash-6.5.1/data/queue"}
[2018-11-28T17:38:43,957][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/home/yx/ma/logstash-6.5.1/data/dead_letter_queue"}
[2018-11-28T17:38:44,791][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-11-28T17:38:44,813][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.5.1"}
[2018-11-28T17:38:44,866][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"469786ff-14c5-4b35-acc9-087b5e2ee47f", :path=>"/home/yx/ma/logstash-6.5.1/data/uuid"}
[2018-11-28T17:38:51,366][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-11-28T17:38:51,561][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x4af52480 run>"}
The stdin plugin is now waiting for input:
[2018-11-28T17:38:51,664][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2018-11-28T17:38:52,103][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

新建調用nginx日誌的配置文件

input {
	file{
	path =>"/home/yx/server/nginx/logs/*.log"            #通配符*號
	start_position=>"beginning"
	}
}

filter{
	grok{
		patterns_dir => "*.log"                            #通配符*號
		match=>{"message"=>"%{DATA:clientIp} - - \[%{HTTPDATE:accessTime}\] \"%{DATA:method} %{DATA:requestPath} %{DATA:httpversion}\" %{DATA:retcode} %{DATA:size} \"%{DATA:fromHtml}\" \"%{DATA:useragent}\""                #具體各項配置詳解可參考鏈接：http://udn.yyuap.com/doc/logstash-best-practice-cn/index.html
		}
		remove_field=>"message"
	}
	date{
		match=>["accessTime","dd/MMM/yyyy:HH:mm:ss Z"]
	}
}
output {
    stdout{
    	codec=>rubydebug
    }
}

新建調用PHP日誌的配置文件，只有input的輸入路徑不同，其他都一樣。

input {
	file{
	path =>"/home/yx/server/php56/log/php-error.log"        #單個日誌例子
	start_position=>"beginning"
	}
}

filter{
	grok{
		patterns_dir => "*.log"
		match=>{"message"=>"%{DATA:clientIp} - - \[%{HTTPDATE:accessTime}\] \"%{DATA:method} %{DATA:requestPath} %{DATA:httpversion}\" %{DATA:retcode} %{DATA:size} \"%{DATA:fromHtml}\" \"%{DATA:useragent}\""
		}
		remove_field=>"message"
	}
	date{
		match=>["accessTime","dd/MMM/yyyy:HH:mm:ss Z"]
	}
}
output {
    stdout{
    	codec=>rubydebug
    }
}

指定多配置文件啓動，個人感覺也可以將所有需顯示的日誌軟連接到一個單獨目錄裏，然後配置文件只寫一個就好。

./logstash-6.5.1/bin/logstash -f logstash-6.5.1/config/std_nginx.conf -f logstash-6.5.1/config/std_php.conf     //前臺啓動，可以實時查看日誌輸出。
nohup ./logstash-6.5.1/bin/logstash -f logstash-6.5.1/config/std_nginx.conf -f logstash-6.5.1/config/std_php.conf & //後臺啓動並且不會影響輸出。

此處可以另開兩個窗口，tail -f  查看nginx的日誌和PHP的日誌，從而和logstash輸出的日誌做對比。

如果沒有意外，到此logstash已經安裝成功。請看第二篇

elasticsearch的安裝

參考鏈接：https://blog.csdn.net/BuquTianya/article/details/72027209