搭建簡單地最入門級的elk框架，因爲是入門測試用，所以所有組件都在同一臺機器上。
| Elasticsearch | 官網elasticsearch-6.3.0.tar | elasticsearch官方文檔 |
| Kibana | 官網kibana-6.3.0下載 linux64位 | kibana官方文檔 |
| Logstash | 官網logstash-6.3.0.tar | logstash官方文檔 |
下載後上傳到虛擬機並解壓縮（可以複製鏈接使用迅雷下載，會快很多）。
首先要保證jdk環境爲1.8版本及以上。
然後配置elasticsearch，elasticsearch-6.3.0/config/elasticsearch.yml：
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0 ##服務器ip 本機
#
# Set a custom port for HTTP:
#
http.port: 9200 ##服務端口
#
# For more information, consult the network module documentation.
#
保存後啓動elasticsearch，由於使用的是root用戶，啓動elasticsearch導致報錯：
[root@flink1 ELK]# ./elasticsearch-6.3.0/bin/elasticsearch
[2019-11-20T05:47:06,500][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:140) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.3.0.jar:6.3.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86) ~[elasticsearch-6.3.0.jar:6.3.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:104) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:171) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.3.0.jar:6.3.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.3.0.jar:6.3.0]
... 6 more
因此需要創建單獨的用戶組和用戶來啓動elasticsearch，可以參考這裏。
使用新創建的elasticsearch用戶啓動又報錯：
[elasticsearch@flink1 ELK]$ ./elasticsearch-6.3.0/bin/elasticsearch
[2019-11-20T05:56:51,500][INFO ][o.e.n.Node ] [] initializing ...
[2019-11-20T05:56:51,621][INFO ][o.e.e.NodeEnvironment ] [gynAdXC] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [31.7gb], net total_space [36.9gb], types [rootfs]
[2019-11-20T05:56:51,621][INFO ][o.e.e.NodeEnvironment ] [gynAdXC] heap size [1015.6mb], compressed ordinary object pointers [true]
[2019-11-20T05:56:51,622][INFO ][o.e.n.Node ] [gynAdXC] node name derived from node ID [gynAdXC0RKWep3-1m0VnNg]; set [node.name] to override
[2019-11-20T05:56:51,622][INFO ][o.e.n.Node ] [gynAdXC] version[6.3.0], pid[23393], build[default/tar/424e937/2018-06-11T23:38:03.357887Z], OS[Linux/3.10.0-1062.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_191/25.191-b12]
[2019-11-20T05:56:51,622][INFO ][o.e.n.Node ] [gynAdXC] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.eWH4ZVvn, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/opt/ELK/elasticsearch-6.3.0, -Des.path.conf=/opt/ELK/elasticsearch-6.3.0/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [aggs-matrix-stats]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [analysis-common]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [ingest-common]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-expression]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-mustache]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-painless]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [mapper-extras]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [parent-join]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [percolator]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [rank-eval]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [reindex]
[2019-11-20T05:56:57,492][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [repository-url]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [transport-netty4]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [tribe]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-core]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-deprecation]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-graph]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-logstash]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-ml]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-monitoring]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-rollup]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-security]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-sql]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-upgrade]
[2019-11-20T05:56:57,493][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-watcher]
[2019-11-20T05:56:57,494][INFO ][o.e.p.PluginsService ] [gynAdXC] no plugins loaded
[2019-11-20T05:57:03,810][INFO ][o.e.x.s.a.s.FileRolesStore] [gynAdXC] parsed [0] roles from file [/opt/ELK/elasticsearch-6.3.0/config/roles.yml]
[2019-11-20T05:57:04,758][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/23441] [Main.cc@109] controller (64 bit): Version 6.3.0 (Build 0f0a34c67965d7) Copyright (c) 2018 Elasticsearch BV
[2019-11-20T05:57:06,078][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-11-20T05:57:06,325][INFO ][o.e.d.DiscoveryModule ] [gynAdXC] using discovery type [zen]
[2019-11-20T05:57:07,403][INFO ][o.e.n.Node ] [gynAdXC] initialized
[2019-11-20T05:57:07,403][INFO ][o.e.n.Node ] [gynAdXC] starting ...
[2019-11-20T05:57:08,026][INFO ][o.e.t.TransportService ] [gynAdXC] publish_address {172.21.89.128:9300}, bound_addresses {[::]:9300}
[2019-11-20T05:57:08,047][INFO ][o.e.b.BootstrapChecks ] [gynAdXC] bound or publishing to a non-loopback address, enforcing bootstrap checks
ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2019-11-20T05:57:08,082][INFO ][o.e.n.Node ] [gynAdXC] stopping ...
[2019-11-20T05:57:08,199][INFO ][o.e.n.Node ] [gynAdXC] stopped
[2019-11-20T05:57:08,199][INFO ][o.e.n.Node ] [gynAdXC] closing ...
[2019-11-20T05:57:08,220][INFO ][o.e.n.Node ] [gynAdXC] closed
[2019-11-20T05:57:08,224][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started
可以看到有兩個錯誤，第一個是elasticsearch進程的最大文件描述符的值最小要求65536，而目前系統允許的是4096。第二個是當前系統的最大虛擬機內存爲65530，而要求的最小爲262144。因此需要調整這兩個配置。
首先查看當前系統的最大和最小文件描述符：
[elasticsearch@flink1 ELK]$ ulimit -Hn
4096
[elasticsearch@flink1 ELK]$ ulimit -Sn
1024
然後修改配置文件/etc/security/limits.conf：
保存後退出登錄再重新登錄即可生效。
接下來針對第二個報錯，修改/etc/sysctl.conf文件，增加配置vm.max_map_count=262144：
保存後執行命令sysctl -p使其生效。
再次啓動：
[elasticsearch@flink1 ELK]$ ./elasticsearch-6.3.0/bin/elasticsearch
[2019-11-20T06:24:15,158][INFO ][o.e.n.Node ] [] initializing ...
[2019-11-20T06:24:15,281][INFO ][o.e.e.NodeEnvironment ] [gynAdXC] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [31.6gb], net total_space [36.9gb], types [rootfs]
[2019-11-20T06:24:15,281][INFO ][o.e.e.NodeEnvironment ] [gynAdXC] heap size [1015.6mb], compressed ordinary object pointers [true]
[2019-11-20T06:24:15,282][INFO ][o.e.n.Node ] [gynAdXC] node name derived from node ID [gynAdXC0RKWep3-1m0VnNg]; set [node.name] to override
[2019-11-20T06:24:15,282][INFO ][o.e.n.Node ] [gynAdXC] version[6.3.0], pid[23548], build[default/tar/424e937/2018-06-11T23:38:03.357887Z], OS[Linux/3.10.0-1062.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_191/25.191-b12]
[2019-11-20T06:24:15,282][INFO ][o.e.n.Node ] [gynAdXC] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.Loc5bfKs, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/opt/ELK/elasticsearch-6.3.0, -Des.path.conf=/opt/ELK/elasticsearch-6.3.0/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-11-20T06:24:19,622][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [aggs-matrix-stats]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [analysis-common]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [ingest-common]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-expression]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-mustache]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [lang-painless]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [mapper-extras]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [parent-join]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [percolator]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [rank-eval]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [reindex]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [repository-url]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [transport-netty4]
[2019-11-20T06:24:19,623][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [tribe]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-core]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-deprecation]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-graph]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-logstash]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-ml]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-monitoring]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-rollup]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-security]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-sql]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-upgrade]
[2019-11-20T06:24:19,624][INFO ][o.e.p.PluginsService ] [gynAdXC] loaded module [x-pack-watcher]
[2019-11-20T06:24:19,625][INFO ][o.e.p.PluginsService ] [gynAdXC] no plugins loaded
[2019-11-20T06:24:25,052][INFO ][o.e.x.s.a.s.FileRolesStore] [gynAdXC] parsed [0] roles from file [/opt/ELK/elasticsearch-6.3.0/config/roles.yml]
[2019-11-20T06:24:26,182][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/23596] [Main.cc@109] controller (64 bit): Version 6.3.0 (Build 0f0a34c67965d7) Copyright (c) 2018 Elasticsearch BV
[2019-11-20T06:24:27,843][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2019-11-20T06:24:30,194][INFO ][o.e.d.DiscoveryModule ] [gynAdXC] using discovery type [zen]
[2019-11-20T06:24:31,432][INFO ][o.e.n.Node ] [gynAdXC] initialized
[2019-11-20T06:24:31,432][INFO ][o.e.n.Node ] [gynAdXC] starting ...
[2019-11-20T06:24:32,231][INFO ][o.e.t.TransportService ] [gynAdXC] publish_address {172.21.89.128:9300}, bound_addresses {[::]:9300}
[2019-11-20T06:24:32,246][INFO ][o.e.b.BootstrapChecks ] [gynAdXC] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2019-11-20T06:24:38,304][INFO ][o.e.c.s.MasterService ] [gynAdXC] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {gynAdXC}{gynAdXC0RKWep3-1m0VnNg}{EiA0q6OuRxeJ-8k3_yYATg}{172.21.89.128}{172.21.89.128:9300}{ml.machine_memory=1907789824, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2019-11-20T06:24:38,865][INFO ][o.e.c.s.ClusterApplierService] [gynAdXC] new_master {gynAdXC}{gynAdXC0RKWep3-1m0VnNg}{EiA0q6OuRxeJ-8k3_yYATg}{172.21.89.128}{172.21.89.128:9300}{ml.machine_memory=1907789824, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {gynAdXC}{gynAdXC0RKWep3-1m0VnNg}{EiA0q6OuRxeJ-8k3_yYATg}{172.21.89.128}{172.21.89.128:9300}{ml.machine_memory=1907789824, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
[2019-11-20T06:24:38,952][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [gynAdXC] publish_address {172.21.89.128:9200}, bound_addresses {[::]:9200}
[2019-11-20T06:24:38,953][INFO ][o.e.n.Node ] [gynAdXC] started
[2019-11-20T06:24:38,968][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [gynAdXC] Failed to clear cache for realms [[]]
[2019-11-20T06:24:39,603][INFO ][o.e.g.GatewayService ] [gynAdXC] recovered [0] indices into cluster_state
[2019-11-20T06:24:44,679][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.triggered_watches] for index patterns [.triggered_watches*]
[2019-11-20T06:24:44,721][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.watches] for index patterns [.watches*]
[2019-11-20T06:24:44,771][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.watch-history-7] for index patterns [.watcher-history-7*]
[2019-11-20T06:24:44,872][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-6-*]
[2019-11-20T06:24:44,945][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.monitoring-es] for index patterns [.monitoring-es-6-*]
[2019-11-20T06:24:44,996][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.monitoring-beats] for index patterns [.monitoring-beats-6-*]
[2019-11-20T06:24:45,027][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.monitoring-alerts] for index patterns [.monitoring-alerts-6]
[2019-11-20T06:24:45,114][INFO ][o.e.c.m.MetaDataIndexTemplateService] [gynAdXC] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-6-*]
[2019-11-20T06:24:45,211][INFO ][o.e.l.LicenseService ] [gynAdXC] license [9d91c984-5511-4b0c-a9ca-5e61bbf2e622] mode [basic] - valid
啓動成功，打開瀏覽器輸入服務器IP和9200端口，看到如下內容說明啓動正常：
接下來配置kibana，kibana-6.3.0-linux-x86_64/config/kibana.yml
server.port: 5601 ##服務端口
server.host: "0.0.0.0" ##服務器ip 本機
elasticsearch.url: "http://localhost:9200" ##elasticsearch服務地址 與elasticsearch對應
保存後啓動kibana：
[root@flink1 kibana-6.3.0-linux-x86_64]# ./bin/kibana
log [03:53:40.786] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:40.874] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.876] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.880] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.885] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.942] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.943] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:40.955] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:40.957] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:41.682] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.686] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:41.688] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.689] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:41.690] [warning][security] Generating a random key for xpack.security.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.security.encryptionKey in kibana.yml
log [03:53:41.697] [warning][security] Session cookies will be transmitted over insecure connections. This is not recommended.
log [03:53:41.723] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:41.728] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.730] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:41.749] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.754] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.756] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:41.759] [info][status][plugin:[email protected]] Status changed from uninitialized to green - Ready
log [03:53:46.360] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml
log [03:53:46.362] [info][status][plugin:[email protected]] Status changed from uninitialized to yellow - Waiting for Elasticsearch
log [03:53:46.379] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.380] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.380] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.380] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.381] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.381] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.381] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.382] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.382] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.382] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.383] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:53:46.383] [error][status][plugin:[email protected]] Status changed from yellow to red - Request Timeout after 3000ms
log [03:54:01.015] [info][license][xpack] Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active
log [03:54:01.036] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.040] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.041] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.041] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.041] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.041] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.042] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.042] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.042] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.043] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.043] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:01.056] [info][kibana-monitoring][monitoring-ui] Stopping all Kibana monitoring collectors
log [03:54:01.443] [info][license][xpack] Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active
log [03:54:08.214] [info][kibana-monitoring][monitoring-ui] Starting all Kibana monitoring collectors
log [03:54:08.225] [info][status][plugin:[email protected]] Status changed from red to green - Ready
log [03:54:22.613] [info][listening] Server running at http://0.0.0.0:5601
瀏覽器輸入IP地址加端口號5601即可驗證是否正常啓動：
接下來配置安裝logstash
logstash需要有數據源，一般是通過讀取配置文件的方式來識別輸入源和輸出源等組件。下面創建一個es數據源配置文件logback-es.conf，內容如下：
[root@flink1 ELK]# more logstash-6.3.0/config/logback-es.conf
input { ##input 輸入源配置
tcp { ##使用tcp輸入源 官網有詳細文檔
port => 9601 ##服務器監聽端口9061 接受日誌 默認ip localhost
codec => json_lines ##使用json解析日誌 需要安裝json解析插件
}
}
filter { ##數據處理
}
output { ##output 數據輸出配置
elasticsearch { ##使用elasticsearch接收
hosts => "localhost:9200"##集羣地址 多個用，隔開
}
stdout { codec => rubydebug }##輸出到命令窗口
}
保存後就可以通過加載這個配置文件測試logstash了：
[root@flink1 ELK]# ./logstash-6.3.0/bin/logstash -f ./logstash-6.3.0/config/logback-es.conf
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /opt/ELK/hs_err_pid24775.log
結果報錯了，說所需內存不滿足。這時需要修改logstash程序的jvm內存配置logstash-6.3.0/config/jvm.options，這個版本默認的是1g，而我的虛擬機內存只有1g，因此這裏修改成512m：
-Xms512m
-Xmx512m
再次啓動：
[root@flink1 ELK]# ./logstash-6.3.0/bin/logstash -f ./logstash-6.3.0/config/logback-es.conf
Sending Logstash's logs to /opt/ELK/logstash-6.3.0/logs which is now configured via log4j2.properties
[2019-11-21T23:33:44,115][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-11-21T23:33:45,505][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.3.0"}
[2019-11-21T23:33:49,626][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-11-21T23:33:50,530][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-11-21T23:33:50,551][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2019-11-21T23:33:51,234][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-11-21T23:33:51,677][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-11-21T23:33:51,713][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-11-21T23:33:51,806][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2019-11-21T23:33:51,857][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2019-11-21T23:33:53,210][INFO ][logstash.outputs.elasticsearch] Installing elasticsearch template to _template/logstash
[2019-11-21T23:33:56,281][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2019-11-21T23:33:56,381][INFO ][logstash.inputs.tcp ] Starting tcp input listener {:address=>"0.0.0.0:9601", :ssl_enable=>"false"}
[2019-11-21T23:33:57,195][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x55340a94 run>"}
[2019-11-21T23:33:57,484][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-11-21T23:33:58,194][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
接下來爲了方便的使用elk，使用idea創建一個springboot項目：
新建的logback.xml文件：
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration>
<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
<destination>172.21.89.128:9601</destination> <!--指定logstash ip：監聽端口 tcpAppender 可自己實現如kafka傳輸等-->
<encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
<include resource="org/springframework/boot/logging/logback/base.xml"/> <!--引用springboot默認配置-->
<root>
<appender-ref ref="LOGSTASH" /> <!--使用上述訂閱logstash數據tcp傳輸 -->
<appender-ref ref="CONSOLE" /> <!--使用springboot默認配置 調試窗口輸出-->
</root>
</configuration>
pom.xml文件中添加的依賴：
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>4.11</version>
</dependency>
應用代碼：
package com.example.demo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class DemoApplication {
private final static Logger logger = LoggerFactory.getLogger(DemoApplication.class);
public static void main(String[] args) {
new Thread(()->{
for (int i=0;i<100;i++){
logger.info("---test---"+i);
}
}).start();
SpringApplication.run(DemoApplication.class, args);
}
}
啓動應用，之後看到logstash在控制檯輸出了記錄 此爲默認無過濾器打印logback包裝的全部信息（這裏截取一部分）：
{
"thread_name" => "main",
"message" => "Tomcat started on port(s): 8080 (http) with context path ''",
"host" => "172.21.89.1",
"logger_name" => "org.springframework.boot.web.embedded.tomcat.TomcatWebServer",
"port" => 8955,
"level_value" => 20000,
"level" => "INFO",
"@version" => 1,
"@timestamp" => 2019-11-22T02:18:33.769Z
}
{
"thread_name" => "main",
"message" => "Started DemoApplication in 1.251 seconds (JVM running for 2.157)",
"host" => "172.21.89.1",
"logger_name" => "com.example.demo.DemoApplication",
"port" => 8955,
"level_value" => 20000,
"level" => "INFO",
"@version" => 1,
"@timestamp" => 2019-11-22T02:18:33.772Z
}
然後打開kibana瀏覽器，進行索引配置，首先是創建索引隨便取個名字就好，然後選擇以時間屬性作爲過濾條件，之後再次重啓程序就能在kibana看到日誌信息了：
參考博客：https://blog.csdn.net/qq_22211217/article/details/80764568#commentBox
https://blog.csdn.net/oschina_41140683/article/details/93007721