1. 介紹

Logstash 是開源的服務器端數據處理管道，能夠同時從多個來源採集數據，轉換數據，然後將數據發送到您最喜歡的“存儲庫”中。
Beats 平臺集合了多種單一用途數據採集器。它們從成百上千或成千上萬臺機器和系統向 Logstash 或 Elasticsearch 發送數據。
通過Kibana ，您可以對自己的 Elasticsearch 進行可視化，還可以在 Elastic Stack 中進行導航，這樣您便可以進行各種操作了，從跟蹤查詢負載，到理解請求如何流經您的整個應用，都能輕鬆完成
Elasticsearch 是一個分佈式、RESTful 風格的搜索和數據分析引擎，能夠解決不斷湧現出的各種用例。作爲 Elastic Stack 的核心，它集中存儲您的數據，幫助您發現意料之中以及意料之外的情況。
官方文檔

名稱	下載	安裝
Logstash	logstash	yum
Filebeat	filebeat	yum
Kibana	Kibana	yum
Elasticsearch	Elasticsearch	yum

2. 安裝與配置

1. 在/etc/yum.repos.d/目錄新建文件elasticsearch.repo

# 新建文件
touch /etc/yum.repos.d/elasticsearch.repo

# 編輯文件內容 
vim /etc/yum.repos.d/elasticsearch.repo

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[elastic-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

2. 安裝ELK

sudo yum install elasticsearch logstash kibana

# 按需要安裝
sudo yum install filebeat

3. 配置

默認配置文件路徑：/usr/share/xxx/
xxx爲elasticsearch,filebeat,logstash等等

3.1 配置elasticsearch

vim /etc/elasticsearch/elasticsearch.yml

配置信息如下：

# 配置集羣的名稱
cluster.name: my-elasticsearch 
# 當前節點的名稱
node.name: node-1

# 綁定IP地址，外網訪問0.0.0.0 否則綁定localhost
network.host: 0.0.0.0
http.port: 9200
# 允許跨域請求
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type
# 訪問需要密碼
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

# 初始化，必須要配置
cluster.initial_master_nodes: ["node-1"]

3.2 配置TLS和身份驗證：

參考文檔

1. 創建證書

# 生成證書, 兩次回車
/usr/share/elasticsearch/bin/elasticsearch-certutil ca

# 三次回車
/usr/share/elasticsearch//bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 

# 創建目錄
mkdir /etc/elasticsearch/cert
# 複製證書
mv /usr/share/elasticsearch/*.p12 /etc/elasticsearch/cert/

# 修改權限
chown -R elasticsearch:elasticsearch /etc/elasticsearch/cert/

2. 修改配置：

vim /etc/elasticsearch/elasticsearch.yml

http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: cert/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: cert/elastic-certificates.p12

重啓elasticsearch

service elasticsearch restart

3. 生成客戶端證書：

/usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca \
/etc/elasticsearch/cert/elastic-stack-ca.p12 \
-name "CN=esuser,OU=dev,DC=weqhealth,DC=com"

回車
client.p12
回車

拆分證書

mv /usr/share/elasticsearch/client.p12 /etc/elasticsearch/cert/
cd  /etc/elasticsearch/cert/

openssl pkcs12 -in client.p12 -nocerts -nodes > client-key.pem
openssl pkcs12 -in client.p12 -clcerts -nokeys  > client.crt
openssl pkcs12 -in client.p12 -cacerts -nokeys -chain > client-ca.crt

chown -R elasticsearch:elasticsearch /etc/elasticsearch/cert/

3.2 配置kibana.yml

vim /etc/kibana/kibana.yml

配置內容如下:

# 綁定端口
server.port: 5601
# 綁定IP
server.host: "0.0.0.0"
# 
elasticsearch.hosts: ["http://localhost:9200"]

# 訪問密碼,這裏等下要設置,先配置好
elasticsearch.password: "kibanapassword"

# 界面使用中文
i18n.locale: "zh-CN"

3.3 配置logstash

vim /etc/logstash/logstash.yml

配置內容如下：

http.host: "0.0.0.0"
http.port: 9600-9700

# 訪問需要驗證, 先配置，等下設置密碼
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: logstashpassword
xpack.monitoring.elasticsearch.hosts: ["https://localhost:9200"]

作爲系統服務啓動，需要指定Java地址
修改文件：/etc/logstash/startup.options

# 取消註釋,並設置爲自己的Java路徑
JAVACMD=/opt/jdk/bin/java

4. 啓動

4.1 啓動elasticsearch

# 啓動
service elasticsearch start

# 停止
service elasticsearch stop

備註：
啓動失敗可以在/var/log/elasticsearch/中查看詳細日誌

可以通過curl 查看是否啓動成功

curl http://127.0.0.1:9200

4.2 修改各組件密碼

官方文檔

運行命令，設置默認用戶密碼

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

按照提示設置密碼,這裏設置的密碼需要與配置文件中配置的密碼一致。

如果使用filebeat同樣需要指定密碼

monitoring.enabled: true
xpack.monitoring.elasticsearch.username: beats_system
xpack.monitoring.elasticsearch.password: filebeatpasssword

重新啓動elasticsearch

service elasticsearch restart

啓動Kibana

# 啓動
service kibana start

# 停止
service kibana stop

日誌詳細：/var/log/kibana/

啓動logstash
安裝系統服務

# 注意，startup.options 需要已經設置了JAVACMD 否則不能啓動
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

啓動與停止

#啓動
systemctl start logstash.service

# 停止
systemctl stop logstash.service

註冊爲開機啓動

systemctl enable elasticsearch
systemctl enable kibana
systemctl enable logstash.service

使用命令, 查看是否已經enable：

systemctl list-unit-files

5. 查看是否啓動成功

curl -u elastic:changeme http://localhost:9200

結果如下，表示成功：

{
  "name" : "node-1",
  "cluster_name" : "delta_grad",
  "cluster_uuid" : "6JCU_klGTlaVjhzx8hwzXQ",
  "version" : {
    "number" : "7.4.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "2f90bbf7b93631e52bafb59b3b049cb44ec25e96",
    "build_date" : "2019-10-28T20:40:44.881551Z",
    "build_snapshot" : false,
    "lucene_version" : "8.2.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

驗證kibana是否成功

curl -u kibana:changeme http://localhost:9200/_xpack?pretty

通過接口修改密碼：

curl -u elastic:changeme -XPOST 'http://127.0.0.1:9200/_security/user/remote_nitoring_user/_password' -H 'Content-Type: application/json' -d'
> {
>   "password" : "changeme"
> }'

問題:

1. the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

解決方法：

配置文件添加：

node.name: node-1

cluster.initial_master_nodes: [“node-1”]

2. 運行bin/elasticsearch-setup-passwords interactive 報錯：ERROR: X-Pack Security is disabled by configuration.

解決方法：

配置文件中添加：

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

3. 在阿里雲1G內存運行報錯Out of Memory Error:

解決方法：修改 jvm.options
-Xms128m
-Xmx128m

4. Failed to start logstash.service: Unit not found.

解決方法：

/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

5. /usr/share/logstash/bin/system-install: line 88: #: command not found

解決方法：
參考參考2

增大Linux交換內存

sudo /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
sudo /sbin/mkswap /var/swap.1
sudo chmod 600 /var/swap.1
sudo /sbin/swapon /var/swap.1

Centos7 安裝配置ELK

1. 介紹

2. 安裝與配置

1. 在/etc/yum.repos.d/目錄新建文件elasticsearch.repo

2. 安裝ELK

3. 配置

3.1 配置elasticsearch

3.2 配置TLS和身份驗證：

1. 創建證書

2. 修改配置：

3. 生成客戶端證書：

3.2 配置kibana.yml

3.3 配置logstash

4. 啓動

4.1 啓動elasticsearch

4.2 修改各組件密碼

5. 查看是否啓動成功

問題:

1. the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

2. 運行bin/elasticsearch-setup-passwords interactive 報錯：ERROR: X-Pack Security is disabled by configuration.

3. 在阿里雲1G內存運行報錯Out of Memory Error:

4. Failed to start logstash.service: Unit not found.

5. /usr/share/logstash/bin/system-install: line 88: #: command not found

Python多線程編程深度探索：從入門到實戰

35K*14 薪，入職了！這公司只要不裁員，我能一直呆下去！

Centos7 安裝配置ELK

阿里雲Liux SMTP 25端口發送郵件 Connection timeout

SpringBoot2項目部署到Linux(Centos)Log日誌輸出問號

SpringBoot2 靜態資源訪問問題

Mysql集羣-----(01) Oracle VM VirtualBox 安裝Centos7

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結