Prometheus的安裝配置與grafana安裝

一、Prometheus與Grafana

Prometheus是由SoundCloud開發的開源監控報警系統和時序列數據庫(TSDB)。Prometheus使用Go語言開發，是Google BorgMon監控系統的開源版本。2016年由Google發起Linux基金會旗下的原生雲基金會(Cloud Native Computing Foundation), 將Prometheus納入其下第二大開源項目。Prometheus目前在開源社區相當活躍。Prometheus和Heapster(Heapster是K8S的一個子項目，用於獲取集羣的性能數據。)相比功能更完善、更全面。Prometheus性能也足夠支撐上萬臺規模的集羣。其架構圖如下：

Grafana是一個跨平臺的開源的度量分析和可視化工具，可以通過將採集的數據查詢然後可視化的展示，並及時通知。它主要有以下六大特點：

• 1、展示方式：快速靈活的客戶端圖表，面板插件有許多不同方式的可視化指標和日誌，官方庫中具有豐富的儀表盤插件，比如熱圖、折線圖、圖表等多種展示方式；
• 2、數據源：Graphite，InfluxDB，OpenTSDB，Prometheus，Elasticsearch，CloudWatch和KairosDB等；
• 3、通知提醒：以可視方式定義最重要指標的警報規則，Grafana將不斷計算併發送通知，在數據達到閾值時通過Slack、PagerDuty等獲得通知；
• 4、混合展示：在同一圖表中混合使用不同的數據源，可以基於每個查詢指定數據源，甚至自定義數據源；
• 5、註釋：使用來自不同數據源的豐富事件註釋圖表，將鼠標懸停在事件上會顯示完整的事件元數據和標記；
• 6、過濾器：Ad-hoc過濾器允許動態創建新的鍵/值過濾器，這些過濾器會自動應用於使用該數據源的所有查詢。

 

二、服務端prometheus安裝 

1、下載

wget https://github.com/prometheus/prometheus/releases/download/v2.19.2/prometheus-2.19.2.linux-amd64.tar.gz
tar zxvf prometheus-2.19.2.linux-amd64.tar.gz
mv prometheus-2.19.2.linux-amd64 /usr/local/prometheus

2、創建用戶

groupadd prometheus
useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
chown prometheus.prometheus -R /usr/local/prometheus

3、創建Systemd服務

cat > /etc/systemd/system/prometheus.service <<EOF
[Unit]
Description=prometheus
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus/data 
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

4、啓動Prometheus

systemctl daemon-reload
systemctl start prometheus
systemctl status prometheus
systemctl enable prometheus

5、訪問WEB界面

訪問http://10.207.10.248:9090

三、節點端node_exporter安裝

1、下載

wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz
tar zxvf node_exporter-1.0.1.linux-amd64.tar.gz
mv node_exporter-1.0.1.linux-amd64 /usr/local/node_exporter

2、創建用戶

groupadd prometheus
useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus
chown prometheus.prometheus -R /usr/local/node_exporter

3、創建Systemd服務

cat > /etc/systemd/system/node_exporter.service <<EOF
[Unit]
Description=node_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=/usr/local/node_exporter/node_exporter
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

4、啓動node_exporter

systemctl daemon-reload
systemctl start node_exporter
systemctl status node_exporter
systemctl enable node_exporter

5、驗證啓動成功

curl 127.0.0.1:9100
curl 127.0.0.1:9100/metrics

四、服務端配置

prometheus的服務端通過pull向各個node_exporter節點端抓取信息，需要在各個node上安裝exporter。可以利用 Prometheus 的 static_configs 來拉取 node_exporter 的數據。

編輯prometheus.yml文件，添加內容:

- job_name: 'node'
    static_configs:
    - targets: ['XXX.XXX.XXX.XXX:9100']

完整配置如下：

[root@localhost prometheus]# cat prometheus.yml 
# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'server248'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
    - targets: ['172.17.10.248:9090']

  - job_name: 'node247'
    static_configs:
    - targets: ['172.17.10.247:9100']
  - job_name: 'node246'
    static_configs:
    - targets: ['172.17.10.246:9100']
  - job_name: 'node244'
    static_configs:
    - targets: ['172.17.10.244:9100']

重啓prometheus，然後在Prometheus頁面中的Targets中就能看到新加入的node:

systemctl restart prometheus

然後再次查詢web界面如下：

訪問http://10.207.10.248:9090

五、集成各服務監控

1、 nginx

https://github.com/knyar/nginx-lua-prometheus

2、 php
https://github.com/bakins/php-fpm-exporter

3、 mysql
https://github.com/prometheus/mysqld_exporter

（1）下載

wget https://github.com/prometheus/mysqld_exporter/releases/download/v0.12.1/mysqld_exporter-0.12.1.linux-amd64.tar.gz
tar zxvf mysqld_exporter-0.12.1.linux-amd64.tar.gz
mv mysqld_exporter-0.12.1.linux-amd64 /usr/local/mysqld_exporter

（2）修改所屬

chown prometheus.prometheus -R /usr/local/mysqld_exporter

（3）創建Systemd服務

cat > /etc/systemd/system/mysqld_exporter.service <<EOF
[Unit]
Description=mysqld_exporter
After=network.target
[Service]
Type=simple
User=prometheus
ExecStart=ExecStart=/usr/local/mysqld_exporter/mysqld_exporter --config.my-cnf=/usr/local/mysqld_exporter/.my.cnf
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF

（4）創建密碼填寫內容.my.cnf文件

vim .my.cnf
[client]
host=127.0.0.1
port=3306
user=root
password=root

（5）啓動mysqld_exporter

systemctl daemon-reload
systemctl start mysqld_exporter
systemctl status mysqld_exporter
systemctl enable mysqld_exporter

4、 zookeeper
https://github.com/jiankunking/zookeeper_exporter

5、 kafka
https://github.com/danielqsj/kafka_exporter

6、 redis
https://github.com/oliver006/redis_exporter

7、 postgresql

https://github.com/wrouesnel/postgres_exporter

8、 springboot

https://blog.csdn.net/qq_33257527/article/details/88294016

更多expoter參見官網：https://prometheus.io/docs/instrumenting/exporters/#exporters-and-integrations

 

六、Grafana安裝

官方地址：https://grafana.com/grafana/download?platform=linux

1、下載

wget https://dl.grafana.com/oss/release/grafana-7.0.5-1.x86_64.rpm

2、安裝

yum install -y grafana-7.0.5-1.x86_64.rpm

3、配置

配置文件位於/etc/grafana/grafana.ini，這裏暫時保持默認配置即可

4、啓動

systemctl enable grafana-server
systemctl start grafana-server

5、訪問

http://10.207.10.248:3000
用戶名：admin
密碼：admin