grafana+prometheus部署中間件監控

       根據grafana+prometheus組件來監控中間件，例如kafka集羣，mysql數據庫，rabbitmq集羣，mongodb集羣和redis集羣。整個類似於埋點的功能，收集的metrics參數送給grafana圖形化界面展示。在建立這些exporter時，需要注意本機器的端口使用情況和集羣角色的建立，如rabbitmq它有監控着的角色。grafana選擇有很多，建議選擇需要界面展示重點參數的面板。

監控kafka集羣：

下載地址：wget https://github.com/danielqsj/kafka_exporter/releases/download/v1.2.0/kafka_exporter-1.2.0.linux-amd64.tar.gz

在prometheus.yml文件中加入：  

- job_name: 'kafka'

     static_configs:

      - targets: ['kafkaIP或者域名:9308']

        labels:

          instance: kafka@kafkaIP或者域名

例如：        

- job_name: 'kafka'

  static_configs:

    - targets:

      - '192.168.xx.01:9308'

      - '192.168.xx.02:9308'

      - '192.168.xx.03:9308'

 

監控mysql數據庫：      

由於mysqld_exporter需要連接到MySQL，創建用戶並賦予所需的權限

mysql> GRANT REPLICATION CLIENT,PROCESS ON *.* TO 'mysql_monitor'@'localhost' identified by 'mysql_monitor';

mysql> GRANT SELECT ON *.* TO 'mysql_monitor'@'localhost';

 

 

創建.my.cnf配置文件

$ vim /usr/local/mysqld_exporter-0.10.0.linux-amd64/.my.cnf

[client]

user=mysql_monitor

password=mysql_monitor

 

執行腳本：nohup /usr/local/mysqld_exporter-0.10.0.linux-amd64/mysqld_exporter --config.my-cnf="/usr/local/mysqld_exporter-0.10.0.linux-amd64/.my.cnf" &

 

 

監控rabbitmq集羣：

 

https://github.com/kbudde/rabbitmq_exporter/releases

tar zxvf rabbitmq_exporter-0.20.0.linux-amd64.tar.gz

 

cd   rabbitmq_exporter-0.20.0.linux-amd64.tar.gz

 

啓動  RABBIT_USER=aisino RABBIT_PASSWORD=hubeiyfzx2019 OUTPUT_FORMAT=JSON RABBIT_URL=http://192.168.xx.xx:15672 nohup ./rabbitmq_exporter >rabbitmq.log 2>&1 &

 

 

注意RabbitMQ的用戶必須具有monitoring權限，否則會報如下錯誤

 

{"error":"Error while retrieving data from rabbitHost","level":"warning","msg":"retrieving node failed","time":"2019-09-19T20:40:49+08:00"}

{"error":null,"host":"http://192.168.10.24:15672","level":"error","msg":"Error while retrieving data from rabbitHost","statusCode":401,"time":"2019-09-19T20:40:49+08:00"}

{"error":"Error while retrieving data from rabbitHost","level":"warning","msg":"retrieving queue failed","time":"2019-09-19T20:40:49+08:00"}

 

 

RABBIT_USER=guest RABBIT_PASSWORD=guest OUTPUT_FORMAT=JSON PUBLISH_PORT=9099 RABBIT_URL=http://192.168.2.4:15672 nohup  ./rabbitmq_exporter >/dev/null 2>&1 &

 

 

Promthues.yml 添加監控

  - job_name：'RabbitMq'

    static_configs：

      - 目標：['localhost：9099']

 

grafana監控：

 在控制面板加入4371，導入UI模板

 

監控MongoDB集羣

https://github.com/percona/mongodb_exporter      

      1、mongodb的插件安裝：

wget https://github.com/dcu/mongodb_exporter/releases/download/v1.0.0/mongodb_exporter-linux-amd64

tar -zxvf mongodb_exporter-linux-amd64 -C /usr/local/prometheus

cd /usr/local/prometheus

mongod配置副本集啓動

nohup ./mongodb_exporter -mongodb.uri mongodb://192.168.10.xx:27017,192.168.10.xx:27017,192.168.10.xx:27017 > mongodb.log 2>&1 &

mongodb配置單機啓動：

/mongodb_exporter -mongodb.uri mongodb://192.168.152.128:27017

 

監控Redis集羣     

目前監控redis集羣有一個痛點，無法通過一個固定的ip:port來訪問集羣，現在redis的exporter，需要監控每個節點的redis，從而達到監控整個redis集羣。

 nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.24:6380,redis://192.168.10.24:6381 -redis.password aisino &

  - job_name: 'redis_exporter_targets'

    static_configs:

      - targets:

        - redis://first-redis-host:6379

        - redis://second-redis-host:6379

        - redis://third-redis-host:6380

 

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.xx:6380 -redis.password aisino &

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9122 -redis.addr redis://192.168.10.xx:6380 -redis.password aisino > 6380.log 2>&1 &

        nohup ./redis_exporter -web.listen-address=0.0.0.0:9121 -redis.addr redis://192.168.10.xx:6379 -redis.password aisino > 6379.log 2>&1 &