Prometheus 监控 Ceph 集群

Prometheus 监控 Ceph 集群

Prometheus Module — Ceph Documentation

Prometheus 模块启用：

[root@node1 ~]# ceph mgr module enable prometheus

[root@node1 ~]# ceph mgr services
{
    "dashboard": "https://node1:8443/",
    "prometheus": "http://node1:9283/"
}
[root@node1 ~]# netstat -tnlp | grep 9283
tcp6       0      0 :::9283                 :::*                    LISTEN      1507703/ceph-mgr

默认情况下，模块将在主机上的所有 IPv4 和 IPv6 地址的端口上接受 HTTP 请求。

ceph config set mgr mgr/prometheus/server_addr 0.0.0.0
ceph config set mgr mgr/prometheus/server_port 9283

安装 prometheus 步骤请移步我之前的博客

掌握Prometheus 监控思科交换机技能，这篇文章就够了！-余生还长，何必慌张-51CTO博客

(一) Prometheus 监控思科交换机---各中间件安装部署

ceph 各节点部署 node_exporter 程序包做监控

下载

wget https://github.com/prometheus/node_exporter/releases/download/v1.0.1/node_exporter-1.0.1.linux-amd64.tar.gz

解压安装：

tar -xf node_exporter-0.17.0.linux-amd64.tar.gz -C /usr/local/
cd /usr/local/
ln -sv node_exporter-0.17.0.linux-amd64/ node_exporter

Unit文件：/usr/lib/systemd/system/node_exporter.service

[Unit]
After=network.target

[Service]
EnvironmentFile=-/etc/sysconfig/node_exporter
User=ceph
ExecStart=/usr/local/node_exporter/node_exporter \
  $NODE_EXPORTER_OPTS
Restart=on-failure
StartLimitInterval=1
RestartSec=3

[Install]
WantedBy=multi-user.target

启动node_exporter：

systemctl daemon-reload
systemctl start node_exporter.service

默认监听9100端口：

curl localhost:9100/metrics

Prometheus.yml 添加如下内容

  - job_name: 'ceph-node'
    static_configs:
      - targets: ["192.168.6.160:9100","192.168.6.161:9100","192.168.6.162:9100"]
        labels:
          cluster: ceph-cluster

  - job_name: 'ceph-cluster'
    honor_labels: true
    static_configs:
      - targets: ['192.168.6.160:9283']
        labels:
          cluster: ceph-cluster

重启 prometheus

systemctl restart prometheus

Prometheus 展示

接入 Grafana 展示

默认 mgr dashboard 就已经安装 grafana 的 json文件，导入至 grafana 内即可

[root@node1 ~]# rpm -ql ceph-grafana-dashboards
/etc/grafana/dashboards/ceph-dashboard
/etc/grafana/dashboards/ceph-dashboard/ceph-cluster.json
/etc/grafana/dashboards/ceph-dashboard/cephfs-overview.json
/etc/grafana/dashboards/ceph-dashboard/host-details.json
/etc/grafana/dashboards/ceph-dashboard/hosts-overview.json
/etc/grafana/dashboards/ceph-dashboard/osd-device-details.json
/etc/grafana/dashboards/ceph-dashboard/osds-overview.json
/etc/grafana/dashboards/ceph-dashboard/pool-detail.json
/etc/grafana/dashboards/ceph-dashboard/pool-overview.json
/etc/grafana/dashboards/ceph-dashboard/radosgw-detail.json
/etc/grafana/dashboards/ceph-dashboard/radosgw-overview.json
/etc/grafana/dashboards/ceph-dashboard/rbd-details.json
/etc/grafana/dashboards/ceph-dashboard/rbd-overview.json
/usr/share/doc/ceph-grafana-dashboards-14.2.16
/usr/share/doc/ceph-grafana-dashboards-14.2.16/README
/usr/share/doc/ceph-grafana-dashboards-14.2.16/README.md

# grafana 官方 Dashboard 展示示例
https://grafana.com/grafana/dashboards/2842