prometheus 對接飛書告警

原創

2022-03-24 13:52

prometheus對接飛書告警

環境

名稱	ip地址	cpu	內存
yz-feishu	10.65.6.91	4c	8G

飛書創建羣組機器人，參考飛書官網

https://www.feishu.cn/hc/zh-CN/articles/360024984973

#備註：自定義機器人要增加ip 白名單限制
#還要複製 webhook 地址，後續要和 PrometheusAlert 中的路徑進行拼接
https://open.feishu.cn/open-apis/bot/v2/hook/23b75d6axxxxxxxxxxxxxxxxxxxxx

prometheus、alertmanager 部署

https://www.cnblogs.com/lixinliang/p/14611521.html

docker 啓動安裝 PrometheusAlert

#百度網盤提取docker 鏡像壓縮包
鏈接：https://pan.baidu.com/s/1T4YL64Ni8ZfEo9oXJrFW6Q 
提取碼：0035

#下載並啓動docker
docker load -i prome.tgz
docker tag feiyu563/prometheus-alert:latest feiyu563/prometheus-alert:feishu
docker run -d   -p 8080:8080   -e PA_LOGIN_USER=prometheusalert -e PA_LOGIN_PASSWORD=prometheusalert -e PA_TITLE=PrometheusAlert -e PA_OPEN_FEISHU=1 feiyu563/prometheus-alert:feishu

#查看docker
docker  ps

配置模板

#訪問 
http://10.65.6.91:8080/

#添加並創建模板，模板內容

{{ $var := .externalURL}}{{ range $k,$v:=.alerts }}
{{if eq $v.status "resolved"}}
**[Prometheus恢復信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警級別：{{$v.labels.severity}}
運維團隊：{{$v.labels.team}}
pod 命名空間：{{$v.labels.namespace}}
pod名稱：{{$v.labels.pod}}
開始時間：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
結束時間：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}} 
故障主機IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**
{{else}}
**[Prometheus告警信息]({{$v.generatorURL}})**
*[{{$v.labels.alertname}}]({{$var}})*
告警級別：{{$v.labels.severity}}
運維團隊：{{$v.labels.team}}
pod 命名空間：{{$v.labels.namespace}}
pod名稱：{{$v.labels.pod}}
開始時間：{{TimeFormat $v.startsAt "2006-01-02 15:04:05"}}
結束時間：{{TimeFormat $v.endsAt "2006-01-02 15:04:05"}}
故障主機IP：{{$v.labels.instance}}
**{{$v.annotations.description}}**
{{end}}
{{ end }}

#複製創建模板的路徑，並和飛書中創建機器人的url 進行拼接

alertmanager 對接PrometheusAlert

#修改 alertmanager 中的 webhook_configs地址，將釘釘信息修改爲 飛書, fsurl=https://open.feishu.cn/open-apis/bot/v2/hook/23b75d6axxxxxxxxxxxxxxxxxxxxx，這個地址爲上面自定義機器人生成的 webhook 地址。
cat alert-config.yaml

apiVersion: v1
data:
  config.yml: |-
    global:
      smtp_smarthost: 'smtp.feishu.cn:465'
      smtp_from: 'xxxx'
      smtp_auth_username: 'xxx'
      smtp_auth_password: 'xxx'
      smtp_require_tls: false
    route:
      group_by: ['alertname', 'cluster']
      group_wait: 30s
      group_interval: 1m
      repeat_interval: 50m
      receiver: default
      routes:
      - receiver: email
        group_wait: 10s
        match:
          team: node
      #- receiver: agcm-platform
      #  group_wait: 10s
      #  match:
      #    user: agcm-platform
    receivers:
    - name: 'default'
      email_configs:
      - to: '[email protected]'
      - to: '[email protected]'
        send_resolved: true
      webhook_configs:
      - url: 'http://10.65.6.91:8080/prometheusalert?type=fs&tpl=prometheus-k8s&fsurl=https://open.feishu.cn/open-apis/bot/v2/hook/23b75d6axxxxxxxxxxxxxxxxxxxxx'
        send_resolved: true
    - name: 'email'
      email_configs:
      - to: '[email protected]'
      - to: '[email protected]'
        send_resolved: true
      webhook_configs:
      - url: 'http://10.65.6.91:8080/prometheusalert?type=fs&tpl=prometheus-k8s&fsurl=https://open.feishu.cn/open-apis/bot/v2/hook/23b75d6axxxxxxxxxxxxxxxxxxxxx'
        send_resolved: true
kind: ConfigMap
metadata:
  name: alert-config
  namespace: prometheus


# 更新alert-config.yaml
kubectl apply -f alert-config.yaml

#查看飛書告警羣，對接飛書告警完成！