SpringBoot整合pushgateway、Alertmanager做監控報警

這裏需要通過pushgateway推送數據

引入依賴

        <!--普羅米修斯依賴-->
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_spring_boot</artifactId>
            <version>0.8.0</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_hotspot</artifactId>
            <version>0.8.0</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_servlet</artifactId>
            <version>0.8.0</version>
        </dependency>
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_pushgateway</artifactId>
            <version>0.8.0</version>
        </dependency>

SpringBoot代碼

這裏我在捕獲到異常後將code和服務名推送到pushgateway中

@ControllerAdvice
public class GlobalExceptionHandler {


    @Value("${spring.application.name}")
    String applicationName;


    @Value("${pushgateway.ip}")
    String pushgatewayIp;


    public static final Counter counterDemo = Counter.build()
            .name("push_way_counter")
            .labelNames("code", "instance")
            .help("user-service異常統計")
            .register();


    /**
     * 傳入code做異常統計
     *
     * @param code
     */
    private void pushData(Integer code) {
        //統計異常
        PushGateway prometheusPush = new PushGateway(pushgatewayIp);
        //指標值增加
        counterDemo.labels(code.toString(), applicationName).inc();
        try {
            prometheusPush.push(counterDemo, "ex-user-service");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }


    /**
     * 此處只是一個例子函數, 用來處理 ResourceConflictException 實際上一個 handler 也可以處理多種 Exception.
     * 也可以把@ResponseStatus 放在 handler 的前面, 這樣多個 Exception 可以用同一個 HTTP 返回 code
     * <p>
     * 不一定每個 Exception 都需要專門處理, 只需要在Exception定義的前面加入@ResponseStatus 定義 HTTP 返回 code
     * 即可.
     *
     * @param err
     * @return
     */
    @ResponseStatus(HttpStatus.CONFLICT)
    @ExceptionHandler(ResourceConflictException.class)
    @ResponseBody
    public CommonResult<Object> handleResourceBusyException(ResourceConflictException err) {
        CommonResult<Object> errorInfo = new CommonResult<>();
        errorInfo.setSuccess(false);

        errorInfo.setCodes(201);
        errorInfo.setMessage(err.getMessage());
        errorInfo.setData(err.getData());

        return errorInfo;
    }

    /**
     * 用戶模塊全局異常信息處理
     *
     * @param err
     * @return
     */
    @ExceptionHandler(UserException.class)
    @ResponseBody
    public CommonResult<Object> handleUserException(HttpServletRequest request, UserException err) {
        pushData(err.getCode());
        CommonResult<Object> errorInfo = new CommonResult<>();
        errorInfo.setSuccess(false);

        errorInfo.setCodes(err.getCode());
        errorInfo.setMessage(err.getMessage());

        errorInfo.setData(err.getData());
        errorInfo.setSuccess(false);
        return errorInfo;
    }

}

定義新的報警規則

vim springboot_rules.yml

 groups:
    - name: springboot-rules
      rules:
      - alert: interface_status
        expr: sum by (code,exported_job) (increase(push_way_counter[5m])) >3
        for: 10s
        labels:
          status: 非常嚴重
        annotations:
          summary: "接口報錯5分鐘內超過3次！！！--{{$labels.code}}"

修改Prometheus.yml

vim prometheus.yml 

rule_files:
    - "alertmanager_rules.yml"
    - "springboot_rules.yml"

重啓！！！

測試如下：