k8s與HPA--通過 Prometheus adaptor 來自定義監控指標

自動擴展是一種根據資源使用情況自動擴展或縮小工作負載的方法。 Kubernetes中的自動縮放有兩個維度：Cluster Autoscaler處理節點擴展操作，Horizontal Pod Autoscaler自動擴展部署或副本集中的pod數量。 Cluster Autoscaling與Horizontal Pod Autoscaler一起用於動態調整計算能力以及系統滿足SLA所需的並行度。雖然Cluster Autoscaler高度依賴託管您的集羣的雲提供商的基礎功能，但HPA可以獨立於您的IaaS / PaaS提供商運營。

Horizontal Pod Autoscaler功能最初是在Kubernetes v1.1中引入的，並且從那時起已經發展了很多。 HPA縮放容器的版本1基於觀察到的CPU利用率，後來基於內存使用情況。在Kubernetes 1.6中，引入了一個新的API Custom Metrics API，使HPA能夠訪問任意指標。 Kubernetes 1.7引入了聚合層，允許第三方應用程序通過將自己註冊爲API附加組件來擴展Kubernetes API。 Custom Metrics API和聚合層使Prometheus等監控系統可以向HPA控制器公開特定於應用程序的指標。

Horizontal Pod Autoscaler實現爲一個控制循環，定期查詢Resource Metrics API以獲取CPU /內存等核心指標和針對特定應用程序指標的Custom Metrics API。

以下是爲Kubernetes 1.9或更高版本配置HPA v2的分步指南。您將安裝提供核心指標的Metrics Server附加組件，然後您將使用演示應用程序根據CPU和內存使用情況展示pod自動擴展。在本指南的第二部分中，您將部署Prometheus和自定義API服務器。您將使用聚合器層註冊自定義API服務器，然後使用演示應用程序提供的自定義指標配置HPA。

在開始之前，您需要安裝Go 1.8或更高版本並在GOPATH中克隆k8s-prom-hpa repo。

cd $GOPATH
git clone https://github.com/stefanprodan/k8s-prom-hpa

部署 Metrics Server

kubernetes Metrics Server是資源使用數據的集羣範圍聚合器，是Heapster的後繼者。度量服務器通過彙集來自kubernetes.summary_api的數據來收集節點和pod的CPU和內存使用情況。摘要API是一種內存高效的API，用於將數據從Kubelet / cAdvisor傳遞到度量服務器。

在HPA的第一個版本中，您需要Heapster來提供CPU和內存指標，在HPA v2和Kubernetes 1.8中，只有在啓用horizontal-pod-autoscaler-use-rest-clients時才需要指標服務器。默認情況下，Kubernetes 1.9中啓用了HPA rest客戶端。 GKE 1.9附帶預安裝的Metrics Server。

在kube-system命名空間中部署Metrics Server：

kubectl create -f ./metrics-server

一分鐘後，度量服務器開始報告節點和pod的CPU和內存使用情況。

查看nodes metrics：

kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" | jq .

結果如下：

{
  "kind": "NodeMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
  },
  "items": [
    {
      "metadata": {
        "name": "ip-10-1-50-61.ec2.internal",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/ip-10-1-50-61.ec2.internal",
        "creationTimestamp": "2019-02-13T08:34:05Z"
      },
      "timestamp": "2019-02-13T08:33:38Z",
      "window": "30s",
      "usage": {
        "cpu": "78322168n",
        "memory": "563180Ki"
      }
    },
    {
      "metadata": {
        "name": "ip-10-1-57-40.ec2.internal",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/ip-10-1-57-40.ec2.internal",
        "creationTimestamp": "2019-02-13T08:34:05Z"
      },
      "timestamp": "2019-02-13T08:33:42Z",
      "window": "30s",
      "usage": {
        "cpu": "48926263n",
        "memory": "554472Ki"
      }
    },
    {
      "metadata": {
        "name": "ip-10-1-62-29.ec2.internal",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/nodes/ip-10-1-62-29.ec2.internal",
        "creationTimestamp": "2019-02-13T08:34:05Z"
      },
      "timestamp": "2019-02-13T08:33:36Z",
      "window": "30s",
      "usage": {
        "cpu": "36700681n",
        "memory": "326088Ki"
      }
    }
  ]
}

查看pods metrics:

kubectl get --raw "/apis/metrics.k8s.io/v1beta1/pods" | jq .

結果如下：

{
  "kind": "PodMetricsList",
  "apiVersion": "metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/metrics.k8s.io/v1beta1/pods"
  },
  "items": [
    {
      "metadata": {
        "name": "kube-proxy-77nt2",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-proxy-77nt2",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:00Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-proxy",
          "usage": {
            "cpu": "2370555n",
            "memory": "13184Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "cluster-autoscaler-n2xsl",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/cluster-autoscaler-n2xsl",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:12Z",
      "window": "30s",
      "containers": [
        {
          "name": "cluster-autoscaler",
          "usage": {
            "cpu": "1477997n",
            "memory": "54584Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "core-dns-autoscaler-b4785d4d7-j64xd",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/core-dns-autoscaler-b4785d4d7-j64xd",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:08Z",
      "window": "30s",
      "containers": [
        {
          "name": "autoscaler",
          "usage": {
            "cpu": "191293n",
            "memory": "7956Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "spot-interrupt-handler-8t2xk",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/spot-interrupt-handler-8t2xk",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:04Z",
      "window": "30s",
      "containers": [
        {
          "name": "spot-interrupt-handler",
          "usage": {
            "cpu": "844907n",
            "memory": "4608Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-proxy-t5kqm",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-proxy-t5kqm",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:08Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-proxy",
          "usage": {
            "cpu": "1194766n",
            "memory": "12204Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube-proxy-zxmqb",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube-proxy-zxmqb",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:06Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube-proxy",
          "usage": {
            "cpu": "3021117n",
            "memory": "13628Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "aws-node-rcz5c",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/aws-node-rcz5c",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:15Z",
      "window": "30s",
      "containers": [
        {
          "name": "aws-node",
          "usage": {
            "cpu": "1217989n",
            "memory": "24976Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "aws-node-z2qxs",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/aws-node-z2qxs",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:15Z",
      "window": "30s",
      "containers": [
        {
          "name": "aws-node",
          "usage": {
            "cpu": "1025780n",
            "memory": "46424Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "php-apache-899d75b96-8ppk4",
        "namespace": "default",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/default/pods/php-apache-899d75b96-8ppk4",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:08Z",
      "window": "30s",
      "containers": [
        {
          "name": "php-apache",
          "usage": {
            "cpu": "24612n",
            "memory": "27556Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "load-generator-779c5f458c-9sglg",
        "namespace": "default",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/default/pods/load-generator-779c5f458c-9sglg",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:34:56Z",
      "window": "30s",
      "containers": [
        {
          "name": "load-generator",
          "usage": {
            "cpu": "0",
            "memory": "336Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "aws-node-v9jxs",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/aws-node-v9jxs",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:00Z",
      "window": "30s",
      "containers": [
        {
          "name": "aws-node",
          "usage": {
            "cpu": "1303458n",
            "memory": "28020Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube2iam-m2ktt",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube2iam-m2ktt",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:11Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube2iam",
          "usage": {
            "cpu": "1328864n",
            "memory": "9724Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube2iam-w9cqf",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube2iam-w9cqf",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:03Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube2iam",
          "usage": {
            "cpu": "1294379n",
            "memory": "8812Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "custom-metrics-apiserver-657644489c-pk8rb",
        "namespace": "monitoring",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods/custom-metrics-apiserver-657644489c-pk8rb",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:04Z",
      "window": "30s",
      "containers": [
        {
          "name": "custom-metrics-apiserver",
          "usage": {
            "cpu": "22409370n",
            "memory": "42468Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "kube2iam-qghgt",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/kube2iam-qghgt",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:11Z",
      "window": "30s",
      "containers": [
        {
          "name": "kube2iam",
          "usage": {
            "cpu": "2078992n",
            "memory": "16356Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "spot-interrupt-handler-ps745",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/spot-interrupt-handler-ps745",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:10Z",
      "window": "30s",
      "containers": [
        {
          "name": "spot-interrupt-handler",
          "usage": {
            "cpu": "611566n",
            "memory": "4336Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "coredns-68fb7946fb-2xnpp",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/coredns-68fb7946fb-2xnpp",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:12Z",
      "window": "30s",
      "containers": [
        {
          "name": "coredns",
          "usage": {
            "cpu": "1610381n",
            "memory": "10480Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "coredns-68fb7946fb-9ctjf",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/coredns-68fb7946fb-9ctjf",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:13Z",
      "window": "30s",
      "containers": [
        {
          "name": "coredns",
          "usage": {
            "cpu": "1418850n",
            "memory": "9852Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "prometheus-7d4f6d4454-v4fnd",
        "namespace": "monitoring",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/monitoring/pods/prometheus-7d4f6d4454-v4fnd",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:00Z",
      "window": "30s",
      "containers": [
        {
          "name": "prometheus",
          "usage": {
            "cpu": "17951807n",
            "memory": "202316Ki"
          }
        }
      ]
    },
    {
      "metadata": {
        "name": "metrics-server-7cdd54ccb4-k2x7m",
        "namespace": "kube-system",
        "selfLink": "/apis/metrics.k8s.io/v1beta1/namespaces/kube-system/pods/metrics-server-7cdd54ccb4-k2x7m",
        "creationTimestamp": "2019-02-13T08:35:19Z"
      },
      "timestamp": "2019-02-13T08:35:04Z",
      "window": "30s",
      "containers": [
        {
          "name": "metrics-server-nanny",
          "usage": {
            "cpu": "144656n",
            "memory": "5716Ki"
          }
        },
        {
          "name": "metrics-server",
          "usage": {
            "cpu": "568327n",
            "memory": "16268Ki"
          }
        }
      ]
    }
  ]
}

基於CPU和內存使用情況的Auto Scaling

您將使用基於Golang的小型Web應用程序來測試Horizontal Pod Autoscaler（HPA）。

將podinfo部署到默認命名空間：

kubectl create -f ./podinfo/podinfo-svc.yaml,./podinfo/podinfo-dep.yaml

使用NodePort服務訪問podinfo，地址爲http：// <K8S_PUBLIC_IP>：31198。

接下來定義一個至少維護兩個副本的HPA，如果CPU平均值超過80％或內存超過200Mi，則最多可擴展到10個：

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: podinfo
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: podinfo
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      targetAverageUtilization: 80
  - type: Resource
    resource:
      name: memory
      targetAverageValue: 200Mi

創建這個hpa:

kubectl create -f ./podinfo/podinfo-hpa.yaml

幾秒鐘後，HPA控制器聯繫度量服務器，然後獲取CPU和內存使用情況：

kubectl get hpa

NAME      REFERENCE            TARGETS                      MINPODS   MAXPODS   REPLICAS   AGE
podinfo   Deployment/podinfo   2826240 / 200Mi, 15% / 80%   2         10        2          5m

爲了增加CPU使用率，請使用rakyll / hey運行負載測試：

#install hey
go get -u github.com/rakyll/hey

#do 10K requests
hey -n 10000 -q 10 -c 5 http://<K8S_PUBLIC_IP>:31198/

您可以使用以下方式監控HPA事件：

$ kubectl describe hpa

Events:
  Type    Reason             Age   From                       Message
  ----    ------             ----  ----                       -------
  Normal  SuccessfulRescale  7m    horizontal-pod-autoscaler  New size: 4; reason: cpu resource utilization (percentage of request) above target
  Normal  SuccessfulRescale  3m    horizontal-pod-autoscaler  New size: 8; reason: cpu resource utilization (percentage of request) above target

暫時刪除podinfo。稍後將在本教程中再次部署它：

kubectl delete -f ./podinfo/podinfo-hpa.yaml,./podinfo/podinfo-dep.yaml,./podinfo/podinfo-svc.yaml

部署 Custom Metrics Server

要根據自定義指標進行擴展，您需要擁有兩個組件。一個組件，用於從應用程序收集指標並將其存儲在Prometheus時間序列數據庫中。第二個組件使用collect（k8s-prometheus-adapter）提供的指標擴展了Kubernetes自定義指標API。

您將在專用命名空間中部署Prometheus和適配器。

創建monitoring命名空間：

kubectl create -f ./namespaces.yaml

在monitoring命名空間中部署Prometheus v2：

kubectl create -f ./prometheus

生成Prometheus適配器所需的TLS證書：

make certs

生成以下幾個文件：

# ls output
apiserver.csr  apiserver-key.pem  apiserver.pem

部署Prometheus自定義指標API適配器：

kubectl create -f ./custom-metrics-api

列出Prometheus提供的自定義指標：

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq .

獲取monitoring命名空間中所有pod的FS使用情況：

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/monitoring/pods/*/fs_usage_bytes" | jq .

查詢結果如下：


{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/monitoring/pods/%2A/fs_usage_bytes"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "monitoring",
        "name": "custom-metrics-apiserver-657644489c-pk8rb",
        "apiVersion": "/v1"
      },
      "metricName": "fs_usage_bytes",
      "timestamp": "2019-02-13T08:52:30Z",
      "value": "94253056"
    },
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "monitoring",
        "name": "prometheus-7d4f6d4454-v4fnd",
        "apiVersion": "/v1"
      },
      "metricName": "fs_usage_bytes",
      "timestamp": "2019-02-13T08:52:30Z",
      "value": "24576"
    }
  ]
}

基於custom metrics 自動伸縮

在默認命名空間中創建podinfo NodePort服務和部署：

kubectl create -f ./podinfo/podinfo-svc.yaml,./podinfo/podinfo-dep.yaml

podinfo應用程序公開名爲http_requests_total的自定義指標。 Prometheus適配器刪除_total後綴並將度量標記爲計數器度量標準。

從自定義指標API獲取每秒的總請求數：

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/*/http_requests" | jq .

{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/default/pods/%2A/http_requests"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "default",
        "name": "podinfo-6b86c8ccc9-kv5g9",
        "apiVersion": "/__internal"
      },
      "metricName": "http_requests",
      "timestamp": "2018-01-10T16:49:07Z",
      "value": "901m"
    },
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "default",
        "name": "podinfo-6b86c8ccc9-nm7bl",
        "apiVersion": "/__internal"
      },
      "metricName": "http_requests",
      "timestamp": "2018-01-10T16:49:07Z",
      "value": "898m"
    }
  ]
}

建一個HPA，如果請求數超過每秒10個，將擴展podinfo部署：

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: podinfo
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: podinfo
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Pods
    pods:
      metricName: http_requests
      targetAverageValue: 10

在默認命名空間中部署podinfo HPA：

kubectl create -f ./podinfo/podinfo-hpa-custom.yaml

幾秒鐘後，HPA從指標API獲取http_requests值：

kubectl get hpa

NAME      REFERENCE            TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
podinfo   Deployment/podinfo   899m / 10   2         10        2          1m

在podinfo服務上應用一些負載，每秒25個請求：

#install hey
go get -u github.com/rakyll/hey

#do 10K requests rate limited at 25 QPS
hey -n 10000 -q 5 -c 5 http://<K8S-IP>:31198/healthz

幾分鐘後，HPA開始擴展部署：

kubectl describe hpa

Name:                       podinfo
Namespace:                  default
Reference:                  Deployment/podinfo
Metrics:                    ( current / target )
  "http_requests" on pods:  9059m / 10
Min replicas:               2
Max replicas:               10

Events:
  Type    Reason             Age   From                       Message
  ----    ------             ----  ----                       -------
  Normal  SuccessfulRescale  2m    horizontal-pod-autoscaler  New size: 3; reason: pods metric http_requests above target

按照當前的每秒請求速率，部署永遠不會達到10個pod的最大值。三個複製品足以使每個吊艙的RPS保持在10以下。

負載測試完成後，HPA會將部署縮到其初始副本：

Events:
  Type    Reason             Age   From                       Message
  ----    ------             ----  ----                       -------
  Normal  SuccessfulRescale  5m    horizontal-pod-autoscaler  New size: 3; reason: pods metric http_requests above target
  Normal  SuccessfulRescale  21s   horizontal-pod-autoscaler  New size: 2; reason: All metrics below target

您可能已經注意到自動縮放器不會立即對使用峯值做出反應。默認情況下，度量標準同步每30秒發生一次，只有在最後3-5分鐘內沒有重新縮放時才能進行擴展/縮小。通過這種方式，HPA可以防止快速執行衝突的決策，併爲Cluster Autoscaler提供時間。

結論

並非所有系統都可以通過單獨依賴CPU /內存使用指標來滿足其SLA，大多數Web和移動後端需要基於每秒請求進行自動擴展以處理任何流量突發。對於ETL應用程序，可以通過作業隊列長度超過某個閾值等來觸發自動縮放。通過使用Prometheus檢測應用程序並公開正確的自動縮放指標，您可以對應用程序進行微調，以更好地處理突發並確保高可用性。

k8s與HPA--通過 Prometheus adaptor 來自定義監控指標

部署 Metrics Server

基於CPU和內存使用情況的Auto Scaling

部署 Custom Metrics Server

基於custom metrics 自動伸縮

結論

《Python進階》學習筆記

Leetcode 3161. 物塊放置查詢

一個docker容器暴露多個端口

leetcode 60 排列序列

微服務實踐之使用 Visual Studio 2022 調試Dapr 應用程序

wpf附加屬性理解 WPF附加屬性

Docker私有倉庫搭建(Harbor)

使用Seata實現分佈式事物帶來的分支事務回滾異常問題

k8s三部曲(2)——kubernetes(TSL)非主節點安裝dashboard

基於k8s的微服務一鍵部署解決方案

k8s三部曲(1)——kubernetes 完整集羣部署(二進制安裝)

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結