基於 Kubernetes v1.14.0 之 kube-state-metrics 部署

1、部署準備

說明：所有的容器組都運行在monitoring 命名空間
本文參考https://github.com/coreos/kube-prometheus
由於官方維護的版本在現有的部署環境出現問題所以下面做了一些修改及變更不影響整體效果
 kube-state-metrics 不使用官方yaml 

2、kube-state-metrics 服務相關yaml準備

2.1、創建kube-state-metrics-rbac

說明：新增storage，ingresses 數據收集

vi kube-state-metrics-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-state-metrics
  namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - replicasets
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectacce***eviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups: 
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs: 
  - list
  - watch
- apiGroups: 
  - storage.k8s.io
  resources:
  - storageclasses
  verbs: 
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: kube-state-metrics
  namespace: monitoring
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - extensions
  resourceNames:
  - kube-state-metrics
  resources:
  - deployments
  verbs:
  - get
  - update
- apiGroups:
  - apps
  resourceNames:
  - kube-state-metrics
  resources:
  - deployments
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: kube-state-metrics
  namespace: monitoring
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics

2.2、創建 kube-state-metrics-service

vi kube-state-metrics-service.yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    prometheus.io/scrape: 'true'
  name: kube-state-metrics
  namespace: monitoring
  labels:
    k8s-app: kube-state-metrics
spec:
  ports:
  - name: http-main
    port: 8080
    protocol: TCP
  - name: http-self
    port: 8081
    protocol: TCP
  selector:
    app: kube-state-metrics

2.3、創建kube-state-metrics-serviceMonitor

vi kube-state-metrics-serviceMonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: kube-state-metrics
  name: kube-state-metrics
  namespace: monitoring
spec:
  endpoints:
  - honorLabels: true
    interval: 30s
    port: http-main
    scheme: http
    scrapeTimeout: 30s
  - interval: 30s
    port: http-self
    scheme: http
  jobLabel: k8s-app
  selector:
    matchLabels:
      k8s-app: kube-state-metrics

2.4、創建kube-state-metrics-deployment

vi kube-state-metrics-deployment.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: nanny-config
  namespace: default
data:
  NannyConfiguration: |-
    apiVersion: nannyconfig/v1alpha1
    kind: NannyConfiguration
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  labels:
    app: kube-state-metrics
  name: kube-state-metrics
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-state-metrics
  template:
    metadata:
      labels:
        app: kube-state-metrics
    spec:
      serviceAccountName: kube-state-metrics
      containers:
      - name: kube-state-metrics
        image: googlecontainer/kube-state-metrics:v1.6.0
        ports:
        - name: http-metrics
          containerPort: 8080
        - name: telemetry
          containerPort: 8081
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 10
          timeoutSeconds: 10
      - name: addon-resizer
        image: juestnow/addon-resizer:1.8.5
        resources:
          limits:
            cpu: 100m
            memory: 30Mi
          requests:
            cpu: 100m
            memory: 30Mi
        env:
          - name: MY_POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: MY_POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        command:
          - /pod_nanny
          - --container=kube-state-metrics
          - --cpu=100m
          - --extra-cpu=1m
          - --memory=200Mi
          - --extra-memory=2Mi
          - --threshold=5
          - --deployment=kube-state-metrics
      nodeSelector:
        beta.kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics

3、執行yaml 創建kube-state-metrics 服務

kubectl apply -f .

查看kube-state-metrics 狀態

[root@jenkins state-metrics]# kubectl get pod -n monitoring -o wide | grep kube-state-metrics
kube-state-metrics-684969d97d-vv8lt       2/2     Running   0          3d20h   10.65.2.90      nginx-1   <none>           <none>
[root@jenkins state-metrics]# kubectl get service -n monitoring  | grep kube-state-metrics            
kube-state-metrics        ClusterIP   10.64.219.124   <none>        8080/TCP,8081/TCP   56d
訪問kube-state-metrics
http://10.64.219.124:8080/metrics
查看prometheus監控是否有對應的kube-state-metrics  標籤

打開grafana 查看 kubernetes開頭dashboards 如果有數據證明正常

下一篇： Kubernetes 生產環境安裝部署 基於 Kubernetes v1.14.0 之 node-exporter與process-exporter 部署