K8S監控方案

kubernetes集羣監控方案有許多種組合對其進行監控，但是在1.12版本後通常選擇prometheus-operator + grafana 進行監控下面我們進行部署監控

1 下載項目

git clone https://github.com/coreos/kube-prometheus.git

是因爲今年prometheus-operator項目搬到了coreos/kube-prometheus

2 排版

cd kube-prometheus/manifests/

mkdir -p operator node-exporter alertmanager grafana kube-state-metrics prometheus serviceMonitor adapter

mv *-serviceMonitor* serviceMonitor/
mv 0prometheus-operator* operator/
mv grafana-* grafana/
mv kube-state-metrics-* kube-state-metrics/
mv alertmanager-* alertmanager/
mv node-exporter-* node-exporter/
mv prometheus-adapter* adapter/
mv prometheus-* prometheus/

3 創建namespace

kubectl apply -f 00namespace-namespace.yaml

4 創建operator

kubectl apply -f operator/

查看pod
kubectl get pods  -n monitoring
等pod創建起來在進行下一步

5 創建其他CRD

kubectl apply -f adapter/
kubectl apply -f alertmanager/
kubectl apply -f node-exporter/
kubectl apply -f kube-state-metrics/
kubectl apply -f grafana/
kubectl apply -f prometheus/
kubectl apply -f serviceMonitor/

查看pod
kubectl get pods  -n monitoring
查看service
kubectl get svc  -n monitoring
發現全部是prometheus和grafana都是可以集羣內部訪問

6 使集羣外可以訪問prometheus 和 grafana

有兩種方案可以實現此功能
1） nodeport方式
2）ingress-nginx方式

我麼這邊爲了測試 選擇nodeport方式

修改prometheus
# vim prometheus/prometheus-service.yaml 

apiVersion: v1
kind: Service
metadata:
  labels:
    prometheus: k8s
  name: prometheus-k8s
  namespace: monitoring
spec:
  ports:
  - name: web
    port: 9090
    targetPort: web
  type:
    NodePort
  selector:
    app: prometheus
    prometheus: k8s
  sessionAffinity: ClientIP
~  
增加nodeport

kubectl apply -f  prometheus/prometheus-service.yaml     


修改grafana
# vim grafana/grafana-service.yaml 

apiVersion: v1
kind: Service
metadata:
  labels:
    app: grafana
  name: grafana
  namespace: monitoring
spec:
  ports:
  - name: http
    port: 3000
    targetPort: http
  type:
    NodePort
  selector:
    app: grafana

kubectl apply -f grafana/grafana-service.yaml

查看sevice主要是查看nodeport端口我們可以訪問


# kubectl get svc  -n monitoring

NAME                    TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
alertmanager-main       ClusterIP   10.99.60.117     <none>        9093/TCP            67m
alertmanager-operated   ClusterIP   None             <none>        9093/TCP,6783/TCP   67m
grafana                 NodePort    10.105.159.125   <none>        3000:31746/TCP      67m
kube-state-metrics      ClusterIP   None             <none>        8443/TCP,9443/TCP   67m
node-exporter           ClusterIP   None             <none>        9100/TCP            67m
prometheus-adapter      ClusterIP   10.102.146.18    <none>        443/TCP             67m
prometheus-k8s          NodePort    10.107.201.125   <none>        9090:32040/TCP      67m
prometheus-operated     ClusterIP   None             <none>        9090/TCP            67m
prometheus-operator     ClusterIP   None 
~                  

7 訪問grafana web頁面

http://masterip:31746

8 未完。。。。（主要是持久化存儲）和無法訪問的pod