k8s，盤他！k8s的安全機制與RBAC使用方法

原創 深海有座未亡的城！ 2020-05-22 13:31

文章目錄

前言

一：k8s安全機制

安全框架、

傳輸安全、認證，授權，准入控制

使用rbac授權

1.1：kubernetes安全框架

mark

  • 安全框架的流程

流程：kubectl先請求api資源，然後是過三關，第一關是認證（Authentication），第二關是授權（Authorization），第三關是准入控制（Admission Control），只有通過這三關纔可能會被k8s創建資源。

K8S安全控制框架主要由下面3個階段進行控制，每一個階段都支持插件方式，通過API Server配置來啓用插件。

普通用戶若要安全訪問集羣API Server，往往需要證書、Token或者用戶名+密碼；Pod訪問，需要ServiceAccount

  • apiserver使用的是token認證

    [root@master ~]# ps aux |grep apiserver
root      12636  3.0 23.6 420460 235620 ?       Ssl  5月17 156:09 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root      22849  0.0  0.0 112728   976 pts/0    S+   10:20   0:00 grep --color=auto apiserver
'//其中能夠查詢到token認證等信息'

  • 查看ServiceAccount，可以通過ServiceAccount在pod中去訪問apiserver

    [root@master ~]# kubectl get sa
NAME      SECRETS   AGE
default   1         21d
'//Service Account它並不是給kubernetes集羣的用戶使用的，而是給pod裏面的進程使用的，它爲pod提供必要的身份認證。'

  • 傳輸安全方面：8080用於內部通訊，6443是提供給外部訪問的端口

    [root@master ~]# netstat -ntap |grep 8080 |grep LISTEN	'//默認8080監聽本地（是通過master及其他組件連接使用）'
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      12636/kube-apiserve 
[root@master ~]# netstat -ntap |grep 6443 |grep LISTEN	'//對外提供服務端口是6443'
tcp        0      0 192.168.233.131:6443    0.0.0.0:*               LISTEN      12636/kube-apiserve

1.2：第一關：Authentication認證

三種客戶端身份認證：

1、HTTPS 證書認證：基於CA證書籤名的數字證書認證

2、HTTP Token認證：通過一個Token來識別用戶（生產環境中使用廣泛）

3、HTTP Base認證：用戶名+密碼的方式認證

  • 1、HTTPS證書認證

    [root@master ~]# cat k8s/k8s-cert/k8s-cert.sh
cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF

cat > ca-csr.json <<EOF
{
    "CN": "kubernetes",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing",
      	    "O": "k8s",
            "OU": "System"
        }
    ]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

#-----------------------

cat > server-csr.json <<EOF
{
    "CN": "kubernetes",
    "hosts": [
      "10.0.0.1",
      "127.0.0.1",
      "192.168.233.131",	'//此處直接指定了負載均衡和master的節點'
      "192.168.233.130",
      "192.168.233.100",
      "192.168.233.128",
      "192.168.233.129",
      "kubernetes",
      "kubernetes.default",
      "kubernetes.default.svc",
      "kubernetes.default.svc.cluster",
      "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing",
            "O": "k8s",
            "OU": "System"
        }
    ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server

#-----------------------

cat > admin-csr.json <<EOF
{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin

#-----------------------

cat > kube-proxy-csr.json <<EOF
{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy

  • 2、httpd的token認證

    [root@master ~]# cat /opt/kubernetes/cfg/token.csv 
7ea8f86b157225fd4b9273765e88a3ca,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

1.3：第二關：Authorization授權

  • RBAC（Role-Based Access Control，基於角色的訪問控制）：負責完成授權（Authorization）工作，允許通過Kubernetes API動態配置策略。

  • 使用RBAC授權

  • mark

    角色：

    1、Role：授權特定命名空間的訪問權限

    2、ClusterRole：授權所有命名空間的訪問權限

    角色綁定

    1、RoleBinding：將角色綁定到主體（即subject）

    2、ClusterRoleBinding：將集羣角色綁定到主體

    主體（subject）

    1、User：用戶

    2、Group：用戶組

    3、ServiceAccount：服務賬號

1.3.1：RBAC使用測試

  • 1、創建名稱空間dabao

    [root@master ~]# kubectl create ns dabao
namespace/dabao created
[root@master ~]# kubectl get ns
NAME          STATUS   AGE
dabao         Active   7s
default       Active   21d
kube-public   Active   21d
kube-system   Active   21d

  • 2、創建測試pod

    [root@master ~]# kubectl run nginx01  --image=nginx -n dabao
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx-test created
[root@master ~]# kubectl get pod -n dabao
NAME                         READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-p4xbh     1/1     Running   0          15m

  • 3、擴容成3個副本，使用scale命令

    [root@master ~]# kubectl scale deploy/nginx01 --replicas=3 -n dabao
deployment.extensions/nginx01 scaled
[root@master ~]# kubectl get pod -n dabao
NAME                         READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-98jc4     1/1     Running   0          31s
nginx01-77c44977fd-nt424     1/1     Running   0          31s
nginx01-77c44977fd-p4xbh     1/1     Running   0          17m

  • 3、創建角色

    [root@master ~]# vim rbac-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: dabao
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]	'//創建角色只有pod資源的操作權限'
  verbs: ["get", "watch", "list"]	'//只有這些操作可以使用'
[root@master ~]# kubectl apply -f rbac-role.yaml 
role.rbac.authorization.k8s.io/pod-reader created
[root@master ~]# kubectl get role -n dabao
NAME         AGE
pod-reader   8s
[root@master ~]#

  • 4、角色綁定

    [root@master ~]# vim rbac-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: dabao
subjects:
- kind: User
  name: zhangsan
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role   
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master ~]# kubectl apply -f rbac-rolebinding.yaml 
rolebinding.rbac.authorization.k8s.io/read-pods created
[root@master ~]# kubectl get role,rolebinding -n dabao
NAME                                        AGE
role.rbac.authorization.k8s.io/pod-reader   2m14s

NAME                                              AGE
rolebinding.rbac.authorization.k8s.io/read-pods   13s

  • 5、創建兩個rbac的文件

    [root@master ~]# mkdir zhangsan
[root@master ~]# cd zhangsan/
[root@master zhangsan]# vim rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: jane
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io




apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: default

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: default
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master zhangsan]#  vim rbac-user.sh
cat > zhangsan-csr.json <<EOF
{
  "CN": "zhangsan",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes zhangsan-csr.json | cfssljson -bare zhangsan 

kubectl config set-cluster kubernetes \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://192.168.233.100:6443 \	'//修改爲負載均衡VIP地址'
  --kubeconfig=zhangsan-kubeconfig
  
kubectl config set-credentials zhangsan \
  --client-key=zhangsan-key.pem \
  --client-certificate=zhangsan.pem \
  --embed-certs=true \
  --kubeconfig=zhangsan-kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=zhangsan \
  --kubeconfig=zhangsan-kubeconfig

kubectl config use-context default --kubeconfig=zhangsan-kubeconfig

  • 6、拷貝證書到張三目錄並安裝格式化工具

    [root@master zhangsan]# cp /root/k8s/k8s-cert/ca* ./
[root@master zhangsan]# ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  rbac-user.sh  rbac.yaml
[root@master zhangsan]# yum install dos2unix -y
[root@master zhangsan]# dos2unix rbac-user.sh 	'//執行格式化'
dos2unix: converting file rbac-user.sh to Unix format ...
[root@master zhangsan]# bash rbac-user.sh 	'//運行腳本'

  • 7、使用zhangsan-kubeconfig訪問資源測試

    [root@master zhangsan]# cat zhangsan-kubeconfig
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get pod -n dabao
NAME                       READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-98jc4   1/1     Running   0          23m
nginx01-77c44977fd-nt424   1/1     Running   0          23m
nginx01-77c44977fd-p4xbh   1/1     Running   0          40m
'//但是訪問不到除了pod的其他資源'
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc -n dabao	'//service無法訪問'
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "dabao"
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc	'//默認名稱空間也無法訪問'
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "default"

  • UI訪問控制

    [root@master zhangsan]# kubectl get svc -n kube-system
NAME                   TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   NodePort   10.0.0.139   <none>        443:30005/TCP   13d
[root@master zhangsan]# kubectl get pod -n kube-system -o wide
NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE
kubernetes-dashboard-7dffbccd68-58qms   1/1     Running   5          13d   172.17.78.3   192.168.233.132   <none>

'//接下來訪問之前搭建的UI界面'

    mark

  • 查看令牌

    [root@master zhangsan]# kubectl get secret -n kube-system	'//這是管理員的令牌'
NAME                               TYPE                                  DATA   AGE
dashboard-admin-token-zwktc        kubernetes.io/service-account-token   3      13d
default-token-cnnbv                kubernetes.io/service-account-token   3      21d
kubernetes-dashboard-certs         Opaque                                11     13d
kubernetes-dashboard-key-holder    Opaque                                2      13d
kubernetes-dashboard-token-qgppd   kubernetes.io/service-account-token   3      13d
[root@master zhangsan]# vim sa.yaml	'//編輯認證yaml文件'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: dabao

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: dabao
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master zhangsan]# kubectl apply -f sa.yaml 
serviceaccount/pod-reader created
rolebinding.rbac.authorization.k8s.io/sa-read-pods created
[root@master zhangsan]# kubectl get sa -n dabao
NAME         SECRETS   AGE
default      1         60m
pod-reader   1         11s
[root@master zhangsan]# kubectl get secret -n dabao
NAME                     TYPE                                  DATA   AGE
default-token-5lkxq      kubernetes.io/service-account-token   3      61m
pod-reader-token-l49zb   kubernetes.io/service-account-token   3      76s
[root@master zhangsan]# kubectl describe secret pod-reader-token-l49zb -n dabao
Name:         pod-reader-token-l49zb
Namespace:    dabao
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: pod-reader
              kubernetes.io/service-account.uid: b262ec71-9b16-11ea-8c4f-000c294b2dd3

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1359 bytes
namespace:  5 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkYWJhbyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJwb2QtcmVhZGVyLXRva2VuLWw0OXpiIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InBvZC1yZWFkZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMjYyZWM3MS05YjE2LTExZWEtOGM0Zi0wMDBjMjk0YjJkZDMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGFiYW86cG9kLXJlYWRlciJ9.vUdzOLAxq4GE6sRGmS4RkK6_s-NeF7mAeNnuZf9zkacYXZSxxWR6yz7h2-xpSPFJJIvZ1IAmP_G0XO4pazlS_DkLn_cNzLwAT7moOp8CxalATySLY-KtXxbRslwpcyLfyxaZ-PSEiI3fKt5f66C0eL3aoFYIM-xukVQNx_UJE2vsmu93WTuUb8XQjMVGUkQW-p7Mw_2f2wCyGbk_Y__LzXv3dRj0df7EoANHPiadYbntO-_k04LNSfrA1cHRZKBgIddO5tF8olw6rTs99IkSWfMUoF4-qCBDROOBf2h8tSMgz3jrOlhAbvq8kuvR4zjBtwUfI4l0c2IS00HZFW3K3g
[root@master zhangsan]#

  • 7、使用令牌登陸UI界面

    mark

1.4：第三關：准入控制Admission Control

Adminssion Control實際上是一個准入控制器插件列表，發送到API Server的請求都需要經過這個列表中的每個准入控制器 插件的檢查，檢查不通過，則拒絕請求。

  • 查看進程信息

    [root@master zhangsan]# ps aux |grep apiserver
root      12636  3.0 20.9 420460 208584 ?       Ssl  5月17 160:57 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction '//此行是准入控制的插件信息'--authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root      23615  0.0  0.0 112728   972 pts/0    S+   12:18   0:00 grep --color=auto apiserver

NamespaceLifecycle： 命令空間回收

LimitRanger：配額管理

ServiceAccount： 每個pod中導入方便訪問apiserver

ResourceQuota： 基於命名空間的高級配額管理

NodeRestriction： Node加入到k8s羣集中以最小權限運行

官網推薦的插件：

1.11版本以上推薦使用的插件：

–enable-admission-plugins= \ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota

如有疑問可評論區交流！

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

k8s，盤他！k8s的五種控制器類型解析

深海有座未亡的城！ 2020-05-21 05:12:29

k8s，盤他！secret和configmap的配置管理

深海有座未亡的城！ 2020-05-21 05:12:18

k8s，盤他！pod資源工作流程與調度約束

深海有座未亡的城！ 2020-05-16 05:21:36

k8s，盤他！pod高級管理（資源控制、重啓策略與探針）

深海有座未亡的城！ 2020-05-16 05:21:36

k8s，盤他！pod容器與鏡像管理和k8s私有倉庫harbor搭建

深海有座未亡的城！ 2020-05-12 04:00:15

k8s，盤他！YAML文件概論與演示！

深海有座未亡的城！ 2020-05-10 21:00:55

k8s，盤他！kubectl常規命令操作集錦

深海有座未亡的城！ 2020-05-10 21:00:55

k8s，盤他！k8s多節點儀表盤（web界面）部署與谷歌瀏覽器訪問k8s儀表盤問題解決！

深海有座未亡的城！ 2020-05-08 18:22:28

k8s，盤他！k8s多master節點使用二進制部署實操

深海有座未亡的城！ 2020-05-01 13:42:48

k8s，盤他！使用單節點二進制方式部署k8s集羣

深海有座未亡的城！ 2020-05-01 13:42:48

K8S學習003--使用kubeadm部署安裝集羣

汤政大人 2020-02-20 21:32:00

K8S學習002--Pod概念與網絡通訊方式

汤政大人 2020-02-20 21:32:00

【Kubernetes（K8S）】學習記錄一：概念總結

文章目錄前言-背景1、什麼是Kubernetes？2、Kubernetes組件2.1、集羣2.2、分佈式2.3、etcd2.4、Node2.5、Pod2.6、Controller Manager2.7、kube-proxy2.8、

3L_csdn 2020-06-20 07:25:08

【Kubernetes（K8S）】學習記錄三：rancher清理

什麼時候會用到清理節點？ 將節點添加到集羣時後，會創建容器、虛擬網絡接口等資源和證書、配置文件。 從集羣中正常刪除節點時(如果處於Active狀態)，將自動清除這些資源，並且只需重新啓動節點即可。 當節點無法訪問且無法使用自動清理

3L_csdn 2020-06-20 07:24:56

k8s，盤他！k8s的五種控制器類型解析

深海有座未亡的城！ 2020-05-21 05:12:29