PersistentVolume持久卷

PersistentVolume(持久卷,簡稱PV)是集羣內,由管理員提供的網絡存儲的一部分。就像集羣中的節點一樣,PV也是集羣中的一種資源。它也像Volume一樣,是一種volume插件,但是它的生命週期卻是和使用它的Pod相互獨立的。PV這個API象,捕獲了諸如NFS、ISCSI、或其他雲存儲系統的實現細節。
PersistentVolumeClaim(持久卷聲明,簡稱PVC)是用戶的一種存儲請求。它和Pod類似,Pod消耗Node資源,而PVC消耗PV資源。Pod能夠請求特定的資源(如CPU和內存)。PVC能夠請求指定的大小和訪問的模式(可以被映射爲一次讀寫或者多次只讀)。
有兩種PV提供的方式:靜態和動態。
- 靜態PV:集羣管理員創建多個PV,它們攜帶着真實存儲的詳細信息,這些存儲對於集羣用
  戶是可用的。它們存在於Kubernetes API中,並可用於存儲使用。
- 動態PV:當管理員創建的靜態PV都不匹配用戶的PVC時,集羣可能會嘗試專門地供給
  volume給PVC。這種供給基於StorageClass。
PVC與PV的綁定是一對一的映射。沒找到匹配的PV,那麼PVC會無限期得處於unbound未綁定
狀態。
使用
- Pod使用PVC就像使用volume一樣。集羣檢查PVC,查找綁定的PV,並映射PV給Pod。對
  於支持多種訪問模式的PV,用戶可以指定想用的模式。一旦用戶擁有了一個PVC,並且
  PVC被綁定,那麼只要用戶還需要,PV就一直屬於這個用戶。用戶調度Pod,通過在Pod的
  volume塊中包含PVC來訪問PV。
釋放
- 當用戶使用PV完畢後,他們可以通過API來刪除PVC對象。當PVC被刪除後,對應的PV就
  被認爲是已經是“released”了,但還不能再給另外一個PVC使用。前一個PVC的屬於還存
  在於該PV中,必須根據策略來處理掉。
回收
- PV的回收策略告訴集羣,在PV被釋放之後集羣應該如何處理該PV。當前,PV可以被
  Retained(保留)、 Recycled(再利用)或者Deleted(刪除)。保留允許手動地再次聲明
  資源。對於支持刪除操作的PV卷,刪除操作會從Kubernetes中移除PV對象,還有對應的外
  部存儲(如AWS EBS,GCE PD,Azure Disk,或者Cinder volume)。動態供給的卷總是
  會被刪除。

nfs+pv（靜態）

創建一個pv：

[root@server2 vol]# vim pv1.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce		/只允許一個人讀寫
  persistentVolumeReclaimPolicy: Recycle		/在利用回收策略
  storageClassName: nfs			/存儲類
#  mountOptions:			/掛載選項，可以不加
#  - hard
#  - nfsvers=4.1
  nfs:
    path: /nfsdata
    server: 172.25.254.2		/指定nfs後端

[root@server2 vol]# kubectl apply -f pv1.yml 
persistentvolume/pv1 created
[root@server2 vol]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv1    5Gi        RWO            Recycle          Available           nfs                     4s

訪問模式
- ReadWriteOnce – 該volume只能被單個節點以讀寫的方式映射
- ReadOnlyMany – 該volume可以被多個節點以只讀方式映射
- ReadWriteMany – 該volume可以被多個節點以讀寫的方式映射
- 在命令行中,訪問模式可以簡寫爲:
  • RWO - ReadWriteOnce
  • ROX - ReadOnlyMany
  • RWX - ReadWriteMany
回收策略
- Retain:保留,需要手動回收
- Recycle:回收,自動刪除卷中數據
- Delete:刪除,相關聯的存儲資產,如AWS EBS,GCE PD,Azure Disk,or OpenStackCinder卷都會被刪除

當前,只有NFS和HostPath支持回收利用,AWS EBS,GCE PD,Azure Disk,or OpenStack Cinder卷支持刪除操作。-

狀態:
- Available:空閒的資源,未綁定給PVC
- Bound:綁定給了某個PVC
- Released:PVC已經刪除了,但是PV還沒有被集羣回收
- Failed:PV在自動回收中失敗了
- 命令行可以顯示PV綁定的PVC名稱。

綁定pvc：

[root@server2 vol]# vim pv1.yml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs
  mountOptions:
  - hard
  - nfsvers=4.1
  nfs:
    path: /nfs
    server: 172.25.254.2

---			/在後面加入這些內容
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc1
spec:
  storageClassName: nfs			/和pv對應
  accessModes:
  - ReadWriteOnce			/訪問類型也和pv對應，不然綁定失敗
  resources:
    requests:
      storage: 1Gi			/請求大小，必須小於 pv

[root@server2 vol]# kubectl apply -f pv1.yml 
persistentvolume/pv1 unchanged
persistentvolumeclaim/pvc1 created
[root@server2 vol]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM          STORAGECLASS   REASON   AGE
pv1    5Gi        RWO            Recycle          Bound    default/pvc1   nfs                     15m
[root@server2 vol]# kubectl get pvc
NAME   STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc1   Bound    pv1      5Gi        RWO            nfs            13s

可見已經綁定了，那末接下來我們就可以直接使用 pod 掛載 pvc 了

[root@server2 vol]# vim pv1.yml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs
  mountOptions:
  - hard
  - nfsvers=4.1
  nfs:
    path: /nfs
    server: 172.25.254.2

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc1
spec:
  storageClassName: nfs
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /usr/share/nginx/html		//掛載pvc
      name: pv1
  volumes:
  - name: pv1
    persistentVolumeClaim:
      claimName: pvc1

[root@server2 vol]# kubectl apply -f pv1.yml 
persistentvolume/pv1 created
persistentvolumeclaim/pvc1 created
pod/test-pd created
[root@server2 vol]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM          STORAGECLASS   REASON   AGE
pv1    5Gi        RWO            Recycle          Bound    default/pvc1   nfs                     6s
[root@server2 vol]# kubectl get pvc
NAME   STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc1   Bound    pv1      5Gi        RWO            nfs            7s
[root@server2 vol]# kubectl get pod
NAME      READY   STATUS    RESTARTS   AGE
test-pd   1/1     Running   0          9s				

[root@server2 vol]# kubectl describe pod test-pd 
Volumes:
  pv1:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  pvc1

掛載成功。

[root@server2 vol]# curl 10.244.141.227
88888		/是nfs中的頁面

[root@server1 nfsdata]# vim index.html  
88888
88888			/更改一下頁面
88888

[root@server2 vol]# curl 10.244.141.227
88888
88888			/直接更新了
88888

多創建幾個pv：
在創建兩個nfs目錄，共享出去

[root@server1 /]# mkdir /nfsdata2
[root@server1 /]# mkdir /nfsdata3
[root@server1 nfsdata2]# echo nfs22222 > index.html
[root@server1 nfsdata2]# cd ../nfsdata3
[root@server1 nfsdata3]# echo nfs33333 >index.html
[root@server1 /]# vim /etc/exports
/nfsdata	*(rw,sync)
/nfsdata2	*(rw,sync)
/nfsdata3	*(ro)
[root@server1 /]# chmod 777 /nfsdata2
[root@server1 /]# exportfs -rv
exporting *:/nfsdata3
exporting *:/nfsdata2
exporting *:/nfsdata
[root@server1 /]# showmount -e
Export list for server1:
/nfsdata3 *
/nfsdata2 *
/nfsdata  *

創建pv2

[root@server2 vol]# vim pv2.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv2
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs
  nfs:
    path: /nfsdata2			/指定路徑爲nfsdata2
    server: 172.25.254.1

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc2
spec:
  storageClassName: nfs
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 2Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: test-pd-2
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: pv2
  volumes:
  - name: pv2
    persistentVolumeClaim:
      claimName: pvc2

[root@server2 vol]# kubectl apply -f pv2.yml 
persistentvolume/pv2 created
persistentvolumeclaim/pvc2 created
pod/test-pd-2 created
[root@server2 vol]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM          STORAGECLASS   REASON   AGE
pv1    5Gi        RWO            Recycle          Bound    default/pvc1   nfs                     21m
pv2    5Gi        RWX            Recycle          Bound    default/pvc2   nfs                     5s
[root@server2 vol]# kubectl get pvc
NAME   STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc1   Bound    pv1      5Gi        RWO            nfs            21m
pvc2   Bound    pv2      5Gi        RWX            nfs            6s
[root@server2 vol]# kubectl get pod -owide
NAME        READY   STATUS    RESTARTS   AGE   IP               NODE      NOMINATED NODE   READINESS GATES
test-pd     1/1     Running   0          21m   10.244.141.227   server3   <none>           <none>
test-pd-2   1/1     Running   0          32s   10.244.141.228   server3   <none>           <none>
[root@server2 vol]# curl 10.244.141.228
nfs22222
[root@server2 vol]# curl 10.244.141.227
88888
88888
88888				/pvc1的pod 和pvc2 的pod訪問的內容不同，以此類推

由於策略是讀寫的，所以我們可以進入pod中進行更改：

[root@server2 vol]# kubectl exec -it test-pd-2 -- bash		/進入pod中
root@test-pd-2:/# cd /usr/share/nginx/html/
root@test-pd-2:/usr/share/nginx/html# ls
index.html
root@test-pd-2:/usr/share/nginx/html# echo nfs222222\nnfs222222 >index.html 
bash: index.html: Permission denied		無法更改原文件，因爲源文件是root寫的
root@test-pd-2:/usr/share/nginx/html# echo baidu.com > test.html			/可以創建新文件
root@test-pd-2:/usr/share/nginx/html# ls
index.html  test.html
root@test-pd-2:/usr/share/nginx/html# exit
[root@server2 vol]# curl 10.244.141.228/test.html
baidu.com				/可以訪問到

[root@server1 nfsdata3]# cd ../nfsdata2
[root@server1 nfsdata2]# ls
index.html  test.html
[root@server1 nfsdata2]# cat test.html 
baidu.com				/nfs主機上也出現了這個文件

創建只讀模式的pv3

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv3
spec:
  capacity:
    storage: 20Gi
  volumeMode: Filesystem
  accessModes:
  - ReadOnlyMany
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs
  nfs:
    path: /nfsdata3
    server: 172.25.254.1

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc3
spec:
  storageClassName: nfs
  accessModes:
  - ReadOnlyMany
  resources:
    requests:
      storage: 20Gi

---
apiVersion: v1
kind: Pod
metadata:
  name: test-pd-3
spec:
  containers:
  - image: nginx
    name: nginx
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: pv3
  volumes:
  - name: pv3
    persistentVolumeClaim:
      claimName: pvc3

[root@server2 vol]# kubectl apply -f pv3.yml 
persistentvolume/pv3 created
persistentvolumeclaim/pvc3 created
pod/test-pd-3 created

[root@server2 vol]# kubectl get pv
NAME   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM          STORAGECLASS   REASON   AGE
pv1    5Gi        RWO            Recycle          Bound    default/pvc1   nfs                     34m
pv2    5Gi        RWX            Recycle          Bound    default/pvc2   nfs                     13m
pv3    20Gi       ROX            Recycle          Bound    default/pvc3   nfs                     4s

[root@server2 vol]# kubectl get pod -owide
NAME        READY   STATUS    RESTARTS   AGE   IP               NODE      NOMINATED NODE   READINESS GATES
test-pd     1/1     Running   0          35m   10.244.141.227   server3   <none>           <none>
test-pd-2   1/1     Running   0          13m   10.244.141.228   server3   <none>           <none>
test-pd-3   1/1     Running   0          14s   10.244.22.28     server4   <none>           <none>

[root@server2 vol]# curl 10.244.22.28
nfs33333

[root@server2 vol]# kubectl exec -it test-pd-3 -- bash
root@test-pd-3:/# 
root@test-pd-3:/# cd /usr/share/nginx/html/
root@test-pd-3:/usr/share/nginx/html# echo 78967 > index.html 
bash: index.html: Read-only file system
root@test-pd-3:/usr/share/nginx/html# echo 8768676 > test.html
bash: test.html: Read-only file system		/不能更改也不能創建，因爲是隻讀的文件系統，

目前我們使用的方式就是使用靜態的方式，手動創建，不可能總是給每個用戶都創建動態pv，當沒有匹配的pv時。
集羣可能會嘗試專門地供給volume給PVC。這種供給基於StorageClass,它可以使用多種後端

NFS動態分配PV

配置一下nfs共享：

[root@server1 nfsdata2]# vim /etc/exports
/nfsdata	*(rw,sync,no_root_squash)			/只要一個就夠了
[root@server1 nfsdata2]# exportfs -rv
exporting *:/nfsdata
[root@server1 nfsdata2]# showmount -e
Export list for server1:
/nfsdata *
[root@server1 nfsdata2]# cd /nfsdata
[root@server1 nfsdata]# rm -f *
裏面將來自動創建pv。

StorageClass提供了一種描述存儲類(class)的方法,不同的class可能會映射到不同的服務質量等級和備份策略或其他策略等。
每個 StorageClass 都包含 provisioner、parameters 和 reclaimPolicy 字段, 這些字段會在StorageClass需要動態分配 PersistentVolume 時會使用到。
StorageClass的屬性
- Provisioner(存儲分配器):用來決定使用哪個卷插件分配 PV,該字段必須指定。可以指定內部分配器,也可以指定外部分配器。外部分配器的代碼地址爲: kubernetes-incubator/external-storage,其中包括NFS和Ceph等。
- Reclaim Policy(回收策略):通過reclaimPolicy字段指定創建的Persistent Volume的回收策略,回收策略包括:Delete 或者 Retain,沒有指定默認爲Delete。
- 更多屬性查看:https://kubernetes.io/zh/docs/concepts/storage/storage-classes/

由於我們這裏使用的是nfs的存儲，所以我們使用nfs的 Provisioner。

NFS Client Provisioner是一個automatic provisioner,使用NFS作爲存儲,自動創建PV和對應的PVC,本身不提供NFS存儲,需要外部先有一套NFS存儲服務。
- PV以 ${namespace}-${pvcName}-${pvName}的命名格式提供(在NFS服務器上)
- PV回收的時候以 archieved-${namespace}-${pvcName}-${pvName} 的命名格式(在NFS服務器上)
- nfs-client-provisioner源碼地址:https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client

配置授權:

[root@server2 vol]# mkdir nfs-client
[root@server2 vol]# cd nfs-client/
[root@server2 nfs-client]# vim rbac.yml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

部署NFS Client Provisioner 分配器：

[root@server2 nfs-client]# vim deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: nfs-client-provisioner:latest			/從harbor倉庫下載
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:				/給pod中指定環境變量
            - name: PROVISIONER_NAME
              value: caoaoyuan.org/nfs
            - name: NFS_SERVER
              value: 172.25.254.1
            - name: NFS_PATH
              value: /nfsdata
      volumes:			/卷
        - name: nfs-client-root
          nfs:
            server: 172.25.254.1		/地址
            path: /nfsdata			/目錄

創建 NFS SotageClass存儲類：

[root@server2 nfs-client]# vim class.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage			/存儲類 的名字
provisioner:  caoaoyuan.org/nfs 		/和上面的PROVISIONER_NAME環境變量值相同
parameters:
  archiveOnDelete: "false"				/刪除的時候不打包

創建PVC:

[root@server2 nfs-client]# vim pvsc.yml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: test-claim
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"		/定義存儲類
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi

創建測試Pod:

[root@server2 nfs-client]# vim pvc.yml 
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox		/harbor私有倉庫中有
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"		/創建/mnt下的文件，相當於在nfs服務器上創建
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"			/掛載到mnt上
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: test-claim			/使用上面的pvc

準備工作就做好了，全部應用：

[root@server2 nfs-client]# kubectl apply -f .
[root@server2 nfs-client]# kubectl get pod
NAME                                      READY   STATUS      RESTARTS   AGE
nfs-client-provisioner-576d464467-6k84n   1/1     Running     0          13s
test-pod                                  0/1     Completed   0          14s		/ 完成

在nfs服務器上查看；

[root@server2 nfs-client]# kubectl get pv
NAME                                  		/當前pv名
pvc-4d20ab64-8a73-4c7c-a70b-22216d7d5812  
[root@server2 nfs-client]# kubectl get pvc
NAME        
test-claim 				/當前pvc名

nfs上查看：
[root@server1 ~]# cd /nfsdata
[root@server1 nfsdata]# ls
default-test-claim-pvc-4d20ab64-8a73-4c7c-a70b-22216d7d5812

PV以 ${namespace}-${pvcName}-${pvName}的命名格式自動在NFS服務器上創建.

[root@server2 nfs-client]# kubectl delete pod test-pod 
pod "test-pod" deleted
[root@server2 nfs-client]# kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
test-claim   Bound    pvc-4d20ab64-8a73-4c7c-a70b-22216d7d5812   100Mi      RWX            managed-nfs-storage   13m

刪除 pod 後 pvc 依然存在，pv也在。

[root@server2 nfs-client]# kubectl delete pvc test-claim 
persistentvolumeclaim "test-claim" deleted
[root@server2 nfs-client]# kubectl get pv
No resources found in default namespace.
[root@server2 nfs-client]# kubectl get pvc
No resources found in default namespace.

當我們刪除pvc的時候，pv也被刪除。

[root@server1 nfsdata]# ls
default-test-claim-pvc-4d20ab64-8a73-4c7c-a70b-22216d7d5812
[root@server1 nfsdata]# ls
[root@server1 nfsdata]# ls

[root@server2 nfs-client]# kubectl get storageclasses.storage.k8s.io 
NAME                  PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   caoaoyuan.org/nfs   Delete          Immediate           false                  19m

因爲我們沒有定義回收策略，默認爲delete，而且我們在存儲類中設置了刪除時不打包.

如果我們需要保存生成的數據時：

[root@server2 nfs-client]# vim class.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner:  caoaoyuan.org/nfs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "true"		/改爲ture,就會自動的爲你保存
  [root@server2 nfs-client]# kubectl describe storageclasses.storage.k8s.io managed-nfs-storage 

Provisioner:           caoaoyuan.org/nfs
Parameters:            archiveOnDelete=true


[root@server2 nfs-client]# kubectl apply -f class.yml 
storageclass.storage.k8s.io/managed-nfs-storage created
[root@server2 nfs-client]# kubectl apply -f pvc.yml 		/創建pvc會自動創建pv
persistentvolumeclaim/test-claim created

[root@server1 nfsdata]# ls
default-test-claim-pvc-f76a4858-2128-48f8-b3dc-3895043968de		/pv創建了

[root@server2 nfs-client]# kubectl delete -f pvc.yml 			/在刪除pvc
persistentvolumeclaim "test-claim" deleted

[root@server1 nfsdata]# ls
archived-default-test-claim-pvc-f76a4858-2128-48f8-b3dc-3895043968de		
	/文件被打包了，以archived 開頭

在存儲比較緊的情況下，還不建議開啓這個參數的。我們現在在改回false。

我們還可以通多創建多個pvc，一次創建多個 pv：

[root@server2 nfs-client]# vim pvc.yml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc1
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1G

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc2
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadOnlyMany
  resources:
    requests:
      storage: 6G


---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc3
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10G

[root@server2 nfs-client]# kubectl apply -f class.yml 
storageclass.storage.k8s.io/managed-nfs-storage created
[root@server2 nfs-client]# kubectl apply -f pvc.yml 
persistentvolumeclaim/pvc1 created
persistentvolumeclaim/pvc2 created
persistentvolumeclaim/pvc3 created
[root@server2 nfs-client]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM          STORAGECLASS          REASON   AGE
pvc-c46eecca-1852-442d-8cfb-7f03423f26c2   1G         RWX            Delete           Bound    default/pvc1   managed-nfs-storage            4s
pvc-e2f0581c-ac78-47a0-b3b6-453bfbbc2582   6G         ROX            Delete           Bound    default/pvc2   managed-nfs-storage            4s
pvc-fae89201-5f59-4a82-a310-a6c40c161eb1   10G        RWO            Delete           Bound    default/pvc3   managed-nfs-storage            4s
[root@server2 nfs-client]# kubectl delete -f pvc.yml 
persistentvolumeclaim "pvc1" deleted
persistentvolumeclaim "pvc2" deleted
persistentvolumeclaim "pvc3" deleted
[root@server2 nfs-client]# kubectl get pv
No resources found in default namespace.

默認的 StorageClass

有這樣一種情況：

[root@server2 nfs-client]# vim pvc.yml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc1
#  annotations:			/註釋掉存儲類
#    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1G
      
[root@server2 nfs-client]# kubectl apply -f pvc.yml 
kube	persistentvolumeclaim/pvc1 created
[root@server2 nfs-client]# kubectl get pvc
NAME   STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc1   Pending  /等待狀態                                             6s
[root@server2 nfs-client]# kubectl get pv
No resources found in default namespace.

當我們不綁定存儲類時，pvc就不能通過存儲類來讓分配器給他分配pv。

因此我們還應該在創建一個 default （默認）的存儲類

默認的StorageClass將被用於動態的爲沒有特定 storage class需求的PersistentVolumeClaims配置存儲:(只能有一個默認StorageClass)
如果沒有默認StorageClass,PVC 也沒有指定storageClassName 的值,那麼意味着它只能夠跟 storageClassName 也是“”的 PV 進行綁定。

當前我們沒有指定存儲類進行綁定，因爲惡哦門門註釋掉了參數。

[root@server2 nfs-client]# kubectl delete -f pvc.yml

[root@server2 nfs-client]# kubectl patch storageclass managed-nfs-storage -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

也可以將參數加到資源清單中，永久生效：
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
  annotations:
    "storageclass.kubernetes.io/is-default-class": "true"		/加到這裏
provisioner:  caoaoyuan.org/nfs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

相當於在 annotations 這裏加了一個參數，將 managed-nfs-storage 這個存儲類設置成了默認的存儲類，這樣我們在動態配置存儲時就可以不指定特定的存儲類而使用這個默認的存儲類。

[root@server2 nfs-client]# kubectl get storageclasses.storage.k8s.io 
NAME                  PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   caoaoyuan.org/nfs   Delete          Immediate           false                  3d17h

[root@server2 nfs-client]# kubectl patch storageclass managed-nfs-storage -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/managed-nfs-storage patched

[root@server2 nfs-client]# kubectl get storageclasses.storage.k8s.io 
NAME                            PROVISIONER         RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage (default)   caoaoyuan.org/nfs   Delete          Immediate           false                  3d17h

前後對比我們可以發現存儲類後面多了一個default。

應用：

[root@server2 nfs-client]# kubectl apply -f pvc.yml 
persistentvolumeclaim/pvc1 created
[root@server2 nfs-client]# kubectl get pvc
NAME   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
pvc1   Bound    pvc-ac06d238-bc06-4126-bdf0-cef617a8ccf9   1G         RWX            managed-nfs-storage   5s

當沒有靜態和動態的存儲類時，就匹配到了默認存儲類。

kubernetes存儲 -- Volumes管理（二）pv持久卷 +NFS、存儲類。

PersistentVolume持久卷

nfs+pv（靜態）

NFS動態分配PV

默認的 StorageClass

【面試準備】又一次失敗的面試經歷，題目離譜～資深軟件測試工程師

dotnet 8 版本與銀河麒麟V10和UOS系統的 glibc 兼容性

kubernetes -- 結點調度控制的幾種方式

kubernetes存儲 -- Volumes管理（三）StatefulSet控制器、StatefulSet部署mysql主從集羣

kubernetes存儲 -- Volumes管理（一）emptyDir卷、hostpath卷、NFS卷

kubernetes存儲 -- Volumes管理（二）pv持久卷 +NFS、存儲類。

kubernetes存儲 -- Secret配置管理

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結