如何在Kubernetes集羣動態使用 NAS 持久卷

1. 介紹：

本文介紹的動態生成NAS存儲卷的方案：在一個已有文件系統上，自動生成一個目錄，這個目錄定義爲目標存儲卷；

鏡像地址：registry.cn-hangzhou.aliyuncs.com/acs/alicloud-nas-controller:v1.11.5.4-433631d-aliyun

默認生成資源：
生成的PV名字爲：pvc-${pvc-uid}
生成目錄的名字：namespace-pvcname-pvname

可以再pvc的annotations中如下聲明，自定義名字：
生成的pv、目錄名字爲下面定義的名字。

  annotations:
    pv-name-created: replace-user-id

2. 部署NAS Controller

創建alicloud-nas-controller，實現動態provider nas pv；
創建alicloud-nas storageclass，爲nas pv provision 提供模板；

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-nas
provisioner: alicloud/nas
reclaimPolicy: Delete
parameters:
  drivertype: flexvolume
  nfsversion: "4.0"
  options: ""

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: alicloud-nas-controller
  namespace: kube-system
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: alicloud-nas-controller
    spec:
      tolerations:
      - effect: NoSchedule
        operator: Exists
        key: node-role.kubernetes.io/master
      - effect: NoSchedule
        operator: Exists
        key: node.cloudprovider.kubernetes.io/uninitialized
      serviceAccount: admin
      containers:
        - name: alicloud-nas-controller
          image: registry.cn-hangzhou.aliyuncs.com/acs/alicloud-nas-controller:v1.11.5.4-433631d-aliyun
          imagePullPolicy: Always
          volumeMounts:
          - mountPath: /persistentvolumes
            name: nfs-client-root
          env:
            - name: NFS_SERVER
              value: 154154b095-**.cn-beijing.nas.aliyuncs.com
            - name: NFS_PATH
              value: /
      volumes:
      - name: nfs-client-root
        flexVolume:
          driver: alicloud/nas
          options:
            path: /
            server: 154154b095-**.cn-beijing.nas.aliyuncs.com
            vers: "4.0"

StorageClass使用說明：

drivertype: 用來表示生成pv存儲類型，可選nfs, flexvolume.
    nfs: 默認選項，表示使用k8s原生NFS驅動掛載；
    flexvolume: 表示使用阿里雲提供的Flexvolume NAS驅動掛載；

nfsversion: 掛載nfs使用的版本，支持3，4.0.默認爲4.0；
    drivertype爲flexvolume的時候在這裏配置；
    爲nfs的時候通過mountOptions 配置；

options：爲掛載nfs的可選項配置；
    drivertype爲flexvolume的時候在這裏配置；
    爲nfs的時候通過mountOptions 配置；

StorageClass舉例：

## 使用kubernetes提供的NFS驅動，並配置mountOptions，reclaimPolicy爲Delete；
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-nas-nfs
mountOptions:
- vers=4.0
- noresvport
provisioner: alicloud/nas
reclaimPolicy: Delete

## 使用阿里雲提供的Flexvolume NAS驅動，配置nfs版本、options；
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: alicloud-nas-flex
provisioner: alicloud/nas
reclaimPolicy: Delete
parameters:
  drivertype: flexvolume
  nfsversion: "3"
  options: "noresvport"

3. 創建應用-Deployment：

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: replace-user-id
  annotations:
    pv-name-created: replace-user-id
spec:
  storageClassName: alicloud-nas
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: "deploy-nas"
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: deploy-nas
    spec:
      containers:
        - name: "nginx"
          image: "nginx"
          volumeMounts:
              - name: pvc-nas
                mountPath: "/data"
      volumes:
      - name: pvc-nas
        persistentVolumeClaim:
            claimName: replace-user-id

執行：
# userID="hello-123"
# cat deploy.yaml | sed  "s/replace-user-id/\"$userID\"/g" | kubectl create -f -

# kubectl get pod | grep deploy-nas
deploy-nas-85696b6bfc-t5dmh         1/1       Running          0          28m

# kubectl get pvc | grep hell
hello-123       Bound     hello-123                                  5Gi        RWX            alicloud-nas-flex   28m

# kubectl get pv | grep hell
hello-123                                  5Gi        RWX            Delete           Bound      default/hello-123       alicloud-nas-flex                       28m

# Nas目錄下查看生成目錄：
# ls -l | grep hello
drwxrwxrwx  2 root root 4096 2月  19 09:58 hello-123

4. 創建應用-StatefulSet：

使用volumeTemplateClaim不支持使用pv-name-created配置pv名字；

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: web
spec:
  replicas: 2
  serviceName: "nginx"
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        volumeMounts:
        - mountPath: "/data"
          name: pvc-sts
  volumeClaimTemplates:
  - metadata:
      name: pvc-sts
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: alicloud-nas-flex
      resources:
        requests:
          storage: 2Gi
          
創建後查看：
# kubectl get pod | grep web
web-0                               1/1       Running          0          7s
web-1                               1/1       Running          0          4s

# kubectl get pvc | grep web
pvc-sts-web-0   Bound     pvc-65ab251a-33ec-11e9-a151-00163e066784   2Gi        RWO            alicloud-nas-flex   13m
pvc-sts-web-1   Bound     pvc-8437c50e-33ed-11e9-a151-00163e066784   2Gi        RWO            alicloud-nas-flex   5m

# kubectl get pv | grep web
pvc-65ab251a-33ec-11e9-a151-00163e066784   2Gi        RWO            Delete           Bound      default/pvc-sts-web-0   alicloud-nas-flex                       13m
pvc-8437c50e-33ed-11e9-a151-00163e066784   2Gi        RWO            Delete           Bound      default/pvc-sts-web-1   alicloud-nas-flex                       5m

# Nas目錄下查看生成目錄：
# ls -l | grep sts
drwxrwxrwx  2 root root 4096 2月  19 10:16 default-pvc-sts-web-0-pvc-65ab251a-33ec-11e9-a151-00163e066784
drwxrwxrwx  2 root root 4096 2月  19 10:24 default-pvc-sts-web-1-pvc-8437c50e-33ed-11e9-a151-00163e066784

5. 創建應用-Pod：

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: replace-user-id
  annotations:
    pv-name-created: replace-user-id
spec:
  storageClassName: alicloud-nas-flex
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: "nas-pod"
spec:
  containers:
    - name: "nginx"
      image: "nginx"
      volumeMounts:
          - name: pvc-nas
            mountPath: "/data"
  volumes:
  - name: pvc-nas
    persistentVolumeClaim:
        claimName: replace-user-id
       

# userID="pod-123"
# cat pod.yaml | sed  "s/replace-user-id/\"$userID\"/g" | kubectl create -f -

# kubectl get pod | grep pod
nas-pod                             1/1       Running          0          32s

# kubectl get pvc | grep pod
pod-123         Bound     pod-123                                    5Gi        RWX            alicloud-nas-flex   44s

# kubectl get pv | grep pod
pod-123                                    5Gi        RWX            Delete           Bound      default/pod-123         alicloud-nas-flex                       48s

# ls -l | grep pod
drwxrwxrwx  2 root root 4096 2月  19 10:54 pod-123