Centos7.6部署k8S(v1.14.2)集羣

系統環境說明：

一、docker安裝

1.安裝依賴包

[root@master ~]# yum install -y yum-utils device-mapper-persistent-data lvm2

2.設置Docker源

[root@master ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

3.Docker安裝版本查看
[root@master ~]# yum list docker-ce --showduplicates | sort -r
4.安裝指定docker(自行選擇)

[root@master ~]# yum install docker-ce-18.09.6 docker-ce-cli-18.09.6 containerd.io

5.啓動docker

[root@master ~]# systemctl start docker

[root@master ~]# systemctl enable docker

6.命令補全，安裝bash-completion，加載bash-completion

[root@master ~]# yum -y install bash-completion

[root@master ~]# source /etc/profile.d/bash_completion.sh

7.鏡像加速,配置鏡像加速器(阿里雲賬號申請)

[root@master ~]# echo ‘{"registry-mirrors": ["https://m2to419r.mirror.aliyuncs.com"]}‘ > /etc/docker/daemon.json

8.重啓服務

[root@master ~]# systemctl daemon-reload

[root@master ~]# systemctl restart docker

9.驗證

[root@master ~]# docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77156
 Built:             Sat May  4 02:34:58 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 02:02:43 2019
  OS/Arch:          linux/amd64
  Experimental:     false

注意：docker安裝默認安裝位置在/var/lib/docker,後期需要存放鏡像需要較大磁盤空間，可修改存儲位置。
1.停止docker容器

[root@master ~]# systemctl stop docker

2.將文件移動到指定目錄

[root@master ~]# cd /var/lib/

[root@master ~]# mv docker /data/

3.建立軟鏈接

[root@master ~]# ln -s /data/docker/ /var/lib/docker

4.啓動docker

[root@master ~]# systemctl start docker

5.查看是否成功

[root@master ~]# docker info|grep 'Docker Root'
Docker Root Dir: /data/docker

二、K8S安裝準備工作

1.配置主機名

[root@master ~]# hostnamectl set-hostname master

退出重新登錄即可顯示
2.修改hosts文件

[root@master ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.3.183 master
192.168.2.110 node1
192.168.2.111 node2

3.驗證mac地址uuid,保證各個節點mac和uuid唯一

[root@master ~]# cat /sys/class/net/ens32/address
00:0c:29:53:4e:c3

[root@master ~]# cat /sys/class/dmi/id/product_uuid 
BB714D56-895D-BBC9-4879-C0BC0C534EC3

4.禁用swap
臨時禁用：

[root@master ~]# swapoff -a

永久禁用：

[root@master ~]# sed -i.bak '/swap/s/^/#/' /etc/fstab

5.內核參數修改
臨時修改：

[root@master ~]# sysctl net.bridge.bridge-nf-call-iptables=1

[root@master ~]# sysctl net.bridge.bridge-nf-call-ip6tables=1

永久修改：

[root@master ~]# cat /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

6.修改Cgroup Driver，消除警告信息。
修改daemon.json，新增‘“exec-opts”: [“native.cgroupdriver=systemd”’

[root@master ~]# more /etc/docker/daemon.json 
{
"registry-mirrors": ["https://m2to419r.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"
}

7.重新加載docker

# systemctl daemon-reload      
[root@master ~]# systemctl restart docker

8.設置Kubernetes源
1.新增Kubernetes源

[root@master ~]# more /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

[] 中括號中的是repository id，唯一，用來標識不同倉庫
name 倉庫名稱，自定義
baseurl 倉庫地址
enable 是否啓用該倉庫，默認爲1表示啓用
gpgcheck 是否驗證從該倉庫獲得程序包的合法性，1爲驗證
repo_gpgcheck 是否驗證元數據的合法性元數據就是程序包列表，1爲驗證
gpgkey=URL 數字簽名的公鑰文件所在位置，如果gpgcheck值爲1，此處就需要指定gpgkey文件的位置，如果gpgcheck值爲0就不需要此項了

2.更新緩存

[root@master yum.repos.d]# yum clean all
[root@master yum.repos.d]# yum -y makecache

三、Master節點安裝

1.版本查看

[root@master ~]# yum list kubelet --showduplicates | sort -r

2.安裝Kubelet、kubeadm和kubectl

[root@master ~]# yum install -y kubelet-1.14.2 kubeadm-1.14.2 kubectl-1.14.2

若不指定版本直接運行yum install -y kubelet kubeadm kubectl則默認安裝最新版。
2.1 安裝包說明

kubelet 運行在集羣所有節點上，用於啓動Pod和容器等對象的工具
kubeadm 用於初始化集羣，啓動集羣的命令工具
kubectl 用於和集羣通信的命令行，通過kubectl可以部署和管理應用，查看各種資源，創建、刪除和更新各種組件
2.2 啓動Kubelet

[root@master ~]# systemctl enable kubelet && systemctl start kubelet

kubelet命令補全

[root@master ~]# echo "source <(kubectl completion bash)" >> ~/.bash_profile
[root@master ~]# source .bash_profile

3.下載鏡像
3.1 鏡像下載的腳本

[root@master ~]# vim images.sh 
#!/bin/bash
url=registry.cn-hangzhou.aliyuncs.com/google_containers
version=v1.14.2
images=(`kubeadm config images list --kubernetes-version=$version|awk -F '/' '{print $2}'`)
for imagename in ${images[@]} ; do
  docker pull $url/$imagename
  docker tag $url/$imagename k8s.gcr.io/$imagename
  docker rmi -f $url/$imagename
done

Url爲阿里雲鏡像倉庫地址，version爲安裝的Kubernetes版本。
3.2 下載鏡像
運行腳本images.sh，下載版本的鏡像，運行腳本腳本前先賦權

[root@master ~]# chmod +x images.sh
[root@master ~]# ./images.sh
[root@master ~]# docker images

4.初始化master
4.1初始化

[root@master ~]# kubeadm init --apiserver-advertise-address 192.168.3.107 --pod-network-cidr=10.244.0.0/16

Apiserver-advertise-address指定master的interface，pod-network-cid指定Pod網絡的範圍，這裏使用flannel網絡方案。192.168.3.107爲master主機的ip。

注意：記錄kubeadm join的輸出，後面需要這個命令將在各個節點上執行，將node節點加入集羣中。

kubeadm join 192.168.3.107:6443 --token iegn1r.sc93aaob0gvs5y2q --discovery-token-ca-cert-hash sha256:cd80d74c457c804389c34d90cb006e719dae1946e7f7fe3dfb1f774b0e83f526

4.2 加載環境變量

[root@master ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
[root@master ~]# source .bash_profile

本文所有操作都在root用戶下執行，若爲非root用戶，則執行如下操作：

[root@master ~]# mkdir -p $HOME/.kube
[root@master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master ~]# chown $(id -u):$(id -g) $HOME/.kube/config

5.安裝pod網絡，該鏈接是我自己的，儘量將文件保留本地。

[root@master tools]# kubectl apply -f https://yyf-docker-image.oss-cn-shenzhen.aliyuncs.com/k8s/kubectl1.14.2/kube-flannel.yml

6.Master節點配置
taint：污點的意思，如果一個節點被打上污點，那麼pod是不允許運行在這個節點上面的
6.1刪除master節點默認污點
默認情況下集羣不會在master上調度pod，如果偏想在master上調度Pod，可以執行如下操作：
查看污點：

[root@master tools]# kubectl describe node master|grep -i taints

刪除默認污點：

[root@master tools]# kubectl taint nodes master node-role.kubernetes.io/master-node/master untaintednode/master untainted

四、Node節點安裝

1.安裝kubelet、kubeadm和kubectl
操作同master節點
2.下載鏡像
操作同master節點
3.加入集羣
一下操作在master節點上執行
3.1查看令牌

[root@master tools]# kubeadm token list

3.2生產新的令牌

[root@master tools]# kubeadm token create
7ah2e7.n9vbb3u94g6861xk

3.3生成新的加密串

[root@master tools]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outformder 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

3.4 node節點加入集羣
在2個node節點上分別執行如下操作：(master節點操作步驟提到)

[root@node1 ~]# kubeadm join 192.168.3.107:6443 --token iegn1r.sc93aaob0gvs5y2q --discovery-token-ca-cert-hash sha256:cd80d74c457c804389c34d90cb006e719dae1946e7f7fe3dfb1f774b0e83f526

查看節點
[root@master ~]# kubectl get nodes

五、Dashboard安裝

1.下載yaml

[root@master ~]# wget https://yyf-docker-image.oss-cn-shenzhen.aliyuncs.com/k8s/k2.配置yaml

2.1 修改鏡像地址(已修改，跳過)

[root@master ~]# sed -i 's/k8s.gcr.io/registry.cn-hangzhou.aliyuncs.com\/kuberneters/g' kubernetes-dashboard.yaml

由於默認的鏡像倉庫網絡訪問不通，故改成阿里雲鏡像
2.2外網訪問(已修改，跳過)

[root@master ~]# sed -i '/targetPort:/a\ \ \ \ \ \ nodePort: 30001\n\ \ type: NodePort' kubernetes-dashboard.yaml

2.3新增管理員賬號

[root@master ~]# cat >> kubernetes-dashboard.yaml << EOF
---
# ------------------- dashboard-admin ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
  name: dashboard-admin
  namespace: kube-system

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: dashboard-admin
subjects:
- kind: ServiceAccount
  name: dashboard-admin
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
 EOF

3.部署訪問
3.1部署Dashboard

[root@master ~]# kubectl apply -f kubernetes-dashboard.yaml

3.2狀態查看

[root@master opt]# kubectl get deployment kubernetes-dashboard -n kube-system
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
kubernetes-dashboard   1/1     1            1           2d20h
[root@master opt]# kubectl get pods -n kube-system -o wide
NAME                                    READY   STATUS    RESTARTS   AGE     IP              NODE     NOMINATED NODE   READINESS GATES
coredns-fb8b8dccf-7r8dw                 1/1     Running   161        3d1h    10.244.0.15     master   <none>           <none>
coredns-fb8b8dccf-sw6wf                 1/1     Running   163        3d1h    10.244.0.17     master   <none>           <none>
etcd-master                             1/1     Running   5          3d1h    192.168.3.107   master   <none>           <none>
kube-apiserver-master                   1/1     Running   197        3d1h    192.168.3.107   master   <none>           <none>
kube-controller-manager-master          1/1     Running   8          3d1h    192.168.4.64    master   <none>           <none>
kube-flannel-ds-amd64-gx88f             1/1     Running   3          3d      192.168.3.107   master   <none>           <none>
kube-flannel-ds-amd64-qgxql             1/1     Running   3          2d21h   192.168.2.110   node1    <none>           <none>
kube-flannel-ds-amd64-r92jq             1/1     Running   3          2d21h   192.168.2.111   node2    <none>           <none>
kube-proxy-d28jv                        1/1     Running   3          3d1h    192.168.3.107   master   <none>           <none>
kube-proxy-d7nrl                        1/1     Running   3          2d21h   192.168.4.65    node2    <none>           <none>
kube-proxy-p7fft                        1/1     Running   3          2d21h   192.168.2.110   node1    <none>           <none>
kube-scheduler-master                   1/1     Running   10         3d1h    192.168.3.107   master   <none>           <none>
kubernetes-dashboard-7b87f5bdd6-ndvtd   1/1     Running   4          2d20h   10.244.2.16     node1    <none>           <none>
[root@master opt]# kubectl get services -n kube-system
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
kube-dns               ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP   3d1h
kubernetes-dashboard   NodePort    10.105.12.125   <none>        443:30001/TCP            2d20h

3.3令牌查看

[root@master ~]# kubectl describe secrets -n kube-system dashboard-admin

令牌爲：eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tN202djUiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYjQ3MTM3OTYtOGY1ZC0xMWVhLWJiZjEtMDAwYzI5NTM0ZWMzIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.09N1uXF2Q0mSW2vLCc3NBONSELk3wRXne6TImkYnFx7JpXNKZlwzNbvSqKf-1fU6Kntjul73HSsmwjbDj-XhSPRh8DWKQPCPfN1ViZAdvbwxo7q0rmUU2UgDoBIkhA3iwczMYoAEV1mgRuDA-ljoNoLEqCr_HLcYRUakdKfABR2hB5J8Oej-RU5OQGt0LUfgXfTl5_QI_Yoh2H4bW_MncNIrqUeAnVHNU1rO1fwaEdsofPywnUz4jJIJ0yA5JWXIqvnVOP-R0DJ5h0mTuDpOixRgLzzCL9XJ_u7Ck30Gp9GQigshHwTF3Hmo3ChKH-U2QBiUXDHJz0gZdMRpk81EWQ

3.4訪問
火狐瀏覽器訪問https://MasterIP:30001

通過令牌方式登錄（使用命令查看令牌）

Dashboard提供了可以實現集羣管理、工作負載、服務發現和負載均衡、存儲、字典配置、日誌視圖等功能。

六、集羣測試

1.部署應用
1.1命令方式

[root@master ~]# kubectl run httpd-app --image=httpd --replicas=3

1.2配置文件方式

[root@master ~]# more nginx.yml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      restartPolicy: Always
      containers:
      - name: nginx
        image: nginx:latest

[root@master ~]# kubectl apply -f nginx.yml

2.狀態插卡
2.1查看節點狀態

[root@master opt]# kubectl get nodes
NAME     STATUS   ROLES    AGE     VERSION
master   Ready    master   3d1h    v1.14.2
node1    Ready    <none>   2d21h   v1.14.2
node2    Ready    <none>   2d21h   v1.14.2

2.2查看pod狀態，默認是default命名空間，加參數 --all-namespace可查看所有pod

[root@master yaml]# kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
nginx-9d4cf4f77-4dccf   1/1     Running   0          40s
nginx-9d4cf4f77-tjcb6   1/1     Running   0          40s
nginx-9d4cf4f77-v7r4q   1/1     Running   0          40s

2.3查看副本數

[root@master yaml]# kubectl get deployments
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   3/3     3            3           2m33s
[root@master yaml]# kubectl get pod -o wide
NAME                    READY   STATUS    RESTARTS   AGE     IP            NODE     NOMINATED NODE   READINESS GATES
nginx-9d4cf4f77-4dccf   1/1     Running   0          2m37s   10.244.1.14   node2    <none>           <none>
nginx-9d4cf4f77-tjcb6   1/1     Running   0          2m37s   10.244.0.18   master   <none>           <none>
nginx-9d4cf4f77-v7r4q   1/1     Running   0          2m37s   10.244.2.18   node1    <none>           <none>

可以看到nginx的3個副本pod均勻分佈在3個節點上

2.4查看剛創建nignx的deployment詳情

[root@master yaml]# kubectl describe deployment nginx
Name:                   nginx
Namespace:              default
CreationTimestamp:      Sat, 09 May 2020 10:53:47 +0800
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 1
                        kubectl.kubernetes.io/last-applied-configuration:
                          {"apiVersion":"extensions/v1beta1","kind":"Deployment","metadata":{"annotations":{},"name":"nginx","namespace":"default"},"spec":{"replica...
Selector:               app=nginx
Replicas:               3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx:latest
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-9d4cf4f77 (3/3 replicas created)
Events:
  Type    Reason             Age    From                   Message
  ----    ------             ----   ----                   -------
  Normal  ScalingReplicaSet  6m10s  deployment-controller  Scaled up replica set nginx-9d4cf4f77 to 3

2.5查看集羣基本組件狀態

[root@master yaml]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}

至此完成centos7.6下k8s(v1.14.2)集羣部署。

(K8S實踐0)Centos7.6部署k8S(v1.14.2)集羣

Centos7.6部署k8S(v1.14.2)集羣

一、docker安裝

二、K8S安裝準備工作

三、Master節點安裝

四、Node節點安裝

五、Dashboard安裝

六、集羣測試

推薦2款開源、美觀的WinForm UI控件庫

NET9 AspnetCore將整合OpenAPI的文檔生成功能而無需三方庫

(K8S實踐2)Pod常用操作

(K8S實踐1)Kubernetes基本概念

(K8S實踐0)Centos7.6部署k8S(v1.14.2)集羣

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結