k8s 一臺master，兩臺node節點安裝過程記錄如下
一、安裝環境
[root@k8sm ~]# cat /proc/version
Linux version 3.10.0-1062.12.1.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Tue Feb 4 23:02:59 UTC 2020
二、安裝規劃
docker版本：k8s v1.16.0 推薦的是docker-ce 18.09.9 這個版本，可以看下，k8s對應的docker版本，否則安裝的時候會有告警。
[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 18.03.1-ce. Latest validated version: 18.09
注意機器必須爲2C，2g以上，否則安裝的時候會報錯
[root@k8sm ~]# docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:20:16 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:23:58 2018
OS/Arch: linux/amd64
Experimental: false
[root@k8sm ~]#
master： 192.168.47.129 k8sm
node1：192.168.47.132 k8ss1
node2：192.168.47.133 k8ss2
k8s版本：v1.16.0
三、安裝前工作
1.確認是否已配置靜態IP
2.修改hostname
3.安裝docker
4.配置k8s安裝環境
（1）關閉防火牆
systemctl disable firewalld
systemctl stop firewalld
（2）關閉selinux
setenforce 0
sed -i 's/SELINUX=permissive/SELINUX=disabled/' /etc/sysconfig/selinux
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
（3）禁用swap
swapoff -a
sed -i 's/.*swap.*/#&/' /etc/fstab
（4）修改內核參數
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
（5）重新加載配置文件
sysctl --system
（6）配置阿里的k8s yum源
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
四、安裝kubeadm、kubelet、kubectl（僅master需要安裝）
master：
yum install -y kubectl-1.16.0-0 kubeadm-1.16.0-0 kubelet-1.16.0-0
node：
yum install -y kubeadm-1.16.0-0 kubelet-1.16.0-0
安裝完成後，重啓k8s服務：
systemctl enable kubelet && systemctl start kubelet
五、初始化k8s
（1）master上：
使用阿里雲的源registry.aliyuncs.com/google_containers
初始化k8s
kubeadm init --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.16.0 --apiserver-advertise-address 192.168.47.129 --pod-network-cidr=10.244.0.0/16 --token-ttl 0
–apiserver-advertise-address 192.168.47.129 根據master的ip修改
初始化安裝完成後根據提示輸入命令：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
獲取node加入集羣的命令
kubeadm token create --print-join-command
（2）node上
執行master中“kubeadm token create --print-join-command” 獲取的命令
（3）驗證
加入集羣后，在master上驗證。三臺機器已在集羣中（status爲notready後續安裝flannel）
[root@k8sm tmp]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm NotReady master 137m v1.16.0
k8ss1 NotReady <none> 96m v1.16.0
k8ss2 NotReady <none> 95m v1.16.0
六、安裝flannel（master上）
已將kube-flannel.yml中的quay.io，修改爲國內可以訪問的地址quay-mirror.qiniu.com
配置文件放網盤了需要的自取：
鏈接：https://pan.baidu.com/s/18La-DOb9-iVLtqldMIh26g
提取碼：q4iv
或者能翻牆的也可以直接wget：
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
安裝fannel
kubectl apply -f kube-flannel.yml
安裝完後驗證：
[root@k8sm tmp]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-58cc8c89f4-6l2bd 1/1 Running 0 177m
kube-system coredns-58cc8c89f4-lfr64 1/1 Running 0 177m
kube-system etcd-k8sm 1/1 Running 0 176m
kube-system kube-apiserver-k8sm 1/1 Running 0 176m
kube-system kube-controller-manager-k8sm 1/1 Running 0 176m
kube-system kube-flannel-ds-amd64-fg7qq 1/1 Running 0 17m
kube-system kube-flannel-ds-amd64-gmhhz 1/1 Running 0 17m
kube-system kube-flannel-ds-amd64-lzm74 1/1 Running 0 17m
kube-system kube-proxy-29p5w 1/1 Running 0 177m
kube-system kube-proxy-9bgr4 1/1 Running 0 135m
kube-system kube-proxy-gzhmw 1/1 Running 0 136m
kube-system kube-scheduler-k8sm 1/1 Running 0 176m
[root@k8sm tmp]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm Ready master 177m v1.16.0
k8ss1 Ready <none> 135m v1.16.0
k8ss2 Ready <none> 134m v1.16.0
三節點的status爲ready，且pod均爲runing,至此安裝完成
七、安裝過程中遇到的問題
1.沒有禁用swap
[ERROR Swap]: running with swap on is not supported. Please disable swap
[preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=...
沒有禁用swap，禁用swap後解決
2.初始化的過程中遇到：
[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
解決方法：
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
3.在裝完flannel後 kubectl get nodes檢查發現有一個節點還是notready
[root@k8sm tmp]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8sm Ready master 161m v1.16.0
k8ss1 NotReady <none> 120m v1.16.0
k8ss2 Ready <none> 119m v1.16.0
檢查 pod發現：kube-system kube-flannel-ds-amd64-fg7qq 0/1 Init:ErrImagePull 0 79s
有報 Init:ErrImagePull的錯誤
[root@k8sm tmp]# kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-58cc8c89f4-6l2bd 1/1 Running 0 161m
kube-system coredns-58cc8c89f4-lfr64 1/1 Running 0 161m
kube-system etcd-k8sm 1/1 Running 0 160m
kube-system kube-apiserver-k8sm 1/1 Running 0 160m
kube-system kube-controller-manager-k8sm 1/1 Running 0 160m
kube-system kube-flannel-ds-amd64-fg7qq 0/1 Init:ErrImagePull 0 79s
kube-system kube-flannel-ds-amd64-gmhhz 1/1 Running 0 79s
kube-system kube-flannel-ds-amd64-lzm74 1/1 Running 0 79s
kube-system kube-proxy-29p5w 1/1 Running 0 161m
kube-system kube-proxy-9bgr4 1/1 Running 0 119m
kube-system kube-proxy-gzhmw 1/1 Running 0 120m
kube-system kube-scheduler-k8sm 1/1 Running 0 160m
通過網上各種搜索，可能是Kubernetes無法找到本地的flannel鏡像導致的。
分別在三臺機器上執行docker images|grep fla
[root@k8sm tmp]# docker images|grep fla
quay-mirror.qiniu.com/coreos/flannel v0.11.0-amd64 ff281650a721 13 months ago 52.6MB
[root@k8ss1 ~]# docker images|grep fla
[root@k8ss2 ~]# docker images|grep fla
quay-mirror.qiniu.com/coreos/flannel v0.11.0-amd64 ff281650a721 13 months ago 52.6MB
發現確實k8ss1上沒有這個本地鏡像
直接在k8ss2上導出進行到共享文件夾並在k8ss1上導入
[root@k8ss2 mnt]# docker save -o /mnt/fannel.tar.gz quay-mirror.qiniu.com/coreos/flannel
[root@k8ss1 mnt]# docker load --input ops.v1.tar.gz
最後再檢查下，狀態都爲running和ready