Centos 8安裝Kubernetes 1.16.3 環境搭建實戰（ipvs篇）

使用 kubeadm 搭建Kubernetes 1.16.3單節點集羣實戰（基於Centos 8）

詳細步驟：

1，修改hosts

vi /etc/hosts

追加enp0s8的ip和hostname，例如

192.168.56.163 k8stest

2，創建install-k8s.sh

#!/bin/bash

echo "Start"

export IP_ADDR=$(ip addr show enp0s8 | grep -Po 'inet \K[\d.]+')
echo $IP_ADDR

sudo su - << FOE

# Stop firewall and selinux
sudo systemctl disable --now firewalld
sudo /usr/sbin/setenforce 0
sudo sed -i 's/SELINUX=enforcing/SELINUX=permissive/g' /etc/selinux/config

# Ignore Swap Error while installing kubernetes cluster with Swap
cat<<EOF > /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=--fail-swap-on=false
EOF

# Install neccessary system tools
sudo yum install -y dnf-utils

# Open ipvs
cat <<EOF >/etc/sysconfig/modules/ipvs.modules
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF

sudo chmod 755 /etc/sysconfig/modules/ipvs.modules
sudo bash /etc/sysconfig/modules/ipvs.modules
sudo lsmod | grep -e ip_vs -e nf_conntrack_ipv4
sudo dnf install ipset ipvsadm -y

# Config iptables
echo "br_netfilter" > /etc/modules-load.d/br_netfilter.conf
cat<<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

sudo modprobe br_netfilter
sudo sysctl --system

# Add Docker Repo
sudo dnf config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# Install Docker-CE
sudo dnf makecache timer
sudo dnf -y install --nobest docker-ce
# Enable Docker
sudo systemctl enable --now docker
# Config Docker
if [ ! -d "/etc/docker" ]; then
  mkdir /etc/docker
fi

cat<<EOF > /etc/docker/daemon.json
{
   "exec-opts": ["native.cgroupdriver=systemd"],
   "log-driver": "json-file",
   "log-opts": {
     "max-size": "100m"
   },
   "storage-driver": "overlay2",
   "storage-opts": [
     "overlay2.override_kernel_check=true"
   ],
   "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"]
}
EOF

sudo systemctl daemon-reload
sudo systemctl restart docker

# Add Kubernetes Repo
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

sudo dnf install -y kubeadm kubectl kubelet
sudo systemctl enable kubelet

# Create Kubernetes Cluster
kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=$IP_ADDR --kubernetes-version stable-1.16 --ignore-preflight-errors=Swap --image-repository registry.aliyuncs.com/google_containers

FOE

sleep 10s

# Add User to docker group
sudo usermod -a -G docker $(id -nu)

# Create .kube folder
if [ -f $HOME/.kube/config ]; then
  rm -rf $HOME/.kube/config
fi

if [ ! -d $HOME/.kube ]; then
  mkdir $HOME/.kube
fi

# Copy Kubernetes config file
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Apply network plugin
result=1
while [ $result -ne 0 ]
do
	kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
	result=$?
	sleep 10s
done
#kubectl apply -f https://docs.projectcalico.org/v3.10/manifests/calico.yaml

# Taint master node
kubectl taint nodes --all node-role.kubernetes.io/master-

echo "Complete"

3，執行install-k8s.sh

./install-k8s.sh

4，查看nodes

kubectl get nodes

結果

NAME      STATUS   ROLES    AGE     VERSION
k8stest   Ready    master   2m58s   v1.16.3

5，查看所用pods

kubectl get pods -A

結果

NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE
kube-system   coredns-58cc8c89f4-7nrwd          1/1     Running   0          2m56s
kube-system   coredns-58cc8c89f4-bfqbl          1/1     Running   0          2m56s
kube-system   etcd-k8stest                      1/1     Running   0          2m12s
kube-system   kube-apiserver-k8stest            1/1     Running   0          115s
kube-system   kube-controller-manager-k8stest   1/1     Running   0          118s
kube-system   kube-flannel-ds-amd64-5z9l2       1/1     Running   0          2m54s
kube-system   kube-proxy-twc54                  1/1     Running   0          2m56s
kube-system   kube-scheduler-k8stest            1/1     Running   0          112s

6，kube-proxy開啓ipvs
修改ConfigMap的kube-system/kube-proxy中的config.conf，mode: “ipvs”

kubectl edit cm kube-proxy -n kube-system

之後重啓各個節點上的kube-proxy pod：

kubectl get pod -n kube-system | grep kube-proxy | awk '{system("kubectl delete pod "$1" -n kube-system")}'

7，發佈一個nginx deployment，進行檢證。使用centos用戶執行。​​​​​​​

發佈

kubectl create deployment nginx --image=nginx

查看deployments

kubectl get deployments

查看deployments的結果

NAME    READY   UP-TO-DATE   AVAILABLE   AGE
nginx   1/1     1            1           2m13s

查看pods

kubectl get pods -o wide

查看pods的結果

NAME                     READY   STATUS    RESTARTS   AGE     IP           NODE      NOMINATED NODE   READINESS GATES
nginx-86c57db685-pgmwq   1/1     Running   0          2m25s   10.244.0.4   k8stest   <none>           <none>

用curl訪問nginx應用（IP是查看pods的結果的IP值）

curl 10.244.0.4

curl的結果

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

8，爲了操作方便，在~/.bashrc中加入如下內容

source <(kubectl completion bash)
alias k=kubectl
complete -F __start_kubectl k

然後執行

source ~/.bashrc

這樣就可以用k代替kubectl命令了。

恭喜，kubernetes 1.16.3正常安裝完成。