K8S的重要性不多说了。今天开始学习K8S，安装阶段竟然遇到很多问题，终于解决了，将安装方法总结下：
1.安装centos 7，如果最小化安装centos，执行
(1)容器内初始安装：yum install -y procps iproute2 net-tools vim
(2)取消firewalld :
systemctl disable firewalld
systemctl stop firewalld
(3)禁用SELinux 修改/etc/sysconfig/selinux，将SELINUX=enforcing修改为SELINUX=disabled，然后setenforce 0
2.centos安装docker：
安装依赖包：yum install -y yum-utils device-mapper-persistent-data lvm2
# 添加Docker软件包源
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
# 安装Docker CE
yum install docker-ce -y
# 启动
systemctl start docker
# 开机启动
systemctl enable docker
# 查看Docker信息
docker info
配置dockerhub加速器
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://bc437cce.m.daocloud.io
3. 修改本地镜像源：/etc/yum.repos.d/k8s.repo
[kubernetes]
name=Kubernetes Repository
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
然后运行yum install 命令安装kubeadm和相关工具:
#yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl start kubelet
systemctl enable kubelet
4.在 执行kubeadm init 的时候会下载类似k8s.gcr.io/kube-apiserver:v1.17.0的镜像，因为国内封的原因，到google的k8s.gcr.io不可达报错。所以提前从docker.io下载，并修改tag。
|
错误信息：[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.17.0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.4.3-0: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.6.5: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) , error: exit status 1 |
提前下载镜像：
docker pull mirrorgooglecontainers/kube-apiserver:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/kube-controller-manager:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/kube-scheduler:v1.16.0-beta.1
docker pull mirrorgooglecontainers/kube-proxy:v1.17.0-alpha.0
docker pull mirrorgooglecontainers/pause:latest
docker pull mirrorgooglecontainers/etcd:3.4.2-0
docker pull coredns/coredns:latest
将镜像tag到kubelet init中要求的镜像名称：
docker tag docker.io/mirrorgooglecontainers/kube-apiserver:v1.17.0-alpha.0 k8s.gcr.io/kube-apiserver:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-controller-manager:v1.17.0-alpha.0 k8s.gcr.io/kube-controller-manager:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-scheduler:v1.16.0-beta.1 k8s.gcr.io/kube-scheduler:v1.17.0
docker tag docker.io/mirrorgooglecontainers/kube-proxy:v1.17.0-alpha.0 k8s.gcr.io/kube-proxy:v1.17.0
docker tag docker.io/mirrorgooglecontainers/pause:latest k8s.gcr.io/pause:3.1
docker tag docker.io/mirrorgooglecontainers/etcd:3.4.2-0 k8s.gcr.io/etcd:3.4.3-0
docker tag docker.io/coredns/coredns:latest k8s.gcr.io/coredns:1.6.5
5.执行kubeadm init，继续报错，多个错误，一个一个看：
|
W1230 19:06:11.581755 103362 validation.go:28] Cannot validate kube-proxy config - no validator is available W1230 19:06:11.583182 103362 validation.go:28] Cannot validate kubelet config - no validator is available [init] Using Kubernetes version: v1.17.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 [ERROR Swap]: running with swap on is not supported. Please disable swap [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher |
(1)[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
通过在/etc/docker/daemon.json中增加：
|
{ "exec-opts": ["native.cgroupdriver=systemd"] } |
并重启docker：systemctl restart docker
(2)[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
通过执行如下命令解决：
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
长期的：
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system
(3)[ERROR Swap]: running with swap on is not supported. Please disable swap
通过执行如下命令解决：
swapoff -a
长期：sed -i "s/\/dev\/mapper\/centos-swap/\#\/dev\/mapper\/centos-swap/g" /etc/fstab
6.此时重置一下 kubeadm 之前执行过的操作，再进行初始化
kubeadm reset && systemctl start kubelet && kubeadm init --ignore-preflight-errors=all
出现如下信息，成功：
|
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.239.128:6443 --token j6aqes.tey9pet7vux0j9q2 \ --discovery-token-ca-cert-hash sha256:7fbc59430a88edd2c7455f61bc9282d691c5c914b80875683fc547163c51e398 |
7.依据提示执行（复制命令到普通用户的home目录下）：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
这样就在Master上安装了kubenets，但在集群内还是没有可用的工作Node，并缺乏容器网络的配置
可以执行kubectl get -n kube-system configmap
后记：后续又做了一次初始化，却总是出现如下错误，
|
Unfortunately, an error has occurred: This error is likely caused by: If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands: Additionally, a control plane component may have crashed or exited when started by the container runtime. |
通过设置了代理解决
export HTTP_PROXY=http://hgc1.host.myzone.fun:574
export HTTPS_PROXY=$HTTP_PROXY
export NO_PROXY="192.168.239.128" //不代理本机，很重要
docker 并不会使用系统的环境变量，需要对 docker 单独进行配置。
mkdir -p /etc/systemd/system/docker.service.d
touch /etc/systemd/system/docker.service.d/http-proxy.conf
echo -e '[Service]\nEnvironment="HTTP_PROXY=http://hgc1.host.myzone.fun:574"' > /etc/systemd/system/docker.service.d/http-proxy.conf
touch /etc/systemd/system/docker.service.d/https-proxy.conf
echo -e '[Service]\nEnvironment="HTTPS_PROXY=https://hgc1.host.myzone.fun:574"' > /etc/systemd/system/docker.service.d/http-proxy.conf
重启 docker 使配置生效
systemctl daemon-reload && systemctl restart docker
docker info | grep -i proxy
HTTP Proxy: http://hgc1.host.myzone.fun:574
HTTPS Proxy: https://hgc1.host.myzone.fun:574
完成可以看到docker启动的进程，
备注：该文章有参考https://blog.csdn.net/u012570862/article/details/80150988
8.节点kubeadm join 遇到很多问题，如下错误，查看kubelet无法启动
| [root@localhost docker]# kubeadm join 192.168.239.128:6443 --token jv5na2.e3atk25qcqs372ux --discovery-token-ca-cert-hash sha256:d2d21911885b6682e5557d264f88fa926d917730f44973933cc2ee56d1e26597 --ignore-preflight-errors=all W0101 02:36:18.667712 79434 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set. [preflight] Running pre-flight checks [WARNING FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists [WARNING FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 [WARNING FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1 [WARNING Port-10250]: Port 10250 is in use [WARNING FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists [preflight] Reading configuration from the cluster... [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system namespace [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Starting the kubelet [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap... [kubelet-check] Initial timeout of 40s passed. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused. error execution phase kubelet-start: error uploading crisocket: timed out waiting for the condition To see the stack trace of this error execute with --v=5 or higher |
先执行kubeadm reset，然后执行kubeadm join 192.168.239.128:6443 --token jv5na2.e3atk25qcqs372ux --discovery-token-ca-cert-hash sha256:d2d21911885b6682e5557d264f88fa926d917730f44973933cc2ee56d1e26597 --ignore-preflight-errors=all
节点加入集群成功
接下来发现kubectl get nodes出现错误，需要从master上将admin.conf拷贝到nodes上，并做成环境变量KUBECONFIG
[root@localhost kubernetes]# kubectl get nodes
W0101 05:15:03.349748 28952 loader.go:223] Config not found: /etc/kubernetes/admin.conf
The connection to the server localhost:8080 was refused - did you specify the right host or port?
scp [email protected]:/etc/kubernetes/admin.conf /etc/kubernetes/.
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
source ~/.bash_profile