目的
在安裝了win10操作系統個人筆記本電腦上搭建Kubernetes集羣實驗環境,做爲Kubernetes學習之用。
虛擬環境
Oracle VM VirtualBox”建立三臺虛擬機,虛擬機安裝CentOs(http://mirrors.aliyun.com/centos/7.6.1810/isos/x86_64/CentOS-7-x86_64-DVD-1810.iso),每個虛擬機設置2塊網卡,一塊網卡通過NAT連接外網,另一塊用於三臺虛擬機之間通信。
軟件 | 版本 |
CentOS | Linux version 3.10.0-957.el7.x86_64 |
Kubernetes | 1.14.0 |
一、配置虛擬機與安裝所需要的軟件
1.1 配置VirtualBox的NAT網絡,VirtualBox 菜單:管理->全局設定->網絡->增加一新的NAT網絡
1.2 配置VM的網卡
每臺VM設兩塊網卡,一塊爲NAT網絡,一塊爲僅主機,如下圖
1.3 安裝操作系統centos
選擇最小化安裝,主機名設爲master.k8s
1.4 安裝docker 和Kubernetes
安裝完成後登錄,禁用兩個安全功能:SELinux以及防火牆
- 禁用SELinux,執行以下命令
修改 /etc/selinux/config文件,把SELINUX=enforcing行改爲SELINUX=disabled
- 禁用防火牆
# systemctl disable firewalld && systemctl stop firewalld
- 加入kubernetes阿里雲的源
# vi /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
- 安裝DOCKER, KUBELET, KUBEADM, KUBECTL, AND KUBERNETES-CNI
# yum install -y docker kubelet kubeadm kubectl kubernetes-cni
# systemctl enable docker && systemctl start docker
# systemctl enable kubelet && systemctl start kubelet
# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
- 啓用net.bridge.bridge-nf-call-iptables內核選項
# sysctl -w net.bridge.bridge-nf-call-iptables=1
# echo "net.bridge.bridge-nf-call-iptables=1" > /etc/sysctl.d/k8s.conf
# echo "1" > /proc/sys/net/ipv4/ip_forward
- 禁用 SWAP
# swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab
1.5 複製VM
- 關閉VM
# shutdown now
- VirtualBox 界面選擇複製,複製2個虛擬機:node1.k8s,node2.k8s
二、配置三臺虛擬機環境
- 啓動 master.k8s,node1.k8s,node2.k83
- 設置node1.k8s, node2.k8s的主機名
# hostnamectl --static set-hostname node1.k8s
# hostnamectl --static set-hostname node2.k8s
3. 修改三臺虛擬機網絡配置
三臺虛擬機的內網配置如下:
主機名 | IP |
master.k8s | 192.168.56.101 |
node1.k8s | 192.168.56.102 |
node2.k83 |
192.168.56.103 |
# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp0s8
UUID=9993cb88-7de1-4038-bd36-f64e681f4d54
DEVICE=enp0s8
ADDRES=192.168.56.101
IPADDR=192.168.56.101
ONBOOT=yes
4. 設置三臺VM的hosts文件,
# cat /etc/hosts
192.168.56.101 master.k8s
192.168.56.102 node1.k8s
192.168.56.103 node2.k8s
三、配置Kubernetes
3.1 拉取docker images ,編寫腳本拉取所需要的image
# vi pullimages.sh
#!/bin/bash
images=(
kube-proxy:v1.14.0
kube-apiserver:v1.14.0
kube-addon-manager:v9.0
coredns:1.3.1
kubernetes-dashboard-amd64:v1.10.1
etcd:3.3.10
k8s-dns-sidecar-amd64:1.14.13
k8s-dns-kube-dns-amd64:1.14.13
k8s-dns-dnsmasq-nanny-amd64:1.14.13
pause:3.1
storage-provisioner:v1.8.1
kube-controller-manager:v1.14.0
kube-scheduler:v1.14.0
)
for imageName in ${images[@]}; do
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done
3.3 在虛擬機master.k8s用kubeadm配置master
# kubeadm init --kubernetes-version v1.14.0 --apiserver-advertise-address 192.168.56.105 --pod-network-cidr 10.244.0.0/16
[init] Using Kubernetes version: v1.14.0
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \
--discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47
- 建立和修改所需要的目錄
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config
3.4 配置虛擬機node1.k8s,與node2.k8s
- 拉取docker image
# vi pullimages.sh
#!/bin/bash
images=(
kube-proxy:v1.14.0
pause:3.1
)
for imageName in ${images[@]}; do
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName k8s.gcr.io/$imageName
docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done
- 加入集羣
# kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \
--discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47
四、配置Kubernetes網絡
4.1 master主機安裝Weave Net container networking plugin
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
4.2 查詢節點情況
[root@master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master.k8s Ready master 178m v1.14.0
node1.k8s Ready <none> 162m v1.14.0
node2.k8s Ready <none> 113m v1.14.0
[root@master ~]# kubectl get pods -n kube-system -l name=weave-net
NAME READY STATUS RESTARTS AGE
weave-net-9vcqs 2/2 Running 1 156m
weave-net-c5gp2 2/2 Running 2 156m
weave-net-dh48f 2/2 Running 0 108m
五、解決token過期
master節點初始化生成的token有有效期,過期後要加入新的節點要重新生成。
5.1 生成一條永久有效的token
# kubeadm token create --ttl 0
# kubeadm token list
5. 2 獲取ca證書sha256編碼hash值
# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
5.3 加入節點
# kubeadm join 10.167.11.153:6443 --token o4avtg.65ji6b778nyacw68 --discovery-token-ca-cert-hash sha256:2cc3029123db737f234186636330e87b5510c173c669f513a9c0e0da395515b0