前期準備
配置hosts
192.168.245.227 iot-node1 \# master節點(etcd,kubernetes-master) 192.168.245.228 iot-node2 \# slave節點(etcd,kubernetes-node,docker) 192.168.245.229 iot-node3 \# slave節點(etcd,kubernetes-node,docker)
其他
同CentOS-7安裝mesos+marathon+zk前期準備
添加安裝源
添加etcd安裝源
sudo rpm -Uvh
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
添加kubernetes安裝源
sudo tee /etc/yum.repos.d/kubernetes.repo <<-‘EOF’
[kubernetesrepo]
name=Kubernetes Repository
baseurl=http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os
enabled=1
gpgcheck=1
gpgkey=https://yum.dockerproject.org/gpg
EOF
在線安裝etcd
安裝etcd
sudo yum install -y etcd
配置etcd
sudo vi /etc/etcd/etcd.conf
1）節點iot-node1中/etc/etcd/etcd.conf內容：
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.227:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.227:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.227:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.227:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380, etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
2）節點iot-node2中/etc/etcd/etcd.conf內容：
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.228:2380”
ETCD_LISTEN_CLIENT_URLS="http://192.168.245.228:2379,http://127.0.0.1:2379 "
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.228:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.228:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
3）節點iot-node3中/etc/etcd/etcd.conf內容：
#[Member]
ETCD_DATA_DIR="/var/lib/etcd"
ETCD_LISTEN_PEER_URLS=“http://192.168.245.229:2380”
ETCD_LISTEN_CLIENT_URLS=“http://192.168.245.229:2379,http://127.0.0.1:2379”
ETCD_NAME=“etcd1”
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=“http://192.168.245.229:2380”
ETCD_ADVERTISE_CLIENT_URLS=“http://192.168.245.229:2379”
ETCD_INITIAL_CLUSTER=“etcd1=http://192.168.245.227:2380,etcd2=http://192.168.245.228:2380,
etcd3=http://192.168.245.229:2380”
ETCD_INITIAL_CLUSTER_TOKEN=“etcd-cluster”
ETCD_INITIAL_CLUSTER_STATE=“new”
順序啓動3個節點的etcd，iot-node1、iot-node2兩個節點啓動etcd會報超
時，等所有節點都啓動後就沒問題了。
針對幾個URLS做下簡單的解釋：
[member]
ETCD_NAME ：ETCD的節點名
ETCD_DATA_DIR：ETCD的數據存儲目錄
ETCD_SNAPSHOT_COUNTER：多少次的事務提交將觸發一次快照
ETCD_HEARTBEAT_INTERVAL：ETCD節點之間心跳傳輸的間隔，單位毫秒
ETCD_ELECTION_TIMEOUT：該節點參與選舉的最大超時時間，單位毫秒
ETCD_LISTEN_PEER_URLS：該節點與其他節點通信時所監聽的地址列表，多個地址使用逗號隔開，其格式可以劃分爲scheme://IP:PORT，這裏的scheme可以是http、https
ETCD_LISTEN_CLIENT_URLS：該節點與客戶端通信時監聽的地址列表
[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS：該成員節點在整個集羣中的通信地址列表，這個地址用來傳輸集羣數據的地址。因此這個地址必須是可以連接集羣中所有的成員的。
ETCD_INITIAL_CLUSTER：配置集羣內部所有成員地址，其格式爲：ETCD_NAME=ETCD_INITIAL_ADVERTISE_PEER_URLS，如果有多個使用逗號隔開
ETCD_ADVERTISE_CLIENT_URLS：廣播給集羣中其他成員自己的客戶端地址列表
啓動/停止/查看etcd
sudo systemctl start etcd
sudo systemctl stop etcd
sudo systemctl status etcd
設置開機啓動
sudo systemctl enable etcd
查看etcd啓動日誌
sudo journalctl -xef
查看etcd集羣節點
etcdctl member list
查看etcd集羣狀態
etcdctl cluster-health
在線安裝kubernetes
安裝kubernetes
1）master節點安裝
sudo yum install -y kubernetes-master
2）slave節點安裝
sudo yum install -y kubernetes-node
配置kubernetes
1）所有節點配置config
sudo vi /etc/kubernetes/config
/etc/kubernetes/config內容：
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="–logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="–v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="–allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="–master=http://192.168.245.227:8080"
2）master節點配置apiserver
sudo vi /etc/kubernetes/apiserver
/etc/kubernetes/apiserver內容：
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="–address=0.0.0.0"
# The port on the local server to listen on.
KUBE_API_PORT="–port=8080"
# Port minions listen on
KUBELET_PORT="–kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="–etcd-servers=http:// 192.168.245.227:2379, http://
192.168.245.228:2379,http:// 192.168.245.229:2379 "
# Address range to use for services
KUBE_SERVICE_ADDRESSES="–service-cluster-ip-range=172.16.0.0/24"
# default admission control policies
KUBE_ADMISSION_CONTROL="–admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota"
# Add your own!
KUBE_API_ARGS="–service-node-port-range=1-65535 --insecure-bind-address=0.0.0.0"
2）master節點配置controller-manager
sudo vi /etc/kubernetes/controller-manager
/etc/kubernetes/controller-manager內容：
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
3）master節點配置scheduler
sudo vi /etc/kubernetes/scheduler
/etc/kubernetes/scheduler：
###
# The following values are used to configure the kubernetes scheduler
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_SCHEDULER_ARGS=""
4）slave節點配置kubelet
sudo vi /etc/kubernetes/kubelet
/etc/kubernetes/kubelet內容：
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or “” for all
interfaces)
KUBELET_ADDRESS="–address=0.0.0.0"
# The port for the info server to serve on
# KUBELET_PORT="–port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="–hostname-override=192.168.245.228"
# location of the api-server
# KUBELET_API_SERVER="–api-servers=http://192.168.245.227:8080"
# pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="–pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="–kubeconfig=/etc/kubernetes/kubeconfig
–runtime-cgroups=/systemd/system.slice
–kubelet-cgroups=/systemd/system.slice --cluster-dns=172.16.0.0
–cgroup-driver=systemd --enable-server=true
–enable-debugging-handlers=true"
注意：
不同slave需修改KUBELET_HOSTNAME
這裏需要說明的是kubernetesV1.9
開始已經不支持–api-servers參數，需要指定-- kubeconfig=/etc/kubernetes/kubeconfig配置文件來指明master的地址，否則master主機執行kubectl get nodes，會提示找不到節點
5）slave節點配置kubeconfig
sudo vi /etc/kubernetes/kubeconfig
/etc/kubernetes/kubeconfig內容：
apiVersion: v1
kind: Config
users:
- name: kubelet
clusters:
- name: kubernetes
cluster:
server: http://192.168.245.227:8080
contexts:
- context:
cluster: kubernetes
user: kubelet
name: service-account-context
current-context: service-account-context
6）slave節點配置proxy
sudo vi /etc/kubernetes/proxy
/etc/kubernetes/proxy內容：
# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS="–bind-address=192.168.245.228
–hostname-override=192.168.245.228"
注意：不同slave需修改KUBE_PROXY_ARGS中的ip
7）修改docker cgroupdriver
sudo vi /etc/docker/daemon.json
/etc/docker/daemon.json內容：
{
“storage-driver”: “devicemapper”,
“storage-opts”: [
“dm.thinpooldev=/dev/mapper/docker-thinpool”,
“dm.use_deferred_removal=true”,
“dm.use_deferred_deletion=true”]
, “exec-opts”:[ “native.cgroupdriver=systemd” ]
}
master節點啓動服務
sudo systemctl daemon-reload
sudo systemctl start kube-apiserver
sudo systemctl start kube-controller-manager
sudo systemctl start kube-scheduler
master節點開機啓動
sudo systemctl enable kube-apiserver
sudo systemctl enable kube-controller-manager
sudo systemctl enable kube-scheduler
slave節點啓動服務
sudo systemctl daemon-reload
sudo systemctl start kubelet
sudo systemctl start kube-proxy
slave節點開機啓動
sudo systemctl enable kubelet
sudo systemctl enable kube-proxy
驗證kubernetes集羣
在master節點，執行
kubectl get nodes
在master節點，執行
kubectl get node --show-labels=true
在master節點，執行
kubectl get cs
離線安裝etcd
下載地址
http://www.rpmfind.net/linux/centos/7.5.1804/extras/x86_64/Packages/
安裝包
etcd-3.2.22-1.el7.x86_64.rpm
安裝etcd
進入到etcd安裝包和依賴包所在目錄執行命令：
sudo rpm -ivh *.rpm
配置etcd
相關配置參考etcd在線安裝
離線安裝kubernetes
下載地址
http://cbs.centos.org/repos/virt7-kubernetes-110-release/x86_64/os/Packages/
安裝包
kubernetes-master-1.10.3-0.el7.x86_64.rpm
kubernetes-client-1.10.3-0.el7.x86_64.rpm
kubernetes-node-1.10.3-0.el7.x86_64.rpm
依賴包
PyYAML-3.10-11.el7.x86_64.rpm
atomic-registries-1.22.1-22.git5a342e3.el7.x86_64.rpm
checkpolicy-2.5-6.el7.x86_64.rpm
conntrack-tools-1.4.4-3.el7_3.x86_64.rpm
audit-libs-python-2.8.1-3.el7_5.1.x86_64.rpm
container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch.rpm
container-selinux-2.68-1.el7.noarch.rpm
docker-client-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
docker-common-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm
libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm
libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm
libseccomp-2.3.1-3.el7.x86_64.rpm
libcgroup-0.41-15.el7.x86_64.rpm
libsemanage-python-2.5-11.el7.x86_64.rpm
docker-1.13.1-74.git6e3bb8e.el7.centos.x86_64.rpm
libyaml-0.1.4-11.el7_0.x86_64.rpm
oci-register-machine-0-6.git2b44233.el7.x86_64.rpm
oci-umount-2.3.3-3.gite3c9055.el7.x86_64.rpm
oci-systemd-hook-0.1.17-2.git83283a0.el7.x86_64.rpm
python-IPy-0.75-6.el7.noarch.rpm
policycoreutils-python-2.5-22.el7.x86_64.rpm
python-backports-1.0-8.el7.x86_64.rpm
python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch.rpm
python-ipaddress-1.0.16-2.el7.noarch.rpm
python-setuptools-0.9.8-7.el7.noarch.rpm
skopeo-containers-0.1.31-1.dev.gitae64ff7.el7.centos.x86_64.rpm
setools-libs-3.3.8-2.el7.x86_64.rpm
python-pytoml-0.1.14-1.git7dea353.el7.noarch.rpm
socat-1.7.3.2-2.el7.x86_64.rpm
subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64.rpm
yajl-2.0.4-4.el7.x86_64.rpm
安裝kubernetes
進入到kubernetes安裝包和依賴包所在目錄執行命令：
sudo rpm -ivh *.rpm
配置kubernetes
相關配置參考kubernetes在線安裝