主機列表
本次實驗選擇5臺主機,3臺作爲master主機,2臺作爲node節點
節點ip | OS版本 | hostname -f | 安裝軟件 |
---|---|---|---|
192.168.0.1 | RHEL7.4 | k8s-master01 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.2 | RHEL7.4 | k8s-master02 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.3 | RHEL7.4 | k8s-master03 | docker,etcd,flanneld,kube-apiserver,kube-controller-manager,kube-scheduler |
192.168.0.4 | RHEL7.4 | k8s-node01 | docker,flanneld,kubelet,kube-proxy |
192.168.0.5 | RHEL7.4 | k8s-node02 | docker,flanneld,kubelet,kube-proxy |
下載安裝包(etcd最新版本3.4.0)
wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar -xvf etcd-v3.3.10-linux-amd64.tar.gz
cp etcd etcdctl /k8s/etcd/bin/
修改配置文件
cat << EOF > /k8s/etcd/cfg/etcd
#[Member]
ETCD_NAME="etcd01"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.0.1:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.0.1:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.0.1:2379"
ETCD_INITIAL_CLUSTER="etcd01=https://192.168.0.1:2380,etcd02=https://192.168.0.2:2380,etcd03=https://192.168.0.3:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_ENABLE_V2="true"
EOF
提示:
- 其他etcd節點按照以上配置文件修改其中紅字部分即可
- flannel操作etcd使用的是v2的API,而kubernetes操作etcd使用的v3的API,爲了兼容flannel,將默認開啓v2版本,故配置文件中設置 ETCD_ENABLE_V2="true"
創建TLS 密鑰和證書
爲了保證通信安全,客戶端(如etcdctl)與etcd 集羣、etcd 集羣之間的通信需要使用TLS 加密。
創建etcd 證書籤名請求:
cat > etcd-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"192.168.0.1",
"192.168.0.2",
"192.168.0.3"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
EOF
生成etcd證書和私鑰
# cfssl gencert -ca=/k8s/kubernetes/ssl/ca.pem -ca-key=/k8s/kubernetes/ssl/ca-key.pem -config=/k8s/kubernetes/ssl/ca-config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
# ls etcd*
etcd.csr etcd-csr.json etcd-key.pem etcd.pem
創建 etcd的 systemd unit 文件
cat << EOF > /lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target[Service]
Type=notify
EnvironmentFile=/k8s/etcd/cfg/etcd
ExecStart=/k8s/etcd/bin/etcd \
--cert-file=/k8s/etcd/ssl/etcd.pem \
--key-file=/k8s/etcd/ssl/etcd-key.pem \
--peer-cert-file=/k8s/etcd/ssl/etcd.pem \
--peer-key-file=/k8s/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/k8s/kubernetes/ssl/ca.pem \
--peer-trusted-ca-file=/k8s/kubernetes/ssl/ca.pem
Restart=on-failure
LimitNOFILE=65536[Install]
WantedBy=multi-user.target
EOF
分發文件至其他節點
cd /k8s/
scp -r etcd/ 192.168.0.2:/k8s/
scp -r etcd/ 192.168.0.3:/k8s/scp /lib/systemd/system/etcd.service 192.168.0.2:/lib/systemd/system/etcd.service
scp /lib/systemd/system/etcd.service 192.168.0.3:/lib/systemd/system/etcd.service
啓動etcd 服務
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
# /k8s/etcd/bin/etcdctl --cacert=/k8s/kubernetes/ssl/ca.pem --cert=/k8s/etcd/ssl/etcd.pem --key=/k8s/etcd/ssl/etcd-key.pem --endpoints="https://192.168.0.3:2379,https://192.168.0.2:2379,https://192.168.0.1:2379" endpoint health
https://192.168.0.2:2379 is healthy: successfully committed proposal: took = 24.271259ms
https://192.168.0.3:2379 is healthy: successfully committed proposal: took = 31.633027ms
https://192.168.0.1:2379 is healthy: successfully committed proposal: took = 37.463262ms
etcd 3.4注意事項
- ETCD3.4版本ETCDCTL_API=3 etcdctl 和 etcd --enable-v2=false 成爲了默認配置,如要使用v2版本,執行etcdctl時候需要設置ETCDCTL_API環境變量,例如:ETCDCTL_API=2 etcdctl
- ETCD3.4版本會自動讀取環境變量的參數,所以EnvironmentFile文件中有的參數,不需要再次在ExecStart啓動參數中添加,二選一,如同時配置,會觸發以下類似報錯“etcd: conflicting environment variable "ETCD_NAME" is shadowed by corresponding command-line flag (either unset environment variable or disable flag)”
- flannel操作etcd使用的是v2的API,而kubernetes操作etcd使用的v3的API