前言
Kubernetes 系統使用 Etcd 存儲集羣配置和一些狀態信息。如果單一部署，etcd如果掛了會導致集羣異常甚至集羣崩潰無法恢復。所以單點部署是很風險的。所以我們要部署Etcd集羣
環境說明：
etcd1 192.168.214.200
etcd2 192.168.214.201
etcd3 192.168.214.202
三臺節點創建相關目錄
[root@etcd1 ~]# mkdir -p /data/etcd
[root@etcd1 ~]# mkdir -p /opt/kubernetes/{bin,conf,ssl}將我們之前生成的證書分發到節點的/opt/kubernetes/ssl/目錄下，詳見上篇文檔《證書詳解》
etcd集羣用到的證書爲ca.pem、kubernetes-key.pem、kubernetes.pem
[root@master1 ssl]# scp *.pem etcd1:/opt/kubernetes/ssl [root@master1 ssl]# scp *.pem etcd2:/opt/kubernetes/ssl [root@master1 ssl]# scp *.pem etcd3:/opt/kubernetes/ssl
三臺etcd節點下載並解壓二進制文件
[root@etcd1 ~]# wget https://github.com/coreos/etcd/releases/download/v3.2.11/etcd-v3.2.11-linux-amd64.tar.gz [root@etcd1 ~]# tar xzvf etcd-v3.2.11-linux-amd64.tar.gz [root@etcd1 ~]# cp etcd-v3.2.11-linux-amd64/etcd* /opt/kubernetes/bin/
創建etcd.service文件
在/usr/lib/systemd/system/下創建etcd.service
[root@etcd1 system]# vim etcd.service [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target Documentation=https://github.com/coreos [Service] Type=notify WorkingDirectory=/data/etcd/ EnvironmentFile=-/opt/kubernetes/conf/etcd.conf ExecStart=/opt/kubernetes/bin/etcd \ --name=etcd1 \ --cert-file=/opt/kubernetes/ssl/kubernetes.pem \ --key-file=/opt/kubernetes/ssl/kubernetes-key.pem \ --peer-cert-file=/opt/kubernetes/ssl/kubernetes.pem \ --peer-key-file=/opt/kubernetes/ssl/kubernetes-key.pem \ --trusted-ca-file=/opt/kubernetes/ssl/ca.pem \ --peer-trusted-ca-file=/opt/kubernetes/ssl/ca.pem \ --initial-advertise-peer-urls=https://192.168.214.200:2380 \ --listen-peer-urls=https://192.168.214.200:2380 \ --listen-client-urls=https://192.168.214.200:2379,http://127.0.0.1:2379 \ --advertise-client-urls=https://192.168.214.200:2379 \ --initial-cluster-token=etcd-cluster-0 \ --initial-cluster=etcd1=https://192.168.214.200:2380,etcd2=https://192.168.214.201:2380,etcd3=https://192.168.214.202:2380 \ --initial-cluster-state=new \ --data-dir=/data/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target
說明：
自行根據節點IP設置其他節點
指定 etcd 的工作目錄和數據目錄爲 /data/etcd，需在啓動服務前創建這個目錄；
--initial-cluster-state 值爲 new 時，--name 的參數值必須位於 --initial-cluster 列表中；
爲了保證通信安全，需要指定 etcd 的公私鑰(cert-file和key-file)、Peers 通信的公私鑰和 CA 證書(peer-cert-file、peer-key-file、peer-trusted-ca-file)、客戶端的CA證書（trusted-ca-file）；
增加環境變量
[root@etcd1 ~]# vim .bash_profile export PATH=/opt/kubernetes/bin:$PATH [root@etcd1 ~]# source .bash_profile
啓動etcd服務
[root@etcd1 ~]# systemctl daemon-reload [root@etcd1 ~]# systemctl enable etcd [root@etcd1 ~]# systemctl start etcd
集羣驗證
（1）驗證集羣狀態
[root@etcd1 ssl]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem cluster-health member 31a87df7577ee4e3 is healthy: got healthy result from https://192.168.214.201:2379 member d009f1b31e51b9c3 is healthy: got healthy result from https://192.168.214.202:2379 member fb94879e6d597fdf is healthy: got healthy result from https://192.168.214.200:2379 cluster is healthy
（2）通過集羣節點etcd1設置一個值，到etcd2和etcd3獲取，如果獲取正常，則存儲也沒問題
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem set dong "diss and peace" diss and peace [root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem get dong diss and peace [root@etcd3 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem get dong diss and peace
（3）可以通過etcdctl member list查看誰是當前集羣中選舉出的leader
[root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem member list 31a87df7577ee4e3: name=etcd2 peerURLs=https://192.168.214.201:2380 clientURLs=https://192.168.214.201:2379 isLeader=false d009f1b31e51b9c3: name=etcd3 peerURLs=https://192.168.214.202:2380 clientURLs=https://192.168.214.202:2379 isLeader=false fb94879e6d597fdf: name=etcd1 peerURLs=https://192.168.214.200:2380 clientURLs=https://192.168.214.200:2379 isLeader=true
（4）驗證leader切換。上步可以看出此時的leader爲etcd1，停掉etcd1後，再次查看，leader變爲了etcd2
[root@etcd1 ~]# systemctl stop etcd [root@etcd2 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem member list 31a87df7577ee4e3: name=etcd2 peerURLs=https://192.168.214.201:2380 clientURLs=https://192.168.214.201:2379 isLeader=true d009f1b31e51b9c3: name=etcd3 peerURLs=https://192.168.214.202:2380 clientURLs=https://192.168.214.202:2379 isLeader=false fb94879e6d597fdf: name=etcd1 peerURLs=https://192.168.214.200:2380 clientURLs=https://192.168.214.200:2379 isLeader=false
創建kubernetes網段，用於後續的flannel請求調用，通過其他etcd集羣節點如果可以查看到這個網段，說明網段創建成功。
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem mk /kubernetes/network/config '{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }}'
{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }} #mk創建一個鍵值存儲，同時會自動創建鍵值目錄
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem ls -r
/dong #ls -r列出鍵值存儲的目錄
/kubernetes
/kubernetes/network
/kubernetes/network/config
[root@etcd1 ~]# etcdctl --ca-file=/opt/kubernetes/ssl/ca.pem --cert-file=/opt/kubernetes/ssl/kubernetes.pem --key-file=/opt/kubernetes/ssl/kubernetes-key.pem get /kubernetes/network/config
{ "Network": "172.20.0.0/16", "Backend": { "Type": "vxlan", "VNI": 1 }} #get查看鍵值存儲