如何在已有的K8S集羣中使用etcdctl工具操作etcd集羣

1、集羣信息

K8S集羣信息，集羣中有三個master節點

# kubectl get nodes
NAME               STATUS     ROLES    AGE   VERSION
k8s-m1             Ready      master   55d   v1.17.0
k8s-m2             Ready      master   55d   v1.17.0
k8s-m3             Ready      master   55d   v1.17.0

etcd集羣以pod方式運行在K8S集羣之上

# kubectl get pods -n kube-system -o wide
NAME                                       READY   STATUS           RESTARTS   AGE   IP              NODE               NOMINATED NODE   READINESS GATES
etcd-k8s-m1                                1/1     Running          44         55d   172.0.2.139     k8s-m1             <none>           <none>
etcd-k8s-m2                                1/1     Running          2          26m   172.0.2.146     k8s-m2             <none>           <none>
etcd-k8s-m3                                1/1     Running          3779       55d   172.0.2.234     k8s-m3             <none>           <none>

2、下載etcdctl

查看etcd版本，需要下載3.4.3版本的etcdctl

[root@k8s-m1 member]# kubectl describe pods etcd-k8s-m1 -n kube-system
Name:                 etcd-k8s-m1
Namespace:            kube-system
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 k8s-m1/172.0.2.139
Start Time:           Mon, 13 Apr 2020 02:28:39 -0400
Labels:               component=etcd
                      tier=control-plane
Annotations:          kubernetes.io/config.hash: 3d4819355a9752ba239aa13c1885dcc1
                      kubernetes.io/config.mirror: 3d4819355a9752ba239aa13c1885dcc1
                      kubernetes.io/config.seen: 2020-02-20T04:27:11.811231481-05:00
                      kubernetes.io/config.source: file
Status:               Running
IP:                   172.0.2.139
IPs:
  IP:           172.0.2.139
Controlled By:  Node/k8s-m1
Containers:
  etcd:
    Container ID:  docker://c8722c4def309777ca9be9fb7a273521f6fe3cb3195105a10121f22c24310fe6
    Image:         k8s.gcr.io/etcd:3.4.3-0

下載etcd版本，解壓，copy etcdctl到k8s master節點的/usr/bin目錄下

# wget https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz .

[root@k8s-m1 member]# ls -l /usr/bin/etcdctl
-rwxr-xr-x. 1 root root 17542688 Mar  4 03:09 /usr/bin/etcdctl
[root@k8s-m1 member]# etcdctl version
etcdctl version: 3.4.3
API version: 3.4

3、使用etcdctl

3.1、獲取etcd的endpoint

endpoint爲https://172.0.2.139:2379

# kubectl get pods etcd-k8s-m1 -o yaml -n kube-system
...
  containers:
  - command:
    - etcd
    - --advertise-client-urls=https://172.0.2.139:2379
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt
    - --client-cert-auth=true
    - --data-dir=/var/lib/etcd
    - --initial-advertise-peer-urls=https://172.0.2.139:2380
    - --initial-cluster=k8s-m1=https://172.0.2.139:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key
    - --listen-client-urls=https://127.0.0.1:2379,https://172.0.2.139:2379
    - --listen-metrics-urls=http://127.0.0.1:2381
    - --listen-peer-urls=https://172.0.2.139:2380
    - --name=k8s-m1
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    - --snapshot-count=10000
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    image: k8s.gcr.io/etcd:3.4.3-0
...

3.2、準備key和cert

etcd的endpoint爲https方式，所以要爲etcdctl命令準備key和cert
在3.1節輸出信息中：
key使用/etc/kubernetes/pki/etcd/peer.key
cert使用/etc/kubernetes/pki/etcd/peer.crt

3.3、執行etcdctl命令

# etcdctl --cert /etc/kubernetes/pki/etcd/peer.crt --key /etc/kubernetes/pki/etcd/peer.key  --endpoints https://172.0.2.139:2379 --insecure-skip-tls-verify  member list
1e2fb9983e528532, started, k8s-m2, https://172.0.2.146:2380, https://172.0.2.146:2379, false
947c9889866d299a, started, k8s-m3, https://172.0.2.234:2380, https://172.0.2.234:2379, false
e97c0cc82d69a534, started, k8s-m1, https://172.0.2.139:2380, https://172.0.2.139:2379, false

注意：因爲集羣證書爲自簽發，所以這裏需要加上–insecure-skip-tls-verify參數，不然會報如下錯誤

# etcdctl --cert /etc/kubernetes/pki/etcd/peer.crt --key /etc/kubernetes/pki/etcd/peer.key  --endpoints https://172.0.2.139:2379  member list
{"level":"warn","ts":"2020-04-16T05:00:52.085-0400","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-c086c9e1-cb96-4c26-890e-b311b761b2c3/172.0.2.139:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate signed by unknown authority\""}
Error: context deadline exceeded