kubernetes-1.11.0集羣部署之etcd集羣 (一)

原創 wangxing0122 2019-02-22 16:03

kubernetes-1.11.0集羣部署之etcd集羣 (一)

kubernetes-1.11.0集羣部署之etcd集羣 (一)

參考文檔文檔編譯

[https://github.com/kubernetes/kubernetes/tree/release-1.11/cluster/images/hyperkube]()

參考文檔文檔編譯

環境安裝

1. docker install
   所有節點都按照docker  
   yum install -y yum-utils   device-mapper-persistent-data   lvm2
   yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
   yum list docker-ce --showduplicates | sort -r  

   安裝docker-ce-17.03.2.ce 不能直接yum安裝，需要安裝docker-ce-selinux-17.03.2.ce
   wget https://download.docker.com/linux/centos/7/x86_64/stackages/docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch.rpm       

   yum install docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch.rpm  docker-ce-17.03.2.ce  -y  

   mkdir /etc/docker/  

   vim /etc/docker/daemon.json 

        {
         "storage-driver": "overlay2",
         "storage-opts": [
         "overlay2.override_kernel_check=true"
         ]
       }

    啓動docker  
    systemctl restart docker  

2. 安裝golang
    10.39.1.43上安裝golang 
    wget https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz 
    tar -C /usr/local/ -xzf go1.10.3.linux-amd64.tar.gz  
    vim /etc/profile 
    export PATH=$PATH:/usr/local/go/bin

    設置GOROOT  GOPATH 
    export GOROOT=$HOME/go1.
    export PATH=$PATH:$GOROOT/bin 
    cd $GOPATH  &&  mkdir  {pkg,src,bin,lib}
    go  version 

3. kubernetes 編譯      

   下載kubernetes 源碼  
   cd $GOPATH/src 
   git clone https://github.com/kubernetes/kubernetes.git 
   git tag 
   git checkout  v1.11.0  
   git branch -v

4. 編譯

   選擇本地go編譯，編譯二進制  
   KUBE_BUILD_PLATFORMS=linux/amd64 make all  
   編譯完成後，生成的二進制文件，會生成一個_output 目錄
 ![](https://s1.51cto.com/images/blog/201809/03/5419919a03c455be4fd732f50ae43faf.png?x-oss-process=image/watermark,size_16,text_QDUxQ1RP5Y2a5a6i,color_FFFFFF,t_100,g_se,x_10,y_10,shadow_90,type_ZmFuZ3poZW5naGVpdGk=)

   版本更新說明

[https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1110]()

5. 機器準備
   三臺master， 一臺node 
   主機              主機名 
    10.39.10.160     kubernetes-node-160
    10.39.10.159     kubernetes-master-159
    10.39.10.156     kubernetes-master-156
    10.39.10.154     kubernetes-master-154

    添加hosts 
   系統
   CentOS Linux release 7.5.1804  3.10.0-862.9.1.el7.x86_64 

6. 創建證書 

   使用CloudFlare的pki 工具來生成CA證書

[https://github.com/cloudflare/cfssl]()

   install cfssl       
    mkdir -p /opt/local/cfssl
   cd /opt/local/cfssl

   wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
   mv cfssl_linux-amd64 cfssl

   wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
   mv cfssljson_linux-amd64 cfssljson

   wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
   mv cfssl-certinfo_linux-amd64 cfssl-certinfo

   chmod +x *          

   創建CA 證書
   mkdir /opt/k8s/ssl
   config.json 文件 
   vim config.json 
   {
     "signing": {
     "default": {
      "expiry": "87600h"
     },
     "profiles": {
       "kubernetes": {
          "usages": [
             "signing",
             "key encipherment",
             "server auth",
             "client auth"
          ],
        "expiry": "87600h"
      }
     }
  }
 }

vim csr.json
   {
   "CN": "kubernetes",
   "key": {
      "algo": "rsa",
      "size": 2048
   },
  "names": [
{
  "C": "CN",
  "ST": "BeiJing",
  "L": "BeiJing",
  "O": "k8s",
  "OU": "System"
     }
 ]
} 

生成CA 證書和私鑰 
cfssl gencert -initca  csr.json | cfssljson -bare ca

kubernetes-1.11.0集羣部署之etcd集羣 (一)

分發證書
 mkdir -p /etc/kubernetes/ssl   拷貝到kubernetes所有機器上
 ansible -i hosts k8s -m shell -a "mkdir /etc/kubernetes/ssl  -p"
 ansible -i hosts  k8s -m copy -a "src=/opt/k8s/ssl  dest=/etc/kubernetes/"

etcd 集羣

  最新的kubernetes v1.11.0 支持的etcd 版本是v3.2.18 版本 
  wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz
  tar zxvf etcd-v3.2.18-linux-amd64.tar.gz
  cd etcd-v3.2.18-linux-amd64
  ansible -i hosts  k8s -m copy -a "src=etcd-v3.2.18-linux-amd64/etcd  dest=/usr/bin/"
  ansible -i hosts  k8s -m copy -a "src=etcd-v3.2.18-linux-amd64/etcdctl  dest=/usr/bin/"

 創建etcd證書

 cd /opt/k8s/ssl 
 vim etcd-csr.json
   {
    "CN": "etcd",
    "hosts": [
    "127.0.0.1",
    "10.39.10.154",
    "10.39.10.156",
    "10.39.10.159"
     ],
    "key": {
    "algo": "rsa",
    "size": 2048
    },
    "names": [
    {
  "C": "CN",
  "ST": "BeiJing",
  "L": "BeiJing",
  "O": "k8s",
  "OU": "System"
   }
  ]
 }     

 生成etcd  
 cfssl gencert -ca=ca.pem  -ca-key=ca-key.pem  -config=config.json   -profile=kubernetes etcd-csr.json | cfssljson -bare etcd

kubernetes-1.11.0集羣部署之etcd集羣 (一)

查看證書
cfssl-certinfo -cert etcd.pem  

拷貝證書到etcd 服務器

etcd-1

 scp  etcd*.pem   [email protected]:/etc/kubernetes/ssl/

etcd-2

 scp  etcd*.pem   [email protected]:/etc/kubernetes/ssl/

etcd-3

 scp  etcd*.pem   [email protected]:/etc/kubernetes/ssl/

如果etcd 非root 用戶，讀取證書會提示沒有權限
chmod 644 /etc/kubernetes/ssl/etcd-key.pem 

ansible -i hosts  etcd -m shell -a "chmod 644 /etc/kubernetes/ssl/etcd-key.pem"

etcd 配置

創建etcd 的數據存儲目錄，默認是在/var/lib/etcd/

useradd  etcd  

mkdir /data/etcd -p 
ansible -i hosts  etcd -m shell -a "mkdir /data/etcd -p"
chown -R etcd:etcd /data/etcd

chmod +x /usr/bin/etcd 
chmod +x /usr/bin/etcdctl

etcd-1

vim  /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
WorkingDirectory=/data/etcd/
User=etcd
# set GOMAXPROCS to number of processors
ExecStart=/usr/bin/etcd \
--name=etcd1 \
--cert-file=/etc/kubernetes/ssl/etcd.pem \
--key-file=/etc/kubernetes/ssl/etcd-key.pem \
--peer-cert-file=/etc/kubernetes/ssl/etcd.pem \
--peer-key-file=/etc/kubernetes/ssl/etcd-key.pem \
--trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
--peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
--initial-advertise-peer-urls=https://10.39.10.154:2380 \
--listen-peer-urls=https://10.39.10.154:2380 \
--listen-client-urls=https://10.39.10.154:2379,http://127.0.0.1:2379 \
--advertise-client-urls=https://10.39.10.154:2379 \
--initial-cluster-token=k8s-etcd-cluster \
--initial-cluster=etcd1=https://10.39.10.154:2380,etcd2=https://10.39.10.156:2380,etcd3=https://10.39.10.159:2380 \
--initial-cluster-state=new \
--data-dir=/data/etcd/
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

etcd 2

 vim  /etc/systemd/system/etcd.service
 [Unit]
 Description=Etcd Server
 After=network.target
 After=network-online.target
 Wants=network-online.target

 [Service]
 Type=notify
 WorkingDirectory=/data/etcd/
 User=etcd
 # set GOMAXPROCS to number of processors
 ExecStart=/usr/bin/etcd \
 --name=etcd2 \
 --cert-file=/etc/kubernetes/ssl/etcd.pem \
 --key-file=/etc/kubernetes/ssl/etcd-key.pem \
 --peer-cert-file=/etc/kubernetes/ssl/etcd.pem \
 --peer-key-file=/etc/kubernetes/ssl/etcd-key.pem \
 --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
 --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
 --initial-advertise-peer-urls=https://10.39.10.156:2380 \
 --listen-peer-urls=https://10.39.10.156:2380 \
 --listen-client-urls=https://10.39.10.156:2379,http://127.0.0.1:2379 \
 --advertise-client-urls=https://10.39.10.156:2379 \
 --initial-cluster-token=k8s-etcd-cluster \
 --initial-cluster=etcd1=https://10.39.10.154:2380,etcd2=https://10.39.10.156:2380,etcd3=https://10.39.10.159:2380 \
 --initial-cluster-state=new \
 --data-dir=/data/etcd
 Restart=on-failure
 RestartSec=5
 LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

etcd-3

vim /etc/systemd/system/etcd.service  
 [Unit]
 Description=Etcd Server
 After=network.target
 After=network-online.target
 Wants=network-online.target

 [Service]
 Type=notify
 WorkingDirectory=/data/etcd/
 User=etcd
 # set GOMAXPROCS to number of processors
 ExecStart=/usr/bin/etcd \
 --name=etcd3 \
 --cert-file=/etc/kubernetes/ssl/etcd.pem \
 --key-file=/etc/kubernetes/ssl/etcd-key.pem \
 --peer-cert-file=/etc/kubernetes/ssl/etcd.pem \
 --peer-key-file=/etc/kubernetes/ssl/etcd-key.pem \
 --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
 --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
 --initial-advertise-peer-urls=https://10.39.10.159:2380 \
 --listen-peer-urls=https://10.39.10.159:2380 \
 --listen-client-urls=https://10.39.10.159:2379,http://127.0.0.1:2379 \
 --advertise-client-urls=https://10.39.10.159:2379 \
 --initial-cluster-token=k8s-etcd-cluster \
 --initial-cluster=etcd1=https://10.39.10.154:2380,etcd2=https://10.39.10.156:2380,etcd3=https://10.39.10.159:2380 \
 --initial-cluster-state=new \
 --data-dir=/data/etcd
 Restart=on-failure
 RestartSec=5
 LimitNOFILE=65536

 [Install]
 WantedBy=multi-user.target

啓動etcd

systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
systemctl status etcd

查看etcd 集羣狀態 
etcdctl --endpoints=https://10.39.10.154:2379,https://10.39.10.156:2379,https://10.39.10.159:2379\
    --cert-file=/etc/kubernetes/ssl/etcd.pem \
    --ca-file=/etc/kubernetes/ssl/ca.pem \
    --key-file=/etc/kubernetes/ssl/etcd-key.pem \
    cluster-health

查看etcd 集羣成員
 etcdctl --endpoints=https://10.39.10.154:2379,https://10.39.10.156:2379,https://10.39.10.159:2379\
    --cert-file=/etc/kubernetes/ssl/etcd.pem \
    --ca-file=/etc/kubernetes/ssl/ca.pem \
    --key-file=/etc/kubernetes/ssl/etcd-key.pem \
    member list

kubernetes-1.11.0集羣部署之etcd集羣 (一)

發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

清理k8s集羣小記（只針對自己的集羣）

moakia 2019-02-23 13:53:44

Kubernetes1.13.1部署Kuberneted-dashboard v1.10.1

三杯水 2019-02-23 00:15:26

kubernetes實戰-基於redis和docker的留言簿案例

msj0905 2019-02-22 23:48:21

『高級篇』docker之kubernetes環境搭建與預先環境準備（32）

IT人故事 2019-02-22 23:28:21

『高級篇』docker之瞭解kubernetes（31）

IT人故事 2019-02-22 23:28:14

時速雲企業級容器PaaS技術沙龍 第十期【上海站】

tenxcloud 2019-02-22 23:21:16

Kubernetes 1.11 手動安裝並啓用ipvs

三和梁朝偉 2019-02-22 23:12:31

Centos7部署Kubernetes

Linux小全 2019-02-22 22:47:53

Kubernetes-先把kubernetes跑起來

Linux小全 2019-02-22 22:47:52

Kubernetes-重要概述

Linux小全 2019-02-22 22:47:52

docker 實踐（十三）k8s 安裝部署

ckl893 2019-02-22 22:45:36

docker 實踐（十二）k8s 初體驗

ckl893 2019-02-22 22:45:34

git-jenkins-k8s構建tomcat項目實踐

jiangxianliang 2019-02-22 22:31:06

openshift基礎概念

毒死人了 2019-02-22 21:50:40

k8s一鍵安裝腳本(二進制)

哈哈怪1111 2019-02-22 21:49:47