kubernetes-----部署多master的二进制集群

目录

一.多master的二进制集群分析

二.实验环境分析

三.具体部署

搭建k8s的单节点集群

搭建master2节点

部署负载均衡

修改node的VIP以及pod的创建

搭建k8s的Dashboard

---

一.多master的二进制集群分析

• 区别於单master的二进制集群，多master集群对master做了一个高可用，如果master1宕机，Load Balance就会将VIP转移到master2，这样就保证了master的可靠性。
• 多节点的核心点就是需要指向一个核心的地址，我们之前在做单节点的时候已经将vip地址定义过写入k8s-cert.sh脚本文件中（192.168.18.100），vip开启apiserver，多master开启端口接受node节点的apiserver请求，此时若有新的节点加入，不是直接找moster节点，而是直接找到vip进行spiserver的请求，然后vip再进行调度，分发到某一个master中进行执行，此时master收到请求之后就会给改node节点颁发证书
   

二.实验环境分析

角色	IP地址	系统与资源	相关组件
master1	192.168.43.101/24	centos7.4(2C 2G)	kube-apiserver kube-controller-manager kube-scheduler etcd
master2	192.168.43.104/24	centos7.4(2C 2G)	kube-apiserver kube-controller-manager kube-scheduler
node1	192.168.43.102/24	centos7.4(2C 2G)	kubelet kube-proxy docker flannel etcd
node2	192.168.43.103/24	centos7.4(2C 2G)	kubelet kube-proxy docker flannel etcd
nginx_lbm	192.168.43.105/24	centos7.4(2C 2G)	nginx keepalived
nginx_lbb	192.168.43.106/24	centos7.4(2C 2G)	nginx keepalived
VIP	192.168.43.100/24	-	-

• 本实验基於单master基础之上操作，添加一个master2
• 利用nginx做负载均衡，利用keepalived做负载均衡器的高可用

注：1.9版本之后nginx具有了四层转发的功能（负载均衡），多了stream模块

• 利用keepalived给master提供的虚拟IP地址，给node访问

三.具体部署

搭建k8s的单节点集群

• 参考，单master集群部署

搭建master2节点

master1的操作

• 复制相关文件、脚本

##递归复制/opt/kubernetes和/opt/etcd下的所有文件到master中
[root@master ~]# scp -r /opt/kubernetes/ [email protected]:/opt/
The authenticity of host '192.168.43.104 (192.168.43.104)' can't be established.
ECDSA key fingerprint is SHA256:AJdR3BBN9kCSEk3AVfaZuyrxhNMoDnzGMOMWlP1gUaQ.
ECDSA key fingerprint is MD5:d4:ab:7b:82:c3:99:b8:5d:61:f2:dc:af:06:38:e7:6c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.43.104' (ECDSA) to the list of known hosts.
[email protected]'s password: 
token.csv                                                 100%   84     5.2KB/s   00:00    
kube-apiserver                                            100%  934   353.2KB/s   00:00    
kube-scheduler                                            100%   94    41.2KB/s   00:00    
kube-controller-manager                                   100%  483   231.5KB/s   00:00    
kube-apiserver                                            100%  184MB  19.4MB/s   00:09    
kubectl                                                   100%   55MB  24.4MB/s   00:02    
kube-controller-manager                                   100%  155MB  26.7MB/s   00:05    
kube-scheduler                                            100%   55MB  31.1MB/s   00:01    
ca-key.pem                                                100% 1679   126.0KB/s   00:00    
ca.pem                                                    100% 1359   514.8KB/s   00:00    
server-key.pem                                            100% 1675   501.4KB/s   00:00    
server.pem                                                100% 1643   649.4KB/s   00:00    

##master2中需要etcd的证书，否则apiserver无法启动
[root@master ~]# scp -r /opt/etcd/ [email protected]:/opt/
[email protected]'s password: 
etcd                                                      100%  516    64.2KB/s   00:00    
etcd                                                      100%   18MB  25.7MB/s   00:00    
etcdctl                                                   100%   15MB  25.9MB/s   00:00    
ca-key.pem                                                100% 1675   118.8KB/s   00:00    
ca.pem                                                    100% 1265   603.2KB/s   00:00    
server-key.pem                                            100% 1675   675.3KB/s   00:00    
server.pem                                                100% 1338   251.5KB/s   00:00    
[root@master ~]# 

##复制执行脚本到master中
[root@master ~]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service [email protected]:/usr/lib/systemd/system/
[email protected]'s password: 
kube-apiserver.service                                    100%  282    30.3KB/s   00:00    
kube-controller-manager.service                           100%  317    45.9KB/s   00:00    
kube-scheduler.service                                    100%  281   151.7KB/s   00:00    

master2的操作

• 基本环境设置

##修改主机名
[root@localhost ~]# hostnamectl set-hostname master2
[root@localhost ~]# su

##永久关闭安全性功能
[root@master2 ~]# systemctl stop firewalld
[root@master2 ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@master2 ~]# setenforce 0
[root@master2 ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config 

• 修改kube-apiserver中的IP地址

[root@master2 ~]# cd /opt/kubernetes/cfg/
[root@master2 cfg]# ls
kube-apiserver  kube-controller-manager  kube-scheduler  token.csv
[root@master2 cfg]# vi kube-apiserver 


KUBE_APISERVER_OPTS="--logtostderr=true \
--v=4 \
--etcd-servers=https://192.168.43.101:2379,https://192.168.43.102:2379,https://192.168.43.103:2379 \
##修改bind地址，绑定本地地址
--bind-address=192.168.43.104 \
--secure-port=6443 \
##修改对外展示的地址
--advertise-address=192.168.43.104 \
--allow-privileged=true \
--service-cluster-ip-range=10.0.0.0/24 \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \
--authorization-mode=RBAC,Node \
--kubelet-https=true \
--enable-bootstrap-token-auth \
--token-auth-file=/opt/kubernetes/cfg/token.csv \
--service-node-port-range=30000-50000 \
--tls-cert-file=/opt/kubernetes/ssl/server.pem  \
--tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \
--client-ca-file=/opt/kubernetes/ssl/ca.pem \
--service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \
--etcd-cafile=/opt/etcd/ssl/ca.pem \
--etcd-certfile=/opt/etcd/ssl/server.pem \
--etcd-keyfile=/opt/etcd/ssl/server-key.pem"

• 开启服务，并且验证

##开启apieserver服务
[root@master2 cfg]# systemctl start kube-apiserver.service 
[root@master2 cfg]# systemctl enable kube-apiserver.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.

##开启控制管理器服务
[root@master2 cfg]# systemctl start kube-controller-manager.service 
[root@master2 cfg]# systemctl enable kube-controller-manager.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.

##开启调度器服务
[root@master2 cfg]# systemctl start kube-scheduler.service 
[root@master2 cfg]# systemctl enable kube-scheduler.service 
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service.

##将执行脚本添加入全局变量
[root@master2 cfg]# echo "export PATH=$PATH:/opt/kubernetes/bin/" >> /etc/profile
[root@master2 cfg]# source /etc/profile

##查看集群节点，说明master2添加成功
[root@master2 cfg]# kubectl get node
NAME             STATUS   ROLES    AGE   VERSION
192.168.43.102   Ready    <none>   26h   v1.12.3
192.168.43.103   Ready    <none>   26h   v1.12.3
[root@master2 cfg]# 

注：能够添加master节点的前提是在部署单节点时，在server-csr.json中指定添加的地址，要不然生成不了证书，也就添加不了

部署负载均衡

以下操作在nginx_lbm和nginx_lbb中都操作，并且以nginx_lbm为例

• 编辑keepalived的配置文件的模板

##keepalibed.conf到nginx_lbm和nginx_lbb
[root@nginx_lbm ~]# ls
anaconda-ks.cfg  initial-setup-ks.cfg  keepalived.conf  公共  模板  视频  图片  文档  下载  音乐  桌面
[root@nginx_lbm ~]# cat keepalived.conf 
! Configuration File for keepalived 
 
global_defs { 
   # 接收邮件地址 
   notification_email { 
     [email protected] 
     [email protected] 
     [email protected] 
   } 
   # 邮件发送地址 
   notification_email_from [email protected]  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 

vrrp_script check_nginx {
    script "/usr/local/nginx/sbin/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state MASTER 
    interface eth0
    virtual_router_id 51 # VRRP 路由 ID实例，每个实例是唯一的 
    priority 100    # 优先级，备服务器设置 90 
    advert_int 1    # 指定VRRP 心跳包通告间隔时间，默认1秒 
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        10.0.0.188/24 
    } 
    track_script {
        check_nginx
    } 
}

[root@nginx_lbm ~]# 

• 关闭安全性功能

systemctl stop firewalld.service
setenforce 0

• 编辑nginx的源，并且安装nginx

[root@nginx_lbm ~]# cat /etc/yum.repos.d/nginx.repo 
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0

##重新加载yum 仓库
[root@nginx_lbm ~]# yum list
##安装nginx
[root@nginx_lbm ~]# yum install nginx -y

• 编辑nginx配置文件，添加负载均衡功能，并且启动服务

##配置负载均衡功能
[root@nginx_lbm ~]# vi /etc/nginx/nginx.conf 
##在第12行以下插入以下内容
     12 stream {
     13 
     14    log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
     15     access_log  /var/log/nginx/k8s-access.log  main;
     16 
     17     upstream k8s-apiserver {
     18 #此处为master1的ip地址
     19         server 192.168.43.101:6443;
     20 #此处为master2的ip地址
     21         server 192.168.43.102:6443;
     22     }
     23     server {
     24                 listen 6443;
     25                 proxy_pass k8s-apiserver;
     26     }
     27     }

##检查配置文件
[root@nginx_lbm ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

##编辑主页面，以区别master和backup
[root@nginx_lbm ~]# cd /usr/share/nginx/html/
[root@nginx_lbm html]# ls
50x.html  index.html
[root@nginx_lbm html]# vi index.html
<h1>master</h1>或者<h1>backup</h1>

##开启nginx服务
[root@nginx_lbm html]# systemctl start nginx

##安装keepalived服务
[root@nginx_lbm html]# yum install keeepalived -y
##覆盖keepalived的配置文件
[root@nginx_lbm ~]# cp keepalived.conf /etc/keepalived/keepalived.conf 
cp：是否覆盖"/etc/keepalived/keepalived.conf"？ yes

nginx_lbm的操作

• 配置keepalived服务

[root@nginx_lbm ~]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived 
 
global_defs { 
   # 接收邮件地址 
   notification_email { 
     [email protected] 
     [email protected] 
     [email protected] 
   } 
   # 邮件发送地址 
   notification_email_from [email protected]  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 
##指定keepalived服务关闭脚本
vrrp_script check_nginx {
    script "/etc/nginx/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state MASTER         ##在nginx_lbm,设置为master
    interface ens33        ##指定网卡名
    virtual_router_id 51     ##24行,vrrp路由ID实例，每个实例是唯一的
    priority 100        ##在master中优先级为100，backup优先级为90
    advert_int 1    
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        192.168.43.100/24         ##指定VIP
    } 
    track_script {
        check_nginx
    } 
}

[root@nginx_lbm ~]# vi /etc/nginx/check_nginx.sh  ##这个keepalived服务关闭脚本需要自行创建
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")	
#统计数量

if [ "$count" -eq 0 ];then
    systemctl stop keepalived
fi
#匹配为0，关闭keepalived服务

[root@nginx_lbm ~]# chmod +x /etc/nginx/check_nginx.sh        ##添加权限

##启动keepalived服务
[root@nginx_lbm ~]# systemctl start keepalived.service

• 查看vip

[root@nginx_lbm ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:92:43:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.43.105/24 brd 192.168.43.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.43.100/24 scope global secondary ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ba5a:8436:895c:4285/64 scope link 
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff

nginx_lbb的操作

• 配置keepalived服务

##编辑keepalived的配置文件
[root@nginx_lbb ~]# vi /etc/keepalived/keepalived.conf 
! Configuration File for keepalived 
 
global_defs { 
   # 接收邮件地址 
   notification_email { 
     [email protected] 
     [email protected] 
     [email protected] 
   } 
   # 邮件发送地址 
   notification_email_from [email protected]  
   smtp_server 127.0.0.1 
   smtp_connect_timeout 30 
   router_id NGINX_MASTER 
} 

vrrp_script check_nginx {
    script "/etc/nginx/check_nginx.sh"
}

vrrp_instance VI_1 { 
    state BACKUP        ##不同于nginx_lbm，此处的state为BACKUP
    interface ens33
    virtual_router_id 51        
    priority 90         ##优先级为90，低于nginx_lbm
    advert_int 1    
    authentication { 
        auth_type PASS      
        auth_pass 1111 
    }  
    virtual_ipaddress { 
        192.168.43.100/24 
    } 
    track_script {
        check_nginx
    } 
}

[root@nginx_lbb ~]# vi /etc/nginx/check_nginx.sh 
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")	

if [ "$count" -eq 0 ];then
    systemctl stop keepalived
fi


[root@nginx_lbb ~]# chmod +x /etc/nginx/check_nginx.sh 


##开启服务
[root@nginx_lbb ~]# systemctl start keepalived.service 

• 查看vip

验证负载均衡器的高可用

• 关闭nginx_lbm中的nginx服务

##主动关闭nginx服务
[root@nginx_lbm ~]# pkill nginx

##查看nginx和keepalived状态
[root@nginx_lbm ~]# systemctl status nginx
● nginx.service - nginx - high performance web server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since 三 2020-04-29 08:40:27 CST; 6s ago
     Docs: http://nginx.org/en/docs/
  Process: 4085 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=1/FAILURE)
 Main PID: 1939 (code=exited, status=0/SUCCESS)


##keepalived服务也被自动关闭
[root@nginx_lbm ~]# systemctl status keepalived.service 
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

4月 29 08:35:44 nginx_lbm Keepalived_vrrp[2204]: VRRP_Instance(VI_1) Send...
4月 29 08:35:44 nginx_lbm Keepalived_vrrp[2204]: Sending gratuitous ARP o...
4月 29 08:35:44 nginx_lbm Keepalived_vrrp[2204]: Sending gratuitous ARP o...
4月 29 08:35:44 nginx_lbm Keepalived_vrrp[2204]: Sending gratuitous ARP o...
4月 29 08:35:44 nginx_lbm Keepalived_vrrp[2204]: Sending gratuitous ARP o...
4月 29 08:40:27 nginx_lbm Keepalived[2202]: Stopping
4月 29 08:40:27 nginx_lbm systemd[1]: Stopping LVS and VRRP High Availab....
4月 29 08:40:27 nginx_lbm Keepalived_vrrp[2204]: VRRP_Instance(VI_1) sent...
4月 29 08:40:27 nginx_lbm Keepalived_vrrp[2204]: VRRP_Instance(VI_1) remo...
4月 29 08:40:28 nginx_lbm systemd[1]: Stopped LVS and VRRP High Availabi....
Hint: Some lines were ellipsized, use -l to show in full.


##查看地址发现，没有vip
[root@nginx_lbm ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:92:43:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.43.105/24 brd 192.168.43.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ba5a:8436:895c:4285/64 scope link 
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff

• 查看nginx_lbm和nginx_lbb的vip

上述界面的出现说明，双机热备成功

• 恢复vip

##在nginx_lbm中启动nginx和keepalived服务
[root@nginx_lbm ~]# systemctl start nginx
[root@nginx_lbm ~]# systemctl start keepalived

##再次查看地址信息，发现vip回到了nginx_lbm
[root@nginx_lbm ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:92:43:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.43.105/24 brd 192.168.43.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.43.100/24 scope global secondary ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::ba5a:8436:895c:4285/64 scope link 
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
    link/ether 52:54:00:72:80:f5 brd ff:ff:ff:ff:ff:ff

修改node的VIP以及pod的创建

修改node1和node2配置文件

• 以node2为例

[root@node2 ~]# cd /opt/kubernetes/cfg/
[root@node2 cfg]# ls
bootstrap.kubeconfig  kubelet.config      kube-proxy.kubeconfig
flanneld              kubelet.kubeconfig
kubelet               kube-proxy
[root@node2 cfg]# vi bootstrap.kubeconfig 
server: https://192.168.142.20:6443		
#改为Vip的地址
[root@node2 cfg]# vi kubelet.kubeconfig 
server: https://192.168.142.20:6443		
#改为Vip的地址
[root@node2 cfg]# vi kube-proxy.kubeconfig 
server: https://192.168.142.20:6443		
#改为Vip的地址
[root@node2 cfg]# grep 100 *
bootstrap.kubeconfig:    server: https://192.168.43.100:6443
kubelet.kubeconfig:    server: https://192.168.43.100:6443
kube-proxy.kubeconfig:    server: https://192.168.43.100:6443


##重启服务
[root@node2 cfg]# systemctl restart kubelet.service 
[root@node2 cfg]# systemctl restart kube-proxy.service 

• 在nginx_lbm上查看nginx的日志，看是否有node访问

[root@nginx_lbm ~]# cd /var/log/nginx/
[root@nginx_lbm nginx]# ls
access.log  error.log  k8s-access.log
[root@nginx_lbm nginx]# tail -f k8s-access.log 
192.168.43.102 192.168.43.101:6443 - [29/Apr/2020:08:49:41 +0800] 200 1119
192.168.43.102 192.168.43.102:6443, 192.168.43.101:6443 - [29/Apr/2020:08:49:41 +0800] 200 0, 1119
192.168.43.103 192.168.43.102:6443, 192.168.43.101:6443 - [29/Apr/2020:08:50:08 +0800] 200 0, 1120
192.168.43.103 192.168.43.101:6443 - [29/Apr/2020:08:50:08 +0800] 200 1121

在master上创建pod并且测试

• 创建pod

[root@master ~]# kubectl run nginx --image=nginx
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created

• 查看pod状态

[root@master ~]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-8qt6q   1/1     Running   0          24m

• 绑定群集中的匿名用户赋予管理员权限(解决日志不可看问题)

[root@master ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

• 查看pod日志

##在对应的node1上访问，因为这个pod创建在node1上
[root@node1 ~]# curl 172.17.36.2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@node1 ~]# 



##在master上查看日志
[root@master ~]# kubectl logs nginx-dbddb74b8-8qt6q
172.17.36.1 - - [29/Apr/2020:13:37:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
[root@master ~]# 

• 查看Pod网络

[root@master ~]# kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE             NOMINATED NODE
nginx-dbddb74b8-8qt6q   1/1     Running   0          30m   172.17.36.2   192.168.43.102   <none>

搭建k8s的Dashboard

master1上的操作

• 在dashboard官网中下载，安装web界面所需要的yaml文件，网址如下：
• https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dashboard

• 上传yaml文件

##创建dashboard目录
[root@master ~]# mkdir dashboard
[root@master ~]# cd dashboard/
##创建文件
[root@master dashboard]# ls
dashboard-configmap.yaml   dashboard-rbac.yaml    dashboard-service.yaml
dashboard-controller.yaml  dashboard-secret.yaml  k8s-admin.yaml

• 加载、创建所需文件，注意顺序如下：

##在/root/dashboard目录下操作

#授权访问api
kubectl create -f dashboard-rbac.yaml

#进行加密
kubectl create -f dashboard-secret.yaml

#配置应用
kubectl create -f dashboard-configmap.yaml

#控制器
kubectl create -f dashboard-controller.yaml

#发布出去进行访问
kubectl create -f dashboard-service.yaml

• 查看创建在指定的kube-system命名空间下的pod

[root@master dashboard]# kubectl get pods -n kube-system
NAME                                    READY   STATUS    RESTARTS   AGE
kubernetes-dashboard-65f974f565-bwmlx   1/1     Running   0          47s

##查看如何访问dashboard
[root@master dashboard]# kubectl get pods,svc -n kube-system
NAME                                        READY   STATUS    RESTARTS   AGE
pod/kubernetes-dashboard-65f974f565-bwmlx   1/1     Running   0          82s

NAME                           TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
service/kubernetes-dashboard   NodePort   10.0.0.199   <none>        443:30001/TCP   70s

由上可知，访问dashboard可以访问：

https://node的IP地址:30001/

比如：https://192.168.43.102:30001/

但是访问dashboard需要令牌，所以下面还需要生成令牌

• 生成自签证书

##制作证书脚本
[root@master dashboard]# vi dashboard-cert.sh
{
   "CN": "Dashboard",
   "hosts": [],
   "key": {
       "algo": "rsa",
       "size": 2048
   },
   "names": [
       {
           "C": "CN",
           "L": "BeiJing",
           "ST": "BeiJing"
       }
   ]
}
EOF

K8S_CA=$1
cfssl gencert -ca=$K8S_CA/ca.pem -ca-key=$K8S_CA/ca-key.pem -config=$K8S_CA/ca-config.json -profile=kubernetes dashboard-csr.json | cfssljson -bare dashboard
kubectl delete secret kubernetes-dashboard-certs -n kube-system
kubectl create secret generic kubernetes-dashboard-certs --from-file=./ -n kube-system


##执行脚本
[root@master dashboard]# bash dashboard-cert.sh /root/k8s/k8s-cert/
[root@master dashboard]# ls
dashboard-cert.sh          dashboard.csr       dashboard.pem          dashboard-service.yaml
dashboard-configmap.yaml   dashboard-csr.json  dashboard-rbac.yaml    k8s-admin.yaml
dashboard-controller.yaml  dashboard-key.pem   dashboard-secret.yaml



##在控制器yaml文件中添加证书，注意yaml文件的格式，使用空格
[root@master dashboard]# vi dashboard-controller.yaml 
#在47行下追加以下内容
          - --tls-key-file=dashboard-key.pem
          - --tls-cert-file=dashboard.pem


##重新部署控制器
[root@master dashboard]# kubectl apply -f dashboard-controller.yaml 

• 生成登录token

##生成令牌
[root@master dashboard]# kubectl create -f k8s-admin.yaml 

##将令牌保存
[root@master dashboard]# kubectl get secret -n kube-system
NAME                               TYPE                                  DATA   AGE
dashboard-admin-token-4zpgd        kubernetes.io/service-account-token   3      66s
default-token-pdn6p                kubernetes.io/service-account-token   3      39h
kubernetes-dashboard-certs         Opaque                                11     11m
kubernetes-dashboard-key-holder    Opaque                                2      15m
kubernetes-dashboard-token-4whmf   kubernetes.io/service-account-token   3      15m

##查看token，并且复制
[root@master dashboard]# kubectl describe secret dashboard-admin-token-4zpgd -n kube-system
Name:         dashboard-admin-token-4zpgd
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: dashboard-admin
              kubernetes.io/service-account.uid: 36095d9f-89bd-11ea-bb1a-000c29ce5f24

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1359 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tNHpwZ2QiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzYwOTVkOWYtODliZC0xMWVhLWJiMWEtMDAwYzI5Y2U1ZjI0Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.wRx71hNjdAOuaG8EPEr_yWaAmw_CF-aXwVFk7XeXwW2bzDLRh0RfQV-7nyBbw-wcPVXLbpoWNSYuHFS0vXHWGezk9ssERnErDXjE164H0lR8LkD1NekUQqB8L9jqW9oAZrZ0CkAxUIuijG14BjbAIV5wXmT1aKsK2sZTC0u-IjDcIT2UhjU3LvSL0Fzi4zyEvfl5Yf0Upx6dZ7yNpUd13ziNIP4KJ5DjWesIK-34IG106Kf6y1ehmRdW1Sg0HNvopXhFJPAhp-BkEz_SCmsf89_RDNVBTBSRWCgZdQC78B2VshbJqMRZOIV2IprBFhYKK6AeOY6exCyk1HWQRKFMRw
[root@master dashboard]# 

登录dashboard