昨天做了一天的flannel網絡問題終於得到了解決,原來是flannel的版本問題。
安裝1.9.1時使用的kubernetes-cni是高版本的,因此可以與自動獲取的高版本flannel兼容,沒出什麼問題。
安裝1.7.10時使用的cni則是0.5低版本的,因此不能兼容需要手動獲取0.8.0版本的flannel
下面進行完整的kubernetes-v1.7.10版本內網安裝過程:
一、安裝docker、go
1、安裝docker,可以直接用apt安裝,版本目前爲1.13.
apt-get update && apt-get install docker.io
2、安裝go,這裏使用的是最新版本,從官網下載安裝
http://www.golangtc.com/download
目前是go1.9.2.linux-amd64.tar.gz,下載後解壓
tar -xzf go1.9.2.linux-amd64.tar.gz -C /usr/local
設置環境變量 :
vim ~/.bashrc
末尾加入
export GOPATH=/opt/go
export GOROOT=/usr/local/go
export GOARCH=386
export GOOS=linux
export GOBIN=$GOROOT/bin/
export GOTOOLS=$GOROOT/pkg/tool/
export PATH=$PATH:$GOBIN:$GOTOOLS
保存後重新加載環境變量
source ~/.bashrc
任意目錄下進行安裝完成驗證
go version
出現版本信息即成功。
二、安裝kubeadm等工具
1、從可以fq的機子上通過
apt-get update && apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
獲取相應版本的kubeadm、kubectl、kubelet和kubernetes-cni(apt-cache madison ***查看版本)
然後在/var/cache中找到四個文件,
2、在內網計算機安裝工具
先安裝ebtables和socat工具
apt-get install ebtables
apt-get install socat
再依次安裝
dpkg -i kubernetes-cni_0.5.1-00_amd64.deb
dpkg -i kubectl_1.7.10-00_amd64.deb
dpkg -i kubelet_1.7.10-00_amd64.deb
dpkg -i kubeadm_1.7.10-00_amd64.deb
三、獲取鏡像
1、從fq機上傳鏡像,上傳到自己的docker hub中
先打TAG:
docker tag quay.io/coreos/flannel:v0.8.0-amd64 eavan/flannel:v0.8.0-amd64
docker tag gcr.io/google_containers/kube-apiserver-amd64:v1.7.10 eavan/kube-apiserver-amd64:v1.7.10
docker tag gcr.io/google_containers/kube-proxy-amd64:v1.7.10 eavan/kube-proxy-amd64:v1.7.10
docker tag gcr.io/google_containers/kube-controller-manager-amd64:v1.7.10 eavan/kube-controller-manager-amd64:v1.7.10
docker tag gcr.io/google_containers/kube-scheduler-amd64:v1.7.10 eavan/kube-scheduler-amd64:v1.7.10
docker tag gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.5 eavan/k8s-dns-sidecar-amd64:1.14.5
docker tag gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.5 eavan/k8s-dns-kube-dns-amd64:1.14.5
docker tag gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.5 eavan/k8s-dns-dnsmasq-nanny-amd64:1.14.5
docker tag gcr.io/google_containers/etcd-amd64:3.0.17 eavan/etcd-amd64:3.0.17
docker tag gcr.io/google_containers/pause-amd64:3.0 eavan/pause-amd64:3.0
再通過docker push 命令一一上傳,現在已在eavan/中,不必再次上傳
2、內網機獲取鏡像
docker pull 一一獲取鏡像
docker tag 一一反向打上tag
全部打完tag 後可以通過docker rmi 來刪除其他的鏡像
ps:可以通過編寫腳本文件自動執行減少工作量
在master節點上,del.sh
#!/bin/bash
images=(
kube-proxy-amd64:v1.7.10
kube-controller-manager-amd64:v1.7.10
kube-apiserver-amd64:v1.7.10
kube-scheduler-amd64:v1.7.10
k8s-dns-sidecar-amd64:1.14.5
k8s-dns-kube-dns-amd64:1.14.5
k8s-dns-dnsmasq-nanny-amd64:1.14.5
etcd-amd64:3.0.17
pause-amd64:3.0
)
for imageName in ${images[@]} ; do
docker pull eavan/$imageName
docker tag eavan/$imageName gcr.io/google_containers/$imageName
docker rmi eavan/$imageName
done
在node節點上,del.sh
#!/bin/bash
images=(
kube-proxy-amd64:v1.7.10
etcd-amd64:3.0.17
pause-amd64:3.0
)
for imageName in ${images[@]} ; do
docker pull eavan/$imageName
docker tag eavan/$imageName gcr.io/google_containers/$imageName
docker rmi eavan/$imageName
done
四、master初始化
hostname xxx
vi /etc/hostname
vi /etc/hosts
reboot
kubeadm init --kubernetes-version=v1.7.10 --pod-network-cidr=10.244.0.0/16
初始化完成後按照打印的要求做。
(若想要將master節點佈置爲可以運行pod的節點)
kubectl taint nodes --all node-role.kubernetes.io/master-
五、安裝flannel
wget https://raw.githubusercontent.com/coreos/flannel/v0.8.0/Documentation/kube-flannel-rbac.yml
wget https://raw.githubusercontent.com/coreos/flannel/v0.8.0/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel-rbac.yml
kubectl apply -f kube-flannel.yml
成功後created 5個文件
通過
kubectl get pod --all-namespaces -o wide
查看全部pod,都running則完成。
六、8080端口及DNS問題
1、解決curl localhost:8080無法訪問
vi /etc/kubernetes/manifests/kube-apiserver.yaml
找到insecure-port,原始值爲0,按照正常來說0端口代表所有端口,但在本此問題中,0代表默認爲6443,因此apiserver不通過8080端口開放,需要將0改爲8080即可
(若改完後顯示6443拒絕訪問,等待或者systemctl restart kubelet)
改完後通過curl localhost:8080即可看到api,也可以通過lsof -i:8080觀察8080端口的監聽
2、解決DNS總是crashloopback的問題
修改Kubelet啓動參數
vim /etc/systemd/system/kubelet.service.d
找到並添加
KUBELET_DNS_ARGS=--address=192.168.xxx.xxx(本機Ip)
3、解決DNS無法解析域名的問題
測試DNS是否正常工作:
創建busybox.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
restartPolicy: Always
使用該文件創建pod
kubectl create -f busybox.yaml
等待pod進入running狀態
kubectl get pods busybox
一旦pod處於running狀態時,可以使用exec nslookup來查詢狀態:
kubectl exec -ti busybox -- nslookup kubernetes.default
發現出現域名無法解析的狀況(can't find )
① 修改/etc/resolv.conf,添加nameserver 192.168.224.2(虛擬機dns域名)
systemctl restart docker
systemctl restart kubelet
② 修改防火牆
iptables -P FORWARD ACCEPT
再次查詢狀態即發現已成功。
ps:修改/etc/resolv.conf僅爲一次性,當重啓網絡或reboot後會失效
永久性修改:
vim /etc/network/interfaces
添加
dns-nameservers 192.168.224.2 即可
七、dashboard安裝
注意:目前只安裝上1.6.2成功
https://www.cnblogs.com/aguncn/p/7158881.html
創建kubernetes-dashboard-rbac.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: dashboard-admin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: default
namespace: kube-system
創建kubernetes-dashboard.yaml
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Configuration to deploy release version of the Dashboard UI compatible with
# Kubernetes 1.6 (RBAC enabled).
#
# Example usage: kubectl create -f <this_file>
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
labels:
k8s-app: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- name: kubernetes-dashboard
image: registry.cn-hangzhou.aliyuncs.com/google-containers/kubernetes-dashboard-amd64:v1.6.1
ports:
- containerPort: 9090
protocol: TCP
args:
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
livenessProbe:
httpGet:
path: /
port: 9090
initialDelaySeconds: 30
timeoutSeconds: 30
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
- port: 80
targetPort: 9090
selector:
k8s-app: kubernetes-dashboard
再create便可成功。
通過
kubectl get svc -n kube-system
查看dashboard的端口,然後打開瀏覽器,通過localhost:xxx訪問
八、heapster安裝
mkdir /etc/kubernetes/heapster
cd heapster
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/rbac/heapster-rbac.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml
wget https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml
修改docker images 來源:
registry.cn-shenzhen.aliyuncs.com/rancher_cn/heapster-grafana-amd64:v4.4.3
registry.cn-shenzhen.aliyuncs.com/rancher_cn/heapster-amd64:v1.4.2
registry.cn-shenzhen.aliyuncs.com/rancher_cn/heapster-influxdb-amd64:v1.3.3
kubectl create -f ./
即可