1、認識Ingress
1.1 什麼是Ingress?
通常情況下,service和pod僅可在集羣內部網絡中通過IP地址訪問。所有到達邊界路由器的流量或被丟棄或被轉發到其他地方。從概念上講,可能像下面這樣:
1
2
3
4
|
internet | ------------ [ Services ] |
Ingress是授權入站連接到達集羣服務的規則集合。
1
2
3
4
5
|
internet | [ Ingress ] --|-----|-- [ Services ] |
可以給Ingress配置提供外部可訪問的URL、負載均衡、SSL、基於名稱的虛擬主機等。用戶通過POST Ingress資源到API server的方式來請求ingress。 Ingress controller負責實現Ingress,通常使用負載平衡器,它還可以配置邊界路由和其他前端,這有助於以HA方式處理流量。
可以將 Ingress 配置爲提供服務外部可訪問的 URL、負載均衡流量、終止 SSL / TLS 並提供基於名稱的虛擬主機。Ingress 控制器通常負責通過負載均衡器來實現 Ingress,儘管它也可以配置邊緣路由器或其他前端來幫助處理流量。
Ingress 不會公開任意端口或協議。 將 HTTP 和 HTTPS 以外的服務公開到 Internet 時,通常使用 Service.Type=NodePort 或者 Service.Type=LoadBalancer 類型的服務。
1.2 Ingress工作示意圖
1.3先決條件
在使用Ingress resource之前,有必要先了解下面幾件事情。Ingress是beta版本的resource,在kubernetes1.1之前還沒有。需要一個Ingress Controller來實現Ingress,單純的創建一個Ingress沒有任何意義。
GCE/GKE會在master節點上部署一個ingress controller。你可以在一個pod中部署任意個自定義的ingress controller。你必須正確地annotate每個ingress,比如 運行多個ingress controller 和 關閉glbc.
確定你已經閱讀了Ingress controller的beta版本限制。在非GCE/GKE的環境中,你需要在pod中 ingress-nginx。
1.4 Ingress定義資源清單幾個字段
- apiVersion: v1 版本
- kind: Ingress 類型
- metadata 元數據
- spec 期望狀態
- backend: 默認後端,能夠處理與任何規則不匹配的請求
- rules:用於配置Ingress的主機規則列表
- tls:目前Ingress僅支持單個TLS端口443
- status 當前狀態
2、基於ingress-nginx控制器實現公網訪問
1、在kubernetes官網上下載ingerss-nginx控制器
(1)在githab上下載yaml文件,並創建部署
githab ingress-nginx項目:https://github.com/kubernetes/ingress-nginx
K8s官方部署文檔:
ingress安裝指南:https://kubernetes.github.io/ingress-nginx/deploy/
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml # 基於官網下載ingress-nginx控制器
第二種方式拉取ingress-nginx控制器,裏邊包含了service使用NodePort提供的端口號,可以對外提供公網地址,訪問內部服務,目前還沒有研究透,不通過FQ怎麼下載鏡像。
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/baremetal/deploy.yaml
2、由於是需要在國外下載鏡像,我們可以將ingress-nginx裏的yaml文件的下載鏡像路徑位置改到國內即可
[root@master data]# vim mandatory.yaml # 修改下載下來的yaml文件,鏡像路徑指向阿里雲路徑 image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.30.0
3、執行下載的mandatory.yaml文件。
kubectl apply -f mandatory.yaml # 執行完的yaml文件默認在ingress-nginx名稱空間中
4、查看創建在ingress-nginx名稱空間的控制器
[root@master data]# kubectl get pods -n ingress-nginx # 查看此時的控制器已經創建完成 NAME READY STATUS RESTARTS AGE nginx-ingress-controller-fbf967dd5-r7dfk 1/1 Running 3 5h58m
2、創建一個NodePort的service文件
1、創建一個service,使用NodePort模式將端口暴露到公網上,方便客戶端訪問到ingress調度到後端服務器pod的網頁上。
[root@master data]# cat my-svc.yaml # 創建一個負責暴露端口的service服務,使用NodePort網絡模式 apiVersion: v1 kind: Service metadata: name: ingress namespace: ingress-nginx # 名稱空間要與ingress-nginx一致,否則無法關聯 spec: selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx type: NodePort ports: - name: http port: 80 nodePort: 30080 # 指定暴露到外網的端口號 - name: https port: 443 nodePort: 30443
2、通過my-svc.yaml文件創建一個暴露公網端口的Service
[root@master data]# kubectl apply -f my-svc.yaml service/ingress changed
3、驗證,查看此時對外服務的端口號是30080
[root@master data]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress NodePort 10.107.61.112 <none> 80:30080/TCP,443:30443/TCP 6h30m
3、準備後端pod和service
(1)編寫yaml文件,並創建
創建3個nginx服務的pod副本,並創建一個service綁定
[root@master data]# cat deploy-damo.yaml apiVersion: v1 kind: Service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary ports: - name: http targetPort: 80 port: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: myapp release: canary template: metadata: labels: app: myapp release: canary spec: containers: - name: myapp image: ikubernetes/myapp:v2 ports: - name: http containerPort: 80
(2)基於deploy-damo.yaml文件創建pod和綁定service
[root@master data]# kubectl apply -f deploy-damo.yaml service/myapp unchanged deployment.apps/myapp-deploy unchanged
(3)查詢驗證
[root@master data]# kubectl get svc # 查看創建的服務IP地址 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 17h myapp ClusterIP 10.97.192.121 <none> 80/TCP 7h18m [root@master data]# kubectl get pods -o wide # 查看pod的IP地址 NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp-deploy-798dc9b584-97qb9 1/1 Running 3 7h18m 10.244.1.45 node1 <none> <none> myapp-deploy-798dc9b584-qctz4 1/1 Running 3 7h18m 10.244.2.47 node2 <none> <none> myapp-deploy-798dc9b584-tww7g 1/1 Running 3 7h18m 10.244.2.46 node2 <none> <none> [root@master data]# curl 10.244.1.45 # 訪問service可以訪問 Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a> [root@master data]# curl 10.97.192.121 # 訪問pod也可以訪問 Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
4、 創建ingress,綁定後端nginx服務
查看ingress支持的apiVersion版本
[root@master manifests]# kubectl explain ingress KIND: Ingress VERSION: extensions/v1beta1 # 此時版本屬於此羣組 DESCRIPTION: Ingress is a collection of rules that allow inbound connections to reach the endpoints defined by a backend. An Ingress can be configured to give services externally-reachable urls, load balance traffic, terminate SSL, offer name based virtual hosting etc. DEPRECATED - This group version of Ingress is deprecated by networking.k8s.io/v1beta1 Ingress. See the release notes for more information.
(1)編寫yaml文件,並創建
[root@master data]# cat ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myapp namespace: default # 定義名稱空間,與前面的nginx的名稱空間一致 annotations: # 此項非必須,需不需要做URL的/重定向什麼地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此項非必須 kubernetes.io/ingress.class: "nginx" # 定義ingress類別,如果有多個標誌,就需要進行區分 spec: rules: - host: www.peng.com # 不寫此主機名,默認就是虛擬主機的名稱,一般不寫 http: paths: - path: / backend: serviceName: myapp # 與service名稱一致 servicePort: 80 # 與servie的端口號一致
(2)創建並查詢驗證
[root@master data]# kubectl apply -f ingress-myapp.yaml # 創建Ingress ingress.extensions/ingress-myapp unchanged [root@master data]# kubectl get ingress # 查看此時的ingress信息 NAME HOSTS ADDRESS PORTS AGE ingress-myapp www.peng.com 80 7h21m [root@master ~]# kubectl get pods -n ingress-nginx # 查看pod名稱 NAME READY STATUS RESTARTS AGE nginx-ingress-controller-fbf967dd5-r7dfk 1/1 Running 4 10h [root@master ~]# kubectl exec -it nginx-ingress-controller-fbf967dd5-r7dfk -n ingress-nginx -- /bin/sh # 進入到pod中查看定義nginx的配置信息 /etc/nginx $ more nginx.conf
(3)在集羣外,查詢服務驗證
① 可以先修改一下Linux主機和windows的hosts文件,因爲不是公網域名
192.168.7.101 www.peng.com
② 訪問業務成功
通過Linux進行訪問,實現了輪詢調度到後端服務器的效果:
[root@master data]# while :;do curl http://www.peng.com:30080/hostname.html;sleep 2; done myapp-deploy-798dc9b584-97qb9 myapp-deploy-798dc9b584-97qb9 myapp-deploy-798dc9b584-qctz4 myapp-deploy-798dc9b584-tww7g
4、創建Ingress,代理到後端tomcat服務
4.1 準備後端pod和service
(1)編寫yaml文件,並創建
創建3個tomcat服務的pod,並創建一個service綁定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
|
[root@master ingress] # vim tomcat-deploy.yaml apiVersion: v1 kind: Service metadata: name: tomcat namespace: default spec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 port: 8080 - name: ajp targetPort: 8009 port: 8009 --- apiVersion: apps /v1 kind: Deployment metadata: name: tomcat-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat:8.5.37-jre8-alpine ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009 [root@master ingress] # kubectl apply -f tomcat-deploy.yaml service /tomcat created deployment.apps /tomcat-deploy created |
(2)查詢驗證
1
2
3
4
5
6
7
8
9
|
[root@master ~] # kubectl get pods NAME READY STATUS RESTARTS AGE tomcat-deploy-97d6458c5-hrmrw 1 /1 Running 0 1m tomcat-deploy-97d6458c5-ngxxx 1 /1 Running 0 1m tomcat-deploy-97d6458c5-xchgn 1 /1 Running 0 1m [root@master ~] # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443 /TCP 146d tomcat ClusterIP 10.98.193.252 <none> 8080 /TCP ,8009 /TCP 1m |
4.2 創建ingress,綁定後端tomcat服務
(1)編寫yaml文件,並創建
[root@master ingress]# vim ingress-tomcat.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tomcat namespace: default annotations: # 此項非必須,需不需要做URL重定向到什麼地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此項非必須 kubernetes.io/ingress.class: "nginx" # 定義ingress類別,只對關鍵的nginx字樣的服務進行調度 spec: rules: - host: net.peng.com http: paths: - path: backend: serviceName: tomcat servicePort: 8080 [root@master ingress]# kubectl apply -f ingress-tomcat.yaml ingress.extensions/ingress-tomcat created
(2)查詢驗證
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp www.peng.com 80 17m ingress-tomcat net.peng.com 80 6s [root@master ~] # kubectl describe ingress ingress-tomcat Name: ingress-tomcat Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- net.peng.com tomcat:8080 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tomcat" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "net.peng.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :8080}, "path" :null}]}}]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 17s nginx-ingress-controller Ingress default /ingress-tomcat |
(3)在集羣外,查詢服務驗證
① 可以先修改一下windows和Linux主機的hosts,因爲不是公網域名
192.168.7.101 net.peng.com
② 訪問業務成功
4.3 使用https協議訪問服務
4.3.1 創建證書、私鑰和secret
(1)創建私鑰
1
2
3
4
5
6
7
|
[root@master ingress] # openssl genrsa -out tls.key 2048 Generating RSA private key, 2048 bit long modulus .............................................+++ ...............+++ e is 65537 (0x10001) [root@master ingress] # ls *key tls.key |
(2)創建證書
1
2
3
|
[root@master ingress] # openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=myapp.peng.com [root@master ingress] # ls tls.* tls.crt tls.key |
(3)創建secret
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress] # kubectl create secret tls myapp-ingress-secret --cert=tls.crt --key=tls.key # 證書起名叫myapp-ingress-secret secret /tomcat-ingress-secret created [root@master ingress] # kubectl get secret NAME TYPE DATA AGE myapp-ingress-secret kubernetes.io /tls 2 8s [root@master ingress] # kubectl describe secret myapp-ingress-secret Name: myapp-ingress-secret Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io /tls Data ==== tls.key: 1675 bytes tls.crt: 1294 bytes |
4.3.2 重新創建ingress,使用https協議綁定後端tomcat服務
(1)編寫yaml文件,並創建
[root@master ingress]# vim ingress-tls.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: ingress-tls namespace: default annotations: # 此項非必須,需不需要做URL重定向到什麼地方 nginx.ingress.kubernetes.io/rewrite-target: / # 此項非必須 kubernetes.io/ingress.class: "nginx" # 定義ingress類別,只對關鍵的nginx字樣的服務進行調度 spec: tls: - hosts: - myapp.peng.com # 與自簽名的域名一致 secretName: myapp-ingress-secret # 與創建的secret名稱一致 rules: - host: myapp.peng.com # 與自簽名域名一致 http: paths: - path: backend: serviceName: myapp # 與後端servie名稱一致 servicePort: 80 # 與後端端口一致
(2)查詢驗證
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.peng.com 80 34m ingress-myapp www.peng.com 80 16m ingress-tls net.peng.com 80, 443 8s [root@master ~] # kubectl describe ingress ingress-tls Name: ingress-tls Namespace: default Address: Default backend: default-http-backend:80 (<none>) TLS: tomcat-ingress-secret terminates myapp.peng.com Rules: Host Path Backends ---- ---- -------- myapp.peng.com nginx:80 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tls" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "net.peng.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :80}, "path" :null}]}}], "tls" :[{ "hosts" :[ "net.peng.com" ], "secretName" :myapp -ingress-secret" }]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 14s nginx-ingress-controller Ingress default /ingress-tomcat-tls |
(3)在集羣外,查詢服務驗證
使用https協議,訪問業務成功