kubernetes雲原生紀元：領悟Ingress Nginx(中)
續領悟Ingress Nginx(上)
解決問題如何自定義nginx 配置
自定義nginx 配置
Ingress-nginx 的配置跟原始的nginx的配置nginx.conf 沒什麼區別，可以使用ConfigMap來設置nginx的全局配置
data:以下是設置的nginx 配置
nginx-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app: ingress-nginx
data: # 設置自定義配置
proxy-body-size: "64m" # 文件大小的限制
proxy-read-timeout: "180" #讀寫時間180s
proxy-send-timeout: "180"
可以看下官網可以通過ConfigMap支持的配置
我們創建下
[root@master-001 ~]# kubectl apply -f nginx-config.yaml
configmap/nginx-configuration configured
我們進入到nginx容器裏面看下配置是否生效
[root@node-001 ~]# docker ps |grep nginx
d76f216c99a4 siriuszg/nginx-ingress-controller "/usr/bin/dumb-init …" About an hour ago Up About an hour k8s_nginx-ingress-controller_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0
d26440f04b55 registry.aliyuncs.com/google_containers/pause:3.1 "/pause" About an hour ago Up About an hour k8s_POD_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0
[root@node-001 ~]# docker exec -it d7 sh
$ more nginx.comf
文件太大我們使用more查看
搜索我們剛喫配置的nginx 配置，已經生效了
注意：這裏的生效的nginx 配置跟我們在configmap配置的名字不太一樣，但是作用是一致的我們用之前最好去官網去查閱，不要寫nginx的key，是不生效的。
Nginx 增加header 信息
全局header
如果我們要在nginx加一下header頭部信息 需要增加proxy-set-headers: "ingress-nginx/custom-headers"把custom-headers當作header引入進去，具體如下
custom-header-global.yaml
apiVersion: v1
kind: ConfigMap
data:
proxy-set-headers: "ingress-nginx/custom-headers" #通過這個proxy-set-headers 引入下面的custom-headers定義的header信息
metadata:
name: nginx-configuration
namespace: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: v1
kind: ConfigMap
data: # header配置信息
X-Different-Name: "true"
X-Request-Start: t=${msec}
X-Using-Nginx-Controller: "true"
metadata:
name: custom-headers
namespace: ingress-nginx
創建一下
[root@master-001 ~]# kubectl apply -f custom-header-global.yaml
configmap/nginx-configuration configured
configmap/custom-headers created
我們進入到nginx容器裏面看下配置是否生效
[root@node-001 ~]# docker ps |grep nginx
d76f216c99a4 siriuszg/nginx-ingress-controller "/usr/bin/dumb-init …" About an hour ago Up About an hour k8s_nginx-ingress-controller_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0
d26440f04b55 registry.aliyuncs.com/google_containers/pause:3.1 "/pause" About an hour ago Up About an hour k8s_POD_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0
[root@node-001 ~]# docker exec -it d7 sh
$ more nginx.comf
某個ingress header
唯一區別是增加annotations: 通過nginx.ingress.kubernetes.io/configuration-snippet: |配置多個header，然後通過host: 指定那個ingress-nginx
custom-header-spec-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Request-Id: $req_id";
name: web-demo
namespace: dev
spec:
rules:
- host: web-dev.mooc.com # 指定對應的nignx 代理域名的
http:
paths:
- backend:
serviceName: web-demo
servicePort: 80
path: /
我們進入到nginx容器裏面看下配置，也就是說只在server_name 是web-dev.mooc.com生效
Nginx 模版
可能以上還不能滿足我們的要求，我就就可以用nginx模版custom nginx template
這個模版通過程序生成模版文件 路徑：/etc/nginx/template/nginx.tmpl
-
拿到模版文件
來到按照ingress-nginx的node節點，通過
docker cp b63:/etc/nginx/template/nginx.tmpl .拿取到模版文件，然後發送到主節點[root@node-001 ~]# docker cp b63:/etc/nginx/template/nginx.tmpl . [root@node-001 ~]# ls anaconda-ks.cfg ingress-demo.yaml nginx-config.yaml nginx.tmpl [root@node-001 ~]# scp nginx.tmpl 172.16.126.132:~/ nginx.tmpl 100% 49KB 16.8MB/s 00:00 -
創建模版
來到主節點，把剛纔傳過來的模版文件創建一下
[root@master-001 ~]# kubectl create cm nginx-template --from-file nginx.tmpl -n ingress-nginx configmap/nginx-template created [root@master-001 ~]# kubectl get cm -n ingress-nginx NAME DATA AGE custom-headers 3 75m ingress-controller-leader-nginx 0 26d nginx-configuration 1 26d nginx-template 1 16s tcp-services 1 26d udp-services 0 26dnginx.tmpl 文件太大了這裏不查看了
-
掛載nginx.tmpl
需要修改nginx-ingress-controller 增加數據卷通過configMap指定nginx.tmpl, 在容器級增加volume掛載,具體如下
nginx-ingress-controller.yaml
[root@master-001 ~]# vi nginx-ingress-controller.yaml apiVersion: apps/v1 kind: DaemonSet metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx name: nginx-ingress-controller namespace: ingress-nginx spec: revisionHistoryLimit: 10 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx updateStrategy: rollingUpdate: maxUnavailable: 1 type: RollingUpdate template: metadata: annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" creationTimestamp: null labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: containers: - args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io # 增加數據卷掛載， volumeMounts: - mountPath: /etc/nginx/template name: nginx-template readOnly: true # end env: - name: POD_NAME valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: apiVersion: v1 fieldPath: metadata.namespace image: siriuszg/nginx-ingress-controller:latest imagePullPolicy: Always lifecycle: preStop: exec: command: - /wait-shutdown livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 name: nginx-ingress-controller ports: - containerPort: 80 hostPort: 80 name: http protocol: TCP - containerPort: 443 hostPort: 443 name: https protocol: TCP readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 resources: {} securityContext: allowPrivilegeEscalation: true capabilities: add: - NET_BIND_SERVICE drop: - ALL runAsUser: 33 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst hostNetwork: true nodeSelector: kubernetes.io/os: linux app: ingress restartPolicy: Always schedulerName: default-scheduler securityContext: {} serviceAccount: nginx-ingress-serviceaccount serviceAccountName: nginx-ingress-serviceaccount terminationGracePeriodSeconds: 300 # 通過configmap指定我們上面創建的nginx.tmpl volumes: - name: nginx-template-volume configMap: name: nginx-template items: - key: nginx.tmpl path: nginx.tmpl創建一下
[root@master-001 ~]# kubectl apply -f nginx-ingress-controller.yaml我看可以去 容器運行節點查看下，這裏就不看了…
-
修改模版測試
修改的時候不需要，知道太多語法，直接照貓畫虎就可以
隨便修改一點東西
[root@master-001 ~]# kubectl edit cm -n ingress-nginx nginx-template..... http2_max_field_size {{ $cfg.HTTP2MaxFieldSize }}; http2_max_header_size {{ $cfg.HTTP2MaxHeaderSize }}; http2_max_requests {{ $cfg.HTTP2MaxRequests }}; # 比如把types_hash_max_size 2048改成4096 types_hash_max_size 4096; server_names_hash_max_size {{ $cfg.ServerNameHashMaxSize }}; server_names_hash_bucket_size {{ $cfg.ServerNameHashBucketSize }}; map_hash_bucket_size {{ $cfg.MapHashBucketSize }}; ......保存後我們去容器運行節點查看一下配置
[root@node-001 ~]# docker ps |grep nginx d76f216c99a4 siriuszg/nginx-ingress-controller "/usr/bin/dumb-init …" About an hour ago Up About an hour k8s_nginx-ingress-controller_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0 d26440f04b55 registry.aliyuncs.com/google_containers/pause:3.1 "/pause" About an hour ago Up About an hour k8s_POD_nginx-ingress-controller-s2nt5_ingress-nginx_0f3f90fe-9416-44eb-998b-cd408fd593de_0 [root@node-001 ~]# docker exec -it d7 sh $ more nginx.comf![image-20200128213737783]()
我們看到已經生效，這個原理是由kubelt定期自動檢查configmap，動態更新配置