k8s部署持續集成環境
安裝前的準備：
關閉防火牆
關閉selinux
安裝docker-ce
1、安裝harbor參考前面筆記
2、安裝並配置git，參考前面筆記
3、在git這臺部署nfs服務器，並且在各個節點安裝nfs-utils客戶端
4、創建nfs-client-provisioner客戶端
cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
archiveOnDelete: "true"
cat deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: lizhenliang/nfs-client-provisioner:latest
volumeMounts:- name: nfs-client-root
mountPath: /persistentvolumes
env: - name: PROVISIONER_NAME
value: fuseim.pri/ifs - name: NFS_SERVER
value: 192.168.31.64 - name: NFS_PATH
value: /ifs/kubernetes
volumes:
- name: nfs-client-root
-
name: nfs-client-root
nfs:
server: 192.168.31.64
path: /ifs/kubernetes
cat rbac.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
name: nfs-client-provisionerkind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"] -
apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects: -
kind: ServiceAccount
name: nfs-client-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.iokind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
rules: -
apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
subjects: - kind: ServiceAccount
name: nfs-client-provisionerreplace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
5、部署jenkins服務器，前提是需要部署k8s的core-dns（安裝coredns查看前面筆記），否則沒法做解析就沒法安裝插件
cat ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jenkins
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/tls-acme: "true"如果上傳插件超出默認會報"413 Request Entity Too Large", 增加 client_max_body_size
nginx.ingress.kubernetes.io/proxy-body-size: 50m
nginx.ingress.kubernetes.io/proxy-request-buffering: "off"nginx-ingress controller版本小於 0.9.0.beta-18 的配置
ingress.kubernetes.io/ssl-redirect: "true"
ingress.kubernetes.io/proxy-body-size: 50m
ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
rules: - host: jenkins.example.com
http:
paths:-
path: /
backend:
serviceName: jenkins
servicePort: 80
cat rbac.yml創建名爲jenkins的ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins
-
- apiGroups: [""]
創建名爲jenkins的Role，授予允許管理API組的資源Pod
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"] - apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"] - apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
將名爲jenkins的Role綁定到名爲jenkins的ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins
cat service.yml
apiVersion: v1
kind: Service
metadata:
name: jenkins
spec:
selector:
name: jenkins
type: NodePort
ports:
name: http
port: 80
targetPort: 8080
protocol: TCP
nodePort: 30006
-
name: agent
port: 50000
protocol: TCPcat statefulset.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: jenkins
labels:
name: jenkins
spec:
serviceName: jenkins
replicas: 1
updateStrategy:
type: RollingUpdate
template:
metadata:
name: jenkins
labels:
name: jenkins
spec:
terminationGracePeriodSeconds: 10
serviceAccountName: jenkins
containers: (如果出現dns沒法解析的情況，需要在container這 一行上面加一行dnsPolicy: Default)
- name: jenkins
image: jenkins/jenkins
imagePullPolicy: Always
ports:- containerPort: 8080
- containerPort: 50000
resources:
limits:
cpu: 1
memory: 1Gi
requests:
cpu: 0.5
memory: 500Mi
env: - name: LIMITS_MEMORY
valueFrom:
resourceFieldRef:
resource: limits.memory
divisor: 1Mi - name: JAVA_OPTS
value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
volumeMounts: - name: jenkins-home
mountPath: /var/jenkins_home
livenessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
readinessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
securityContext:
fsGroup: 1000
volumeClaimTemplates:- metadata:
name: jenkins-home
spec:
storageClassName: "managed-nfs-storage"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
6、使用這幾個yaml文件啓動安裝jenkins
PS：當提示以下出錯時
'FailedCreate' create Pod jenkins-0 in StatefulSet jenkins failed error: pods "jenkins-0" is forbidden: pod.Spec.SecurityContext.FSGroup is forbidden
修改/opt/kubernetes/cfg/kube-apiserver，刪除裏面的安全字段SecurityContext
7、使用瀏覽器訪問http://10.1.2.190:30006進行安裝(使用kubectl get svc -o wide查看jenkins當前在190這個pod運行)
8、安裝時不選擇任何插件，使用手動安裝插件，安裝git和kubernetes插件
- metadata: