企業級CI/CD流程搭建
一、CI/CD流程
- 開發push代碼至GitLab
- GitLab的webhook觸發Jenkins的pipeline
- Jenkins pipeline進行構建、打包、生成Docker-image,並且push image至Harbor
- 服務器(生產、開發、測試環境)從Harbor pull image,並且運行 image
- 所有環境使用Rancher來統一進行管理
CI/CD流程圖如下
二、環境要求
1. 機器環境
- CentOS 7 64位
- 內核3.10或以上
- 至少1臺機器,配置建議8核32G 硬盤2T以上,可適當降低,但是一定要留足夠的擴展空間(硬盤、內存、CPU槽),便於後期擴展
[root@ruhr-gitlab workspace]# cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)
[root@ruhr-gitlab workspace]# uname -a
Linux ruhr-gitlab 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 30 14:19:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
2. 應用環境
規劃:所有應用全部安裝於1臺服務器192.168.0.100
應用 | 版本 | 端口 |
---|---|---|
JDK | 1.8 | |
Git | 2.13.1 | |
Maven | 3.6.0 | |
docker-ce | 19.03.3 | |
docker-compose | 1.24.1 | |
gitlab-ce | 12.3.5 | 10000 |
Jenkins | 2.190.1 | 10010 |
SonarQube | 7.3 | 10020 |
Harbor | 1.9.1 | 10030 |
Rancher | v2.3.1 | 80 |
三、環境準備
1.機器環境準備
-
CentOS 7 關閉selinux
vim /etc/selinux/config,全部註釋,只保留SELINUX=disabled
-
CentOS 7 關閉firewalld,開啓iptables
[root@localhost /]# systemctl stop firewalld
[root@localhost /]# systemctl disable firewalld
[root@localhost /]# yum install -y iptables-services
[root@localhost /]# systemctl enable iptables
[root@localhost /]# systemctl start iptables
- 安裝JDK
從orcale官網下載jdk安裝包,直接解壓,配置環境變量即可,下載需要註冊orcale賬號
jdk所有歷史版本鏈接點這裏
[root@localhost ~] mkdir -p /usr/java/
[root@localhost ~] cd /usr/java/ #上傳jdk包到該路徑
[root@localhost ~] tar -xzvf jdk-8u162-linux-x64.tar.gz
配置環境變量
[root@localhost ~] vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_162
export JAR_HOME=/usr/java/jdk1.8.0_162/jre
export CLASSPATH=/usr/java/jdk1.8.0_162/lib
export PATH=$JAVA_HOME/bin:$PATH
:wq保存退出
[root@localhost ~] source /etc/profile
[root@localhost ~] java -version
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
2.gitlab-ce部署
- 安裝Git
yum install -y git
- 安裝依賴
[root@localhost ~] yum -y install policycoreutils openssh-server openssh-clients postfix
- 開啓postfix
[root@localhost ~] systemctl enable postfix && systemctl start postfix
- 下載gitlab-ce
[root@localhost ~] wget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-12.3.5-ce.0.el7.x86_64.rpm
- 安裝gitlab-ce
[root@localhost ~] rpm -ivh gitlab-ce-12.3.5-ce.0.el7.x86_64.rpm
- 配置IP端口
[root@localhost ~] vim /etc/gitlab/gitlab.rb
external_url 'http://192.168.0.100:10000'
- 重置gitlab並啓動
[root@localhost ~] gitlab-ctl reconfigure
[root@localhost ~] gitlab-ctl restart
GitLab漢化過程不在此記錄
3.Jenkins部署
- 下載jenkins.jar
wget http://mirrors.jenkins.io/war-stable/latest/jenkins.war
- 啓動jenkins
java -jar jenkins.war --httpPort=10010
訪問192.168.0.100:10010,根據提示操作即可
4.Maven部署
- 下載安裝
[root@localhost ~] mkdir -p /usr/local/maven
[root@localhost ~] wget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.2/binaries/apache-maven-3.6.2-bin.tar.gz
[root@localhost ~] tar -xzvf apache-maven-3.6.2-bin.tar.gz
- 配置環境變量
export MAVEN_HOME=/usr/local/maven/apache-maven-3.6.2
export PATH=$MAVEN_HOME/bin:$PATH
[root@localhost ~] source /etc/profile
- 更換阿里雲鏡像倉庫(也可以使用默認倉庫,忽略此步即可)
在/usr/local/maven/apache-maven-3.6.0/conf/setting.xml
中加入以下配置
<mirrors>
<mirror>
<id>nexus-aliyun</id>
<mirrorOf>*</mirrorOf>
<name>Nexus aliyun</name>
<url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>
</mirrors>
5.docker-ce部署
- 安裝所需的軟件包
[root@localhost ~] yum install -y yum-utils device-mapper-persistent-data lvm2
- 設置穩定的存儲庫
[root@localhost ~] yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
- 安裝docker-ce
[root@localhost ~] yum install docker-ce
[root@localhost ~] systemctl start docker
- 驗證是否安裝成功
[root@localhost ~] docker -v
Docker version 19.03.3, build a872fc2f86
[root@localhost /] docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:c3b4ada4687bbaa170745b3e4dd8ac3f194ca95b2d0518b417fb47e5879d9b5f
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
6.docker-compose部署
curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
- 查看版本
$ docker-compose --version
docker-compose version 1.24.1, build 1110ad01
7.SonarQube部署
- 添加用戶
useradd sonar
passwd sonar
- 優化系統參數
sysctl -w vm.max_map_count=262144
sysctl -w fs.file-max=65536
ulimit -u 4096 sonarqube
ulimit -n 65536 sonarqube
- 安裝
注:最新版本 7.9.1需要 java11才能運行
wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-7.3.zip
mv sonarqube-7.3.zip /usr/local/sonarqube/
chown -R sonar:sonar /usr/local/sonarqube/
su sonar
unzip sonarqube-7.3.zip
啓動
./sonar start,可以設置成系統服務,通過service啓動
8.Harbor部署
- 下載離線安裝包
wget https://storage.googleapis.com/harbor-releases/release-1.9.0/harbor-offline-installer-v1.9.1.tgz
- 解壓安裝包
tar xvf harbor-offline-installer-v1.9.1.tgz
- 修改配置文件harbor.yml
hostname: 192.168.0.100
http:
port: 10030
harbor_admin_password: Harbor12345
database:
password: root123
max_idle_conns: 50
max_open_conns: 100
data_volume: /data/harbor
clair:
updaters_interval: 0
jobservice:
max_job_workers: 20
notification:
webhook_job_max_retry: 20
chart:
absolute_url: disabled
log:
level: info
local:
rotate_count: 50
rotate_size: 200M
location: /data/logs/harbor
_version: 1.9.0
proxy:
http_proxy:
https_proxy:
no_proxy: 127.0.0.1,localhost,.local,.internal,log,db,redis,nginx,core,portal,postgresql,jobservice,registry,registryctl,clair
components:
- core
- jobservice
- clair
關於配置文件的解釋,在官方github上能找到:harbor配置文件詳解
- 安裝harbor
./install.sh
- 啓動 / 停止harbor
docker-compose start
docker-compose stop
9.Rancher部署
Rancher部署直接通過Docker啓動即可
docker run -d --restart=unless-stopped -p 10040:10040 -p 443:443 rancher/rancher
直接登錄http://192.168.0.100訪問Rancher
四、pipeline配置
1.Jenkins
Pipeline plugin 安裝
- GitLab Plugin
- Pipeline
- Blue Ocean
在系統管理-插件管理中搜索以上插件,勾選直接安裝
Global tools配置
Maven、JDK、Git、Docker都按照服務器上的實際配置進行修改。我配置的截圖如下:
創建pipeline
GitLab webhook URL
和 Secret token
要記住,需要填入GitLab的倉庫中去
pipeline從這裏配置的倉庫中獲取Jenkinsfile,每個業務倉庫一個Jenkinsfile,配置特定路徑的Jenkinsfile來完成整個pipeline
全局安全配置
設置匿名用戶可讀,取消“防止跨站點請求僞造”
重啓Jnekins
通過192.168.0.100:10010/restart重啓。
2.GitLab
- 創建jenkinsfile倉庫
新建一個倉庫,用來放所有項目的Jenkinsfile(和業務代碼要分開,不建議放到一個倉庫,Jenkinsfile倉庫由運維來維護,業務代碼由開發來維護)。
將該倉庫的ssh-git以及分支配置到jenkins的pipeline中。
上傳Jenkinsfile文件,按照以下編寫,測試成功以後再對每個步驟進行修改。pipeline語法看這裏
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
-
允許外發配置
該配置需要管理員權限
-
webhook 配置
在設置-集成中設置webhook,這裏的URL和安全令牌,就是上一步在jenkins記錄的GitLab webhook URL
和Secret token
,觸發器就填分支即可。
五、Rancher配置
- 創建集羣
選擇全局-添加集羣-自定義-下一步-勾選Etcd、Control、Worker
,複製命令,在需要加入集羣的主機上運行即可。
創建時也可設置私有鏡像倉庫,填寫URL、用戶名密碼即可。
- 安裝Rancher-Cli
Rancher右下角下載Rancher-Cli for linux,將解壓後的rancher文件通過軟鏈接或者複製到/usr/bin下。
Rancher生成api-key,通過rancher login 登錄rancher,跟rancher之間進行交互,完成部署,更新,回滾等操作。
- 登錄rancher
rancher login https://192.168.0.100/v3 --token token-nt7gn:jvdldgz2cztrwm8wq9fb24sh2p88vslndztxxvb6fxwttmtbqhc4bg
- 安裝kubectl
添加Kubernetes的yum源
cat >> /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF
查看可用版本
yum list kubectl –showduplicates
已加載插件:fastestmirror
base | 3.6 kB 00:00
docker-main | 2.9 kB 00:00
elrepo | 2.9 kB 00:00
epel/x86_64/metalink | 5.0 kB 00:00
epel | 4.7 kB 00:00
extras | 3.4 kB 00:00
kubernetes | 1.3 kB 00:00
updates | 3.4 kB 00:00
(1/5): epel/x86_64/group_gz | 266 kB 00:01
(2/5): epel/x86_64/updateinfo | 851 kB 00:00
(3/5): kubernetes/primary | 6.0 kB 00:01
(4/5): updates/7/x86_64/primary_db | 3.6 MB 00:03
(5/5): epel/x86_64/primary_db | 6.1 MB 00:06
Loading mirror speeds from cached hostfile
* base: mirrors.neusoft.edu.cn
* elrepo: mirrors.tuna.tsinghua.edu.cn
* epel: mirrors.tongji.edu.cn
* extras: mirrors.neusoft.edu.cn
* updates: mirrors.aliyun.com
kubernetes 49/49
可安裝的軟件包
kubectl.x86_64 1.7.5-0 kubernetes
安裝kubectl
yum install -y kubectl.x86_64
查看node
[root@ruhr-gitlab /]# rancher kubectl get node
NAME STATUS ROLES AGE VERSION
ruhr-test-03 Ready controlplane,etcd,worker 8d v1.15.5
[root@ruhr-gitlab /]#
查看pod
[root@ruhr-gitlab /] rancher kubectl get pod
NAME READY STATUS RESTARTS AGE
ruhrtec-cloud-api-85db4f8866-xx927 1/1 Running 16 159m
ruhrtec-cloud-bridge-8698bd746-rbtmm 1/1 Running 1 159m
ruhrtec-cloud-build-d5c765488-j9bj9 1/1 Running 1 159m
ruhrtec-cloud-rail-68cdbcd986-984cd 1/1 Running 1 159m
ruhrtec-cloud-user-5989dc45f6-p9th5 0/1 CrashLoopBackOff 32 159m
ruhrtec-config-7fc869b946-jm44c 1/1 Running 0 159m
ruhrtec-gateway-554f9bfc66-57wdj 1/1 Running 1 159m
ruhrtec-initial-778f67df4-zw695 0/1 CrashLoopBackOff 33 159m
ruhrtec-job-748fc76ffd-z8k27 1/1 Running 1 159m
ruhrtec-monitor-5c47f47854-ht5h5 1/1 Running 1 159m
ruhrtec-register-5685d75d84-xr6rd 1/1 Running 0 159m
ruhrtec-uaa-d868c586-dkld2 1/1 Running 15 159m
[root@ruhr-gitlab /]
啓動pod
通過yaml文件的方式啓動
rancher kubectl create -f yamls/ruhrtec-register.yaml --record
更新pod
rancher kubectl apply -f yamls/ruhrtec-register.yaml --record
查看pod歷史
rancher kubectl rollout history deploy/ruhrtec-register
回滾pod至版本2
rancher kubectl rollout undo deploy/ruhrtec-register --to-revision=2
yaml文件
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: ruhrtec-register
spec:
template:
metadata:
labels:
name: ruhrtec-register
spec:
containers:
- name: ruhrtec-register-container
image: 192.168.0.100:10030/library/ruhrtec-register:latest
ports:
- name: register-port
containerPort: 10010
hostIP:
hostPort: 30010
protocol: TCP
nodeSelector:
slave: "test-03"
六、SonarQube配置
等待補充…
感謝閱讀,有興趣的小夥伴可以關注我的公衆號DevOps探索之旅
,大家一起學習進步