jenkins X實踐系列（2） —— 基於jx的DevOps實踐

jx是雲原生CICD，devops的一個最佳實踐之一，目前在快速的發展成熟中。最近調研了JX，這裏爲第2篇，使用已經安裝好的jx來實踐CICD，旨在讓大家瞭解基於jx的DevOps是如何運轉的，感興趣的可以繼續關注，下一篇介紹如何安裝。

先上圖：

一、windows 搭建開發環境（可選）

1. 安裝kubectl

使用Chocolatey來安裝，因此install Chocolatey：

安裝Chocolatey

windows + X ，選擇power shell 管理員模式，輸入：

Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

或者cmd.exe（管理員）：

@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

使用Chocolatey安裝kubectl

使用腳本安裝 choco install kubernetes-cli
測試是否安裝成功: kubectl version
創建配置文件，先轉到%HOME% 目錄: For example: cd C:\users\yourusername
創建配置文件目錄: mkdir .kube
轉到配置目錄: cd .kube
配置kubectl使用遠程集羣: New-Item config -type file 修改config文件，或者直接從集羣拷貝配置文件過來
測試

C:\Users\jqpeng>kubectl get pods
NAME                                                READY   STATUS      RESTARTS   AGE
jenkins-bd94b5fb8-5t9kq                             1/1     Running     0          10d
jenkins-x-chartmuseum-75d45b6d7f-2hk99              1/1     Running     0          10d
jenkins-x-controllercommitstatus-675dbb9c86-kth6q   1/1     Running     71         10d
jenkins-x-controllerrole-5458874c-4lnwh             1/1     Running     0          10d
jenkins-x-controllerteam-7f965c8b9c-n4kfm           1/1     Running     0          10d
jenkins-x-controllerworkflow-7675c458d-sjbfd        1/1     Running     0          10d

2.安裝helm

到下載頁： https://github.com/helm/helm/releases

下載最新的helm,選擇windows-amd64

wget https://storage.googleapis.com/kubernetes-helm/helm-v2.11.0-windows-amd64.zip

解壓後，拷貝到C:\Windows\System32。

注意，需要Cross GFW,可以使用（ https://azure.microsoft.com/zh-cn/free/ ）免費一年。

初始化helm

服務端已經安裝過tiller了，因此只需要client-only，另外stable repo指定本地的

helm init --client-only --stable-repo-url=http://charts.youdomain.com/

---
Creating C:\Users\jqpeng\.helm
Creating C:\Users\jqpeng\.helm\repository
Creating C:\Users\jqpeng\.helm\repository\cache
Creating C:\Users\jqpeng\.helm\repository\local
Creating C:\Users\jqpeng\.helm\plugins
Creating C:\Users\jqpeng\.helm\starters
Creating C:\Users\jqpeng\.helm\cache\archive
Creating C:\Users\jqpeng\.helm\repository\repositories.yaml
Adding stable repo with URL: http://charts.youdomain.com/
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at C:\Users\jqpeng\.helm.
Not installing Tiller due to 'client-only' flag having been set
Happy Helming!

3. 安裝jx

到https://github.com/jenkins-x/jx/releases 下載最新的編譯好的jx執行文件，選擇jx-windows-amd64.zip下載，下載後解壓，然後重新命名爲jx.exe，拷貝到C:\Windows\System32。

測試:

C:\Users\jqpeng>jx version
Updated the team settings in namespace incubation
Updated the team settings in namespace incubation
NAME               VERSION
jx                 [32m1.3.572[0m
jenkins x platform [32m0.0.2871[0m
Kubernetes cluster [32mv1.12.2[0m
kubectl            [32mv1.12.1[0m
helm client        [32mv2.11.0+g2e55dbe[0m
helm server        [32mv2.11.0+g2e55dbe[0m
git                [32mgit version 2.19.1.windows.1[0m

將服務器上的.jx目錄下相關配置信息下載到本地用戶目錄下的.jx文件夾。

二、快速開始

1.使用quickstart

作爲演示用，可以直接使用jx create quickstart,會直接使用官方的quickstart項目，鍵入命令，然後按提示選擇即可。

D:\Project>jx create quickstart
? select the quickstart you wish to create spring-boot-rest-prometheus
? Project name spring-boot-rest-prometheus
Generated quickstart at D:\Project\spring-boot-rest-prometheus
### NO charts folder D:\Project\spring-boot-rest-prometheus\charts\spring-boot-rest-prometheus
Created project at D:\Project\spring-boot-rest-prometheus

Updated the team settings in namespace incubation
? Which Git service do you wish to use https://github.com
No username defined for the current Git server!
? Do you wish to use jadepeng as the Git user name: Yes
The directory D:\Project\spring-boot-rest-prometheus is not yet using git
? Would you like to initialise git now? Yes
? Commit message:  Initial import

Git repository created
Updated the team settings in namespace incubation
selected pack: C:\Users\jqpeng\.jx\draft\packs\github.com\jenkins-x\draft-packs\packs\maven
? Which organisation do you want to use? jadepeng
replacing placeholders in directory D:\Project\spring-boot-rest-prometheus
app name: spring-boot-rest-prometheus, git server: github.com, org: jadepeng, Docker registry org: jadepeng
skipping directory "D:\\Project\\spring-boot-rest-prometheus\\.git"
Using Git provider [32mgithub.com at https://github.com[0m


About to create repository [32mspring-boot-rest-prometheus[0m on server [32mhttps://github.com[0m with user [32mjadepeng[0m
? Enter the new repository name:  spring-boot-rest-prometheus


Creating repository [32mjadepeng/spring-boot-rest-prometheus[0m
Pushed Git repository to https://github.com/jadepeng/spring-boot-rest-prometheus

Updated the team settings in namespace incubation
? Do you wish to use jadepeng as the user name for the Jenkins Pipeline Yes
Created Jenkins Project: http://jenkins.incubation.youdomain.com/job/jadepeng/job/spring-boot-rest-prometheus/

Watch pipeline activity via:    jx get activity -f spring-boot-rest-prometheus -w
Browse the pipeline log via:    jx get build logs jadepeng/spring-boot-rest-prometheus/master
Open the Jenkins console via    jx console
You can list the pipelines via: jx get pipelines
When the pipeline is complete:  jx get applications

For more help on available commands see: https://jenkins-x.io/developing/browsing/

Note that your first pipeline may take a few minutes to start while the necessary images get downloaded!

Creating GitHub webhook for jadepeng/spring-boot-rest-prometheus for url http://jenkins.incubation.youdomain.com/github-webhook/

創建過程：

自動創建了spring-boot-rest-prometheus項目，並提交到github（可以使用自己的git服務器），並自動創建GitHub webhook ，這樣當新代碼提交到github後，會自動觸發構建。
將項目提交到jenkins，可以打開http://jenkins.incubation.youdomain.com/job/jadepeng/job/spring-boot-rest-prometheus/ 查看
jenkins會自動進行構建，可以通過jx get activity -f spring-boot-rest-prometheus -w查看構建活動
通過jx get build logs jadepeng/spring-boot-rest-prometheus/master查看構建日誌,
通過jx console打開jenkins bule

2.查看構建日誌

我們查看構建日誌：

jx get build logs jadepeng/spring-boot-rest-prometheus/master


....

+ jx step helm release
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Using helmBinary helm with feature flag: none
Adding missing Helm repo: jenkins-x http://chartmuseum.jenkins-x.io
Successfully added Helm repository jenkins-x.
Adding missing Helm repo: releases http://jenkins-x-chartmuseum:8080
Successfully added Helm repository releases.
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Uploading chart file spring-boot-rest-prometheus-0.0.1.tgz to http://jenkins-x-chartmuseum:8080/api/charts
Received 201 response: {"saved":true}
[Pipeline] sh
[spring-boot-rest-prometheus] Running shell script
++ cat ../../VERSION
+ jx promote -b --all-auto --timeout 1h --version 0.0.1
Using helmBinary helm with feature flag: none
Promoting app spring-boot-rest-prometheus version 0.0.1 to namespace incubation-staging
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34

3.部署到staging環境

可以看到已經構建成功，並停留在最後一步jx promote -b --all-auto --timeout 1h --version 0.0.1

jx promote 是jx的發佈命令，將構建產物部署到k8s環境。回顧下文章開始的一個圖，提交到master後的代碼，自動構建後會部署到staging，由於採用的gitops，會往staging環境的git倉庫 environment-walkertabby-staging推送一個pullrequest：

environment-walkertabby-staging其實就是一個charts項目：

每提交一個pullrequest，其實就是在requirements.yaml，將需要部署的項目作爲-staging環境的依賴，比如剛提交的pullrequest就是增加了spring-boot-rest-prometheus依賴。

@@ -13,6 +13,9 @@ dependencies:
+
 - name: pailitaoservice
+
   repository: http://jenkins-x-chartmuseum:8080
+
   version: 0.0.3
+
+- name: spring-boot-rest-prometheus
+
+  repository: http://jenkins-x-chartmuseum:8080
+
+  version: 0.0.1
+
 - name: springboot-rest-demo
+
   repository: http://jenkins-x-chartmuseum:8080
+
   version: 0.0.4

我們來同意下合併pullrequest，這樣當staging項目構建後就會自動部署spring-boot-rest-prometheus。合併完成後，再回到日誌查看：

...
+ jx promote -b --all-auto --timeout 1h --version 0.0.1
Using helmBinary helm with feature flag: none
Promoting app spring-boot-rest-prometheus version 0.0.1 to namespace incubation-staging
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34

Pull Request http://github.youdomain.com/jqpeng/environment-walkertabby-staging/pulls/34 is merged at sha 906a33b6eec14a49d248d8220e0d88416798ba6e
merge status: pending for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-staging/statuses/906a33b6eec14a49d248d8220e0d88416798ba6e with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-staging/job/master/display/redirect description: Build queued...
merge status: success for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-staging/statuses/906a33b6eec14a49d248d8220e0d88416798ba6e with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-staging/job/master/3/display/redirect description: This commit looks good
Merge status checks all passed so the promotion worked!
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] cleanWs
[WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline

jx已經檢測到Pull Request已經通過，並且啓動staging構建，直到構建結束。

這個時候，我們就可以通過jx get applications來查看部署好的應用。

D:\Project>jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL     
spring-boot-rest-prometheus               0.0.1        http://spring-boot-rest-prometheus.incubation-staging.youdomain.com

PODS 爲0，應該是容器啓動有問題，我們可以看下：

kubectl -n=incubation-staging describe pod incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf
---

Events:
  Type     Reason     Age                    From                 Message
  ----     ------     ----                   ----                 -------
  Normal   Scheduled  6m51s                  default-scheduler    Successfully assigned incubation-staging/incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf to docker86-9
  Normal   Pulling    6m44s                  kubelet, docker86-9  pulling image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1"
  Normal   Pulled     6m38s                  kubelet, docker86-9  Successfully pulled image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1"
  Normal   Created    5m27s (x3 over 6m37s)  kubelet, docker86-9  Created container
  Normal   Started    5m27s (x3 over 6m37s)  kubelet, docker86-9  Started container
  Warning  Unhealthy  5m1s (x9 over 6m31s)   kubelet, docker86-9  Readiness probe failed: Get http://170.22.78.7:8080/actuator/health: dial tcp 170.22.78.7:8080: connect: connection refused
  Normal   Pulled     4m26s (x3 over 6m10s)  kubelet, docker86-9  Container image "registry.youdomain.com/jadepeng/spring-boot-rest-prometheus:0.0.1" already present on machine
  Warning  BackOff    97s (x15 over 5m41s)   kubelet, docker86-9  Back-off restarting failed container

問題在於Readiness probe failed, 來查看下容器日誌：

kubectl -n=incubation-staging logs incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.0.4.RELEASE)

2018-11-20 03:39:45.149  INFO 1 --- [           main] c.g.j.q.s.r.p.RestPrometheusApplication  : Starting RestPrometheusApplication v0.0.1 on incubation-staging-spring-boot-rest-prometheus-66966b8cbb-8fvmf with PID 1 (/opt/app.jar started by root in /opt)
2018-11-20 03:39:45.442  INFO 1 --- [           main] c.g.j.q.s.r.p.RestPrometheusApplication  : No active profile set, falling back to default profiles: default
2018-11-20 03:39:47.942  INFO 1 --- [           main] ConfigServletWebServerApplicationContext : Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@424c0bc4: startup date [Tue Nov 20 03:39:47 UTC 2018]; root of context hierarchy

原來java程序還沒有啓動成功，有可能是默認的資源限制問題，我們去掉相關限制，修改deployment.yaml，刪除resources限制：

git push到倉庫，等待自動構建完成，再次查看：

D:\Project>jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL     
spring-boot-rest-prometheus               0.0.2   1/1  http://spring-boot-rest-prometheus.incubation-staging.youdomain.com 

D:\Project>curl -l http://spring-boot-rest-prometheus.incubation-staging.youdomain.com/
{"hello":"world"}

可以看到，自動構建了0.0.2版本，並且已經部署成功

4. 部署到正式環境

如果在staging測試沒有問題，可以推送到正式環境（production）

jx promote spring-boot-rest-prometheus  --version 0.0.2 --env production

該命令會往production環境發送一個pullrequest，同意後會部署到正式環境。

jx promote spring-boot-rest-prometheus  --version 0.0.2 --env production
Promoting app spring-boot-rest-prometheus version 0.0.2 to namespace incubation-production
? Do you wish to use jqpeng as the user name to submit the Pull Request Yes
Created Pull Request: http://github.youdomain.com/jqpeng/environment-walkertabby-production/pulls/2

pipeline jqpeng/environment-jx-dev/master
Pull Request http://github.youdomain.com/jqpeng/environment-walkertabby-production/pulls/2 is merged at sha 1f2ab843a8037f353020a81ad4289c9ede550447
merge status: pending for URL http://github.youdomain.com/api/v1/jqpeng/environment-walkertabby-production/statuses/1f2ab843a8037f353020a81ad4289c9ede550447 with target: http://jenkins.incubation.youdomain.com/job/jqpeng/job/environment-walkertabby-production/job/master/display/redirect description: Build queued...

然後再jx get applications

jx get applications
APPLICATION                 EDIT PODS URL STAGING PODS URL                                                                    PRODUCTION PODS URL
spring-boot-rest-prometheus               0.0.2   1/1  http://spring-boot-rest-prometheus.incubation-staging.youdomain.com 0.0.2      1/1  http://spring-boot-rest-prometheus.incubation-production.youdomain.com

可以看到0.0.2已經在PRODUCTIONS環境了，測試一下：

curl -l http://spring-boot-rest-prometheus.incubation-production.youdomain.com
{"hello":"world"}

三、爲項目增加新功能

gitops提倡增加新功能，先在新的分支測試通過後，然後提交PR到master分支，因此我們先創建一個新分支jqpeng-dev。

git checkout -b jqpeng-dev
Switched to a new branch 'jqpeng-dev'

hello jx

修改RestPrometheusApplication.java

@GetMapping(path = "/", produces = "application/json")
    @ResponseBody
    public Map<String, Object> landingPage() {
        Counter.builder("mymetric").tag("foo", "bar").register(registry).increment();
        return singletonMap("hello", "world");
    }

修改爲

@GetMapping(path = "/", produces = "application/json")
    @ResponseBody
    public Map<String, Object> landingPage() {
        Counter.builder("mymetric").tag("foo", "bar").register(registry).increment();
        return singletonMap("hello", "jx");
    }

保存，提交

git commit -a -m ''
git push origin jqpeng-dev

提交一個PR：

jx create pullrequest -t "#pr1 hello jx"
? Do you wish to use jadepeng as the user name to use for authenticating with git issues Yes

Created PullRequest #1 at https://github.com/jadepeng/spring-boot-rest-prometheus/pull/1

可以到https://github.com/jadepeng/spring-boot-rest-prometheus/pull/1/files查看代碼變更。

提交PR後，jx會自動構建PR，並部署preview環境，可以打開jenkins查看：

或者通過命令查看：

jx get build log jadepeng/spring-boot-rest-prometheus/PR-1

等自動構建完成，jx會在pr下面提交一個帶預覽地址的評論：

點看鏈接查看：

已經是hello jx了！

合併PR

打開PR頁面，點擊Merge pull request：

填寫合併日誌，提交：

PR1已經合併到master分支，等待自動構建完成，剩下的就和上面“二、快速開始”裏的一樣了，在staging環境進行測試，沒問題的發佈到生產環境。

四、小結

jx 良好的實現了gitops，利於團隊協作，可以在團隊進行推廣實施

jenkins X實踐系列（2） —— 基於jx的DevOps實踐

一、windows 搭建開發環境（可選）

1. 安裝kubectl

安裝Chocolatey

使用Chocolatey安裝kubectl

2.安裝helm

初始化helm

3. 安裝jx

二、快速開始

1.使用quickstart

2.查看構建日誌

3.部署到staging環境

4. 部署到正式環境

三、爲項目增加新功能

hello jx

合併PR

四、小結

【SQL進階】CASE語句的使用

K8S集羣安裝

jenkins X實踐系列（1） —— 背景知識

jenkins X實踐系列（2） —— 基於jx的DevOps實踐

XNginx - nginx 集羣可視化管理工具

google gcr.io、k8s.gcr.io 國內鏡像

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結