目錄
1、crd介紹
CustomResourceDefinitions,是一種可以定製資源擴展Kubernetes API的使用。
2、使用kubebuilder開發crd
Kubebuilder 是一個使用 CRDs 構建 K8s API 的 SDK,主要是:
- 提供腳手架工具初始化 CRDs 工程,自動生成 boilerplate 代碼和配置;
- 提供代碼庫封裝底層的 K8s go-client;
2.1 準備環境
- go version go1.13.3 linux/amd64
- Docker version 18.06.0-ce, build 0ffa825
- kubectl Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T09:23:26Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
- kubectl Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2019-12-07T21:12:17Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}
- kustomize version:Version: {KustomizeVersion:3.0.3 GitCommit:30b378a9244d757c1cda31b95694fabd6cbdb8e1 BuildDate:2019-07-18T17:45:11Z GoOs:linux GoArch:amd64}
- k8s 集羣
2.2 創建一個項目
命令如下:
kubebuilder init --domain my.domain
需要先用go mod init命令初始化模塊,會在項目根目錄下生成 go.mod 文件
創建完project之後,你會得到如下的代碼包結構
2.3 創建api
命令如下:
kubebuilder create api --group webapp --version v1 --kind Guestbook
發現有報錯,cannot load example.com/m/api/v1: cannot find module providing package example.com/m/api/v1
簡單起見,可以把生成的項目代碼放到$GOPATH/src/example.com/$moduleName下
代碼包結構變成了這樣子:
下面開始我們的重頭戲-編譯
2.4、編譯
第一步:修改DockerFile,記得關掉go mod
改一下dockerfile
# Build the manager binary
FROM golang:1.12.7 as builder
WORKDIR /workspace
# Copy the Go Modules manifests
# COPY go.mod go.mod
# COPY go.sum go.sum
# cache deps before building and copying source so that we don't need to re-download as much
# and so that source changes don't invalidate our downloaded layer
# RUN go mod download
# Copy the go source
# COPY main.go main.go
# COPY api/ api/
# COPY controllers/ controllers/
COPY . .
# Build
RUN go build -a -o manager main.go
# Use distroless as minimal base image to package the manager binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM golang:1.12.7
WORKDIR /
COPY --from=builder /workspace/manager .
ENTRYPOINT ["/manager"]
第二步:修改Makefile
# docker-build: test
docker-build:
make docker-build IMG=testaa:0.1,真正的命令有兩個
1、docker build . -t ${IMG},這個其實是調用DockerFile執行的
2、sed -i'' -e 's@image: .*@image: '"${IMG}"'@' ./config/default/manager_image_patch.yaml
第三步:構建鏡像
make docker build controller:latest
輸出如下:
docker build . -t controller:latest
Sending build context to Docker daemon 68.94MB
Step 1/10 : FROM golang:1.12.7 as builder
---> be63d15101cb
Step 2/10 : WORKDIR /workspace
---> Running in e6b2a339d69d
Removing intermediate container e6b2a339d69d
---> 3d31fe5fe0d3
Step 3/10 : RUN mkdir -p /go/src/example.com/m && cd /go/src/example.com/m
---> Running in a3b5bd085aa6
Removing intermediate container a3b5bd085aa6
---> 6d5b1b2d164e
Step 4/10 : WORKDIR /go/src/example.com/m
---> Running in dbbfdb9460d8
Removing intermediate container dbbfdb9460d8
---> 50b2be51a7f2
Step 5/10 : COPY . .
---> 1d274ef3a35e
Step 6/10 : RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=off go build -a -o manager main.go
---> Running in 4bf405566754
Removing intermediate container 4bf405566754
---> d6c506ecfe82
Step 7/10 : FROM golang:1.12.7
---> be63d15101cb
Step 8/10 : WORKDIR /
---> Running in 5f6b7f9f350e
Removing intermediate container 5f6b7f9f350e
---> 76876c7e986f
Step 9/10 : COPY --from=builder /go/src/example.com/m/manager .
---> 92313bcb89dc
Step 10/10 : ENTRYPOINT ["/manager"]
---> Running in e645ad3860de
Removing intermediate container e645ad3860de
---> 17f7fc4debad
Successfully built 17f7fc4debad
Successfully tagged controller:latest
updating kustomize image patch file for manager resource
sed -i'' -e 's@image: .*@image: '"controller:latest"'@' ./config/default/manager_image_patch.yaml
make: *** No rule to make target `testaa:0.1'. Stop.
備註:
以上步驟可以在本地環境中執行,沒有k8s集羣也可以。
以下步驟需要在k8s集羣中進行,有兩個步驟:
- docker save/load遷移controller:latest鏡像
- 把項目拷貝過去
2.5 安裝CRD
第一步:修改Makefile
去掉install的前置條件,否則還會下載包
第二步:執行命令
make install
其實,真正執行的命令是kubectl apply -f config/crd/bases
備註:
crd的yaml文件在/config/crd/bases/webapp.my.domain_guestbooks.yaml
2.6 啓動controller
第一步:修改Makefile
去掉run的前置條件,否則還會下載包~~
第二步:執行命令
make run
真正執行的命令是go run ./main.go
kubectl get crd
然後我們就可以看到創建的CRD了
2.7 創建一個CRD對象
命令如下:
kubectl apply -f config/samples/
查看CRD對象:
kubectl get guestbooks.webapp.my.domain
2.8 把controller部署到集羣中
命令如下:
make deploy IMG=controller:latest
真正執行的命令是:
kubectl apply -f config/crd/bases(這個實際上就是安裝crd的命令)
kustomize build config/default | kubectl apply -f -(部署這個config/default下的配置文件到kubernertes集羣上)
備註:
在config/default路徑下有kustomization.yaml文件
執行此步驟有如下幾種錯誤,現一一說明,避免重複踩坑
錯誤1:kustomize版本不對
原因是kustomize版本不對,詳見:https://github.com/kubernetes-sigs/kustomize/issues/1422
https://github.com/kubernetes-sigs/kustomize/releases/tag/v3.0.3 下載3.0.3二進制文件
錯誤2:
https://sealyun.com/blog/2019/07/20/crd/ 帖子中已經提到:
以上兩種錯誤解決之後,一般可以成功:
kubectl get svc -n kube-system //查看服務
kubectl get deploy -n kube-system //查看deployment