目的
CockroachDB自 23.1開始，編譯模式逐步放棄了Makefile，改用Bazel構建。實際上在23.1代碼執行make buildshort 能夠成功，但make build應該就會因爲生成js文件有問題而失敗。可惜Bazel構建也不是官網說的那樣簡單，本文詳細描述編譯CockroachDB 23.1.x的編譯過程，供對CockroachDB感興趣的同仁參考。
編譯環境
Rocky Linux 9.2,原因是Rocky Linux 9.2 自帶各種模塊版本較高，符合cockroachdb編譯要求，並且外圍模塊趨於穩定，不會出現9.0時因爲openssl版本問題導致nodejs需要降級的異常操作
安裝RockyLinux 9.2 時，僅需選擇Server環境，然後選擇開發工具和效率工具即可。
CockroachDB版本：官網的release-23.1分支，今年國內訪問github的情況有所改善，雖然不穩定，但是多試幾次，尤其是ssh模式還是可以下載代碼的
官方參考資料
準備和檢查編譯環境
-
1、克隆代碼庫
採用ssh模式克隆cockroachdb的release-23.1（本文假設您已經有了一個github賬號）
git clone -b release-23.1 [email protected]:cockroachdb/cockroach.git
![image]()
不要嫌慢，能下載就是好事兒，我的網絡是天津聯通，github下載速度再10K~2M之間 -
2、下載的同時，可以按官網逐項對比Linux x86平臺的環境準備
-
2.1、A C++ compiler that supports C++11. Note that GCC prior to 6.0 doesn't work due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=48891。缺省已經安裝的gcc版本是11.3.1，是支持C++11標準的
g++ -v
gcc version 11.3.1 20221121 (Red Hat 11.3.1-4) (GCC)
- 2.2、The standard C/C++ development headers on your system. 同2.1 缺省已經安裝頭文件
- 2.3、On GNU/Linux, the terminfo development libraries, which may be part of a ncurses development package (e.g.
libncurses-devon Debian/Ubuntu, butncurses-develon CentOS). You also need the libresolv-wrapper package on Ubuntu < 22.04. rocky linux缺省沒有安裝ncurses-devel，需要執行dnf安裝
dnf install ncurses-devel
- 2.4、Git 1.9+, we recommend v2.20+，缺省已經安裝
git -v
git version 2.39.1
- 2.5、Bash (4+ is preferred) ，缺省已經安裝
bash --version
GNU bash, version 5.1.8(1)-release (x86_64-redhat-linux-gnu)
- 2.6、GNU Make (4.2+ is known to work)，缺省已經安裝
make -v
GNU Make 4.3
- 2.7、CMake 3.20.* (on Macs 3.21+ are also known to work)。缺省沒有安裝cmake，需要執行dnf安裝
dnf install cmake
cmake --version
cmake version 3.20.2
- 2.8、Autoconf 2.68+
autoconf --version
autoconf (GNU Autoconf) 2.69
- 2.9、Yacc or Bison
bison --version
bison (GNU Bison) 3.7.4
- 2.10、GNU Patch 2.7+
patch -v
GNU patch 2.7.6
- 2.11、Bazelisk (you can use Bazel directly instead of Bazelisk, although Bazelisk makes sure you are using the same version that we use in CI)
安裝Bazelisk有多種方式，我們採用安裝nodejs，然後用npm安裝Bazelisk
dnf install nodejs
node -v
v16.19.1
npm -v
8.19.3
設置國內代理並驗證
npm config set registry http://registry.npm.taobao.org/
npm get registry
http://registry.npm.taobao.org/
安裝Bazelisk
npm install -g @bazel/bazelisk
安裝完成後檢查
whereis bazel
bazel: /usr/local/bin/bazel
whereis bazelisk
bazelisk: /usr/local/bin/bazelisk
- 3 開始構建
等待clone代碼完成，就開始構建 - 3.1 首先運行 dev doctor 檢查代碼環境
cd cockroach
./dev doctor
/home/ansible/works/cockroach/bin/dev-versions/dev.84 not found, building...
2023/09/06 10:29:45 Downloading https://github.com/cockroachdb/bazel/releases/download/6.2.1/bazel-6.2.1-linux-x86_64...
長時間等待後
Starting local Bazel server and connecting to it...
INFO: Analyzed target //pkg/cmd/dev:dev (86 packages loaded, 10730 targets configured).
INFO: Found 1 target...
Target //pkg/cmd/dev:dev up-to-date:
_bazel/bin/pkg/cmd/dev/dev_/dev
INFO: Elapsed time: 254.017s, Critical Path: 12.70s
INFO: 20 processes: 5 internal, 15 linux-sandbox.
INFO: Build completed successfully, 20 total actions
INFO: Invocation ID: 5039d549-7ae7-4c4f-86b6-8695625c41fc
=============================
=== RUNNING DOCTOR CHECKS ===
=============================
INFO: Invocation ID: 27237479-0160-45db-92af-83445ca320d0
INFO: Build option --//build/toolchains:nogo_disable_flag has changed, discarding analysis cache.
INFO: Analyzed target //build/bazelutil:test_nogo_configured (1 packages loaded, 29 targets configured).
INFO: Found 1 target...
Target //build/bazelutil:test_nogo_configured up-to-date:
_bazel/bin/build/bazelutil/test_nogo_configured.sh
INFO: Elapsed time: 0.465s, Critical Path: 0.01s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
Configuring cache...
INFO: Build option --//build/toolchains:nogo_disable_flag has changed, discarding analysis cache.
INFO: Analyzed target @com_github_buchgr_bazel_remote//:bazel-remote (195 packages loaded, 12422 targets configured).
INFO: Found 1 target...
Target @com_github_buchgr_bazel_remote//:bazel-remote up-to-date:
_bazel/bin/external/com_github_buchgr_bazel_remote/bazel-remote_/bazel-remote
INFO: Elapsed time: 63.612s, Critical Path: 29.20s
INFO: 206 processes: 7 internal, 199 linux-sandbox.
INFO: Build completed successfully, 206 total actions
Using cache configuration file at /home/ansible/.cache/dev-bazel-remote/config.yml
報告成功
我在一臺虛擬機上針對不同提交版本多次執行檢查，有可能報告一些錯誤，但進行交互後都能成功，遇到過如下情況
- DOCTOR >> Which config you want to use (dev,crosslinux)? [dev] > 輸入dev
- DOCTOR >> Do you want to use the
lintonbuildconfiguration? [y] > 輸入 y - DOCTOR >> Please set a tmpdir that will be used for storing test artifacts [/tmp/cockroach] > 直接回車
- DOCTOR >> Do you want me to update your ~/.bazelrc for you to configure the loopback cache? [y] > 直接回車
- 3.2 編譯主程序
./dev build
$ bazel build //pkg/cmd/cockroach:cockroach
INFO: Invocation ID: c009abbc-9b9f-493a-b491-a86df5a3a2c5
INFO: Build options --//build/toolchains:nogo_disable_flag and --run_under have changed, discarding analysis cache.
INFO: Analyzed target //pkg/cmd/cockroach:cockroach (4313 packages loaded, 48317 targets configured).
INFO: Found 1 target...
ERROR: /home/ansible/works/cockroach/pkg/ccl/gssapiccl/BUILD.bazel:4:11: GoCompilePkg pkg/ccl/gssapiccl/gssapiccl.a failed: (Exit 1): builder failed: error executing command (from target //pkg/ccl/gssapiccl:gssapiccl) bazel-out/k8-opt-exec-2B5CBBC6/bin/external/go_sdk/builder_reset/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -tags bazel,gss,bazel,gss -src pkg/ccl/gssapiccl/gssapi.go -src ... (remaining 53 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
/usr/bin/ld.gold: error: cannot find -lresolv_wrapper
external/archived_cdep_libkrb5_linux/lib/libkrb5.a(dnsglue.o):dnsglue.c:function krb5int_dns_init: error: undefined reference to '__res_nsearch'
collect2: error: ld returned 1 exit status
compilepkg: error running subcommand /usr/bin/gcc: exit status 1
Target //pkg/cmd/cockroach:cockroach failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 282.874s, Critical Path: 177.14s
INFO: 2913 processes: 38 internal, 2875 linux-sandbox.
INFO: Build Event Protocol files produced successfully.
FAILED: Build did NOT complete successfully
ERROR: exit status 1
可惜，編譯失敗了，報告libkrb5.a編譯失敗，原因是rockylinux 9.2 上的makefile對krb5不友好，一些宏的判斷有問題，此處問題解決有兩個辦法：
第一種修改方式
vi .bazelrc
刪除gss編譯項
第二種修改方式
因爲我們僅僅是爲了研究cockroachdb的核心技術，也可以簡單刪除krb5涉及的代碼，具體執行如下
- 1、刪除工程中唯一一處使用krb5的代碼
rm pkg/ccl/gssapiccl/ -rf
- 2、修改ccl中引用gssapiccl的地方
vi pkg/ccl/BUILD.bazel
vi pkg/ccl/ccl_init.go
然後重新執行編譯程序
[ansible@xtp2 cockroach]$ ./dev build
$ bazel build //pkg/cmd/cockroach:cockroach
INFO: Invocation ID: 1eb1ac48-7ac9-4d50-8b56-b1b8911c4e8b
INFO: Analyzed target //pkg/cmd/cockroach:cockroach (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
Target //pkg/cmd/cockroach:cockroach up-to-date:
_bazel/bin/pkg/cmd/cockroach/cockroach_/cockroach
INFO: Elapsed time: 40.259s, Critical Path: 38.21s
INFO: 4 processes: 1 internal, 3 linux-sandbox.
INFO: Build Event Protocol files produced successfully.
INFO: Build completed successfully, 4 total actions
Successfully built binary for target //pkg/cmd/cockroach:cockroach at cockroach
然後驗證
./cockroach --version
cockroach version details:
Build Tag: v23.1.10-dev
Build Time:
Distribution: CCL
Platform: linux amd64
Go Version: go1.19.10
C Compiler: gcc 11.3.1 20221121 (Red Hat 11.3.1-4)
Build Commit ID:
Build Type:
Enabled Assertions: false
(use './cockroach version --build-tag' to display only the build tag)
再然後就按手冊測試更多的功能了
按版本驗證的角度，我們應該運行 ./dev test （對應之前的make test）進行完整驗證，可惜 ./dev test 運行過程中佔滿了我的100G虛擬機磁盤，最後執行失敗。
總結
- 1、在Rockylinux9.2上，編譯libkrb5有問題，需要修改bazelrc文件
- 2、執行單元測試 ./dev test 失敗，原因是生成了.cache/bazel 80G以上的文件，導致沒有磁盤剩餘空間執行終止
- 3、建議，如果不關注cockroach ui，僅僅學習研究cockroach的主程序，還是採用傳統的make buildshort， make test 吧