在編碼過程中遇到的問題總結:
問題1 :cmake 函數庫變更引起的問題:
問題描述:
[WARNING] /Users/wyx/Downloads/hadoop-rel-release-3.1.1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:1227:12: error: no matching function for call to 'getgrouplist'
[WARNING] int rc = getgrouplist(user, pw->pw_gid, groups, &ngroups);
[WARNING] ^~~~~~~~~~~~
[WARNING] /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/unistd.h:653:6: note: candidate function not viable: no known conversion from 'gid_t *' (aka 'unsigned int *') to 'int *' for 3rd argument
[WARNING] int getgrouplist(const char *, int, int *, int *);
修改 c 文件:
vim hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c 中 getgrouplist函數調用的 groups參數,強轉成(int *)
具體如下所示:
1384 int rc = getgrouplist(user, pw->pw_gid, (int *)groups, &ngroups);
1385 if (rc < 0) {
1386 groups = (gid_t *) alloc_and_clear_memory(ngroups, sizeof(gid_t));
1387 if (groups == NULL) {
1388 fprintf(ERRORFILE, "Failed to allocate buffer for group lookup for user %s.\n", user);
1389 exit(OUT_OF_MEMORY);
1390 }
1391 if (getgrouplist(user, pw->pw_gid, (int * )groups, &ngroups) == -1) {
1392 fprintf(ERRORFILE, "Fail to lookup groups for user %s.\n", user);
1393 ret = 2;
1394 }
1395 }
問題2:由於 openssl 環境變量引起的
問題描述:
1 2 3 4 5 6 7 8 9 10 11 |
|
由於我本機之前裝過 openssl,但配置不完全,解決方案如下:
1. 先查看 openssl 的安裝目錄:
解決方案:
(base) bogon:hadoop-3.1.2-src johnnybai$ brew list openssl
/usr/local/Cellar/openssl/1.0.2o_1/.bottle/etc/ (8 files)
/usr/local/Cellar/openssl/1.0.2o_1/bin/c_rehash
/usr/local/Cellar/openssl/1.0.2o_1/bin/openssl
/usr/local/Cellar/openssl/1.0.2o_1/include/openssl/ (74 files)
/usr/local/Cellar/openssl/1.0.2o_1/lib/libcrypto.1.0.0.dylib
/usr/local/Cellar/openssl/1.0.2o_1/lib/libssl.1.0.0.dylib
/usr/local/Cellar/openssl/1.0.2o_1/lib/engines/ (12 files)
/usr/local/Cellar/openssl/1.0.2o_1/lib/pkgconfig/ (3 files)
/usr/local/Cellar/openssl/1.0.2o_1/lib/ (4 other files)
/usr/local/Cellar/openssl/1.0.2o_1/share/man/ (1680 files)
2. 配置 openssl 環境變量
sudo vim /etc/profile
添加如下環境變量:
# openssl
export OPENSSL_ROOT_DIR="/usr/local/Cellar/openssl/1.0.2o_1"
export OPENSSL_INCLUDE_DIR="/usr/local/Cellar/openssl/1.0.2o_1/include"
編譯成功之後,可以在 hadoop-3.1.2-src/hadoop-dist/target/ 目錄下找到安裝部署binary 文件: hadoop-3.1.2.tar.gz
注: 編譯步驟,部分參照了 https://github.com/shawfdong/hyades/wiki/Building-Hadoop-from-source 的做法。