hadoop-2.9.2 window 源碼編譯記錄

依賴項

linux安裝的請轉 https://blog.csdn.net/miracle_8/article/details/105111982
參考hadoop-2.9.2-src/BUILDING.txt

* Windows System
* JDK 1.8  
* Maven 3.0 or later 
* ProtocolBuffer 2.5.0  
* CMake 3.1 or newer   path
* Visual Studio 2010 Professional or Higher  (2010已經足夠了)
* Windows SDK 8.1 (if building CPU rate control for the container executor) OK
* zlib headers (if building native code bindings for zlib) zlib_home 
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip. These
  tools must be present on your PATH.  git替代 path
* Python ( for generation of docs using 'mvn site')

Unix command-line tools are also included with the Windows Git package which
can be downloaded from http://git-scm.com/downloads

If using Visual Studio, it must be Professional level or higher.
Do not use Visual Studio Express.  It does not support compiling for 64-bit,
which is problematic if running a 64-bit system.

The Windows SDK 8.1 is available to download at:

http://msdn.microsoft.com/en-us/windows/bg162891.aspx

Cygwin is not required.

----------------------------------------------------------------------------------
Building:

Keep the source code tree in a short path to avoid running into problems related
to Windows maximum path length limitation (for example, C:\hdc).

There is one support command file located in dev-support called win-paths-eg.cmd.
It should be copied somewhere convenient and modified to fit your needs.

win-paths-eg.cmd sets up the environment for use. You will need to modify this
file. It will put all of the required components in the command path,
configure the bit-ness of the build, and set several optional components.

Several tests require that the user must have the Create Symbolic Links
privilege.

All Maven goals are the same as described above with the exception that
native code is built by enabling the 'native-win' Maven profile. -Pnative-win
is enabled by default when building on Windows since the native components
are required (not optional) on Windows.

If native code bindings for zlib are required, then the zlib headers must be
deployed on the build machine. Set the ZLIB_HOME environment variable to the
directory containing the headers.

set ZLIB_HOME=C:\zlib-1.2.7

At runtime, zlib1.dll must be accessible on the PATH. Hadoop has been tested
with zlib 1.2.7, built using Visual Studio 2010 out of contrib\vstudio\vc10 in
the zlib 1.2.7 source tree.

http://www.zlib.net/

----------------------------------------------------------------------------------
Building distributions:

 * Build distribution with native code    : mvn package [-Pdist][-Pdocs][-Psrc][-Dtar][-Dmaven.javadoc.skip=true]

依賴項說明：

1. jdk ： 必須1.8（其他版本沒有測試過），安裝後需要將bin目錄加入到PATH中
2. Maven ： 必須3.0（其他版本沒有測試過），安裝後需要將bin目錄加入到PATH中 https://maven.apache.org/download.cgi
3. ProtocolBuffer ： 必須2.5（其他版本沒有測試過）。需要下載的分別是protobuf-2.5.0和protoc-2.5.0-win32，需要將protoc-2.5.0-win32中的protoc.exe加入到protobuf-2.5.0/src目錄。至於protobuf-2.5.0/java 目錄下的內容是否需要執行mvn install 沒測試過，反正本人是執行過的
4. CMake ： 必須是3.1（其他版本沒有測試過），安裝後需要將bin目錄加入到PATH中 https://cmake.org/download/
5. Visual Studio ： 2010足夠了，不需要安裝高版本 https://www.malavida.com/en/soft/visual-studio-2010/
6. Windows SDK ：8.1足夠了，不需要安裝高版本
7. zlib-header： 編譯 native.sln的時候需要ZLIB_HOME變量，需要設置變量ZLIB_HOME=zlib-header所在路徑。在運行的時候，需要將 zlib1.dll加入到path中（At runtime, zlib1.dll must be accessible on the PATH）
8. Unix command-line tools：可以用git bash 替代，安裝後需要將bin目錄加入到PATH中

安裝命令

必須通過 visual studio 2010 command prompt 運行如下命令， 因爲編譯過程中需要執行msbuild命令

mvn clean compile package -Pdist,native-win -DskipTests -Dmaven.javadoc.skip=true
mvn package -Pdist -DskipTests -Dmaven.javadoc.skip=true

安裝過程中出現的問題

1. （模塊 : Project Dist POM ）提示bash dist-copynativelibs ****
   原因： 可能是因爲安裝的有問題，bash命令無效，需要將bash替換成sh
   解決：將超父類的pom.xml配置文件中的shell-executable由bash改爲sh 即可 。hadoop-2.9.2-src/pom.xml
   

2. （模塊： Common Project）提示native.h中找不到zlib.h文件（如果遵從上文的依賴，應該是不會出現的）
   原因: zlib.h頭文件信息讀取不到，環境變量ZLIB_HOME未配置
   解決: 配置ZLIB_HOME=zlib的頭文件

3. Could not find artifact com.amazonaws:DynamoDBLocal:jar:1.11.86 ，jar包找不到
   原因：配置的maven 鏡像中找不到此包 本人配置的鏡像是阿里雲 (https://maven.aliyun.com/nexus/content/groups/public/)
   解決：可以手動導入，或者採用其他可用的鏡像

<repository>
     <id>dynamodb-local-oregon</id>
     <name>DynamoDB Local Release Repository</name>
     <url>https://s3-us-west-2.amazonaws.com/dynamodb-local/release</url>
</repository>
// 配置過此repository之後，會去下載jar包，但是可能還會提示錯誤，需要將本地倉庫中對應包目錄下的_remote.repositories 文件刪除，之後纔會去讀取本地的jar包文件，否則還是會去鏡像中查找的

注：

1. 本人3.1.2也編譯過，但是最後的時候因爲3.1.2中的bash版本過高，導致最後會提示dist-tools-hooks-maker這個文件 語法錯誤，雖然能解決，但是懶得改，於是就沒有調整了
   2