spark 1.6 編譯
1、下載spark1.6源碼
2、安裝maven
3、解壓spark1.6 執行
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
cd spark1.6.0
./make-distribution.sh --name hadoop2.6 --tgz -Psparkr -Phadoop-2.6.0 -Phive -Phive-thriftserver -Pyarn
出現錯誤:
Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.10: Command execution failed. Process exited with an error: 127 (Exit value: 127) -> [Help 1]
解決辦法:
./make-distribution.sh --name hadoop2.6 --tgz -PR -Phadoop-2.6.0 -Phive -Phive-thriftserver -Pyarn
如果想讓spark支持tachyon
make-distribution.sh 文件
SPARK_TACHYON=true