windows下使用maven编译spark源码

1.环境

jdk1.8

scala2.11.8

spark2.11

maven4.0

2.github客户端

官方给出的命令是在Linux上的

./build/mvn -DskipTests clean package

调用shell脚本取只想一些命令然后在用maven编译,一些过程如下

install_app() {
  local remote_tarball="$1/$2"
  local local_tarball="${_DIR}/$2"
  local binary="${_DIR}/$3"

  # setup `curl` and `wget` silent options if we're running on Jenkins
  local curl_opts="-L"
  local wget_opts=""
  if [ -n "$AMPLAB_JENKINS" ]; then
    curl_opts="-s ${curl_opts}"
    wget_opts="--quiet ${wget_opts}"
  else
    curl_opts="--progress-bar ${curl_opts}"
    wget_opts="--progress=bar:force ${wget_opts}"
  fi

  if [ -z "$3" -o ! -f "$binary" ]; then
    # check if we already have the tarball
    # check if we have curl installed
    # download application
    [ ! -f "${local_tarball}" ] && [ $(command -v curl) ] && \
      echo "exec: curl ${curl_opts} ${remote_tarball}" 1>&2 && \
      curl ${curl_opts} "${remote_tarball}" > "${local_tarball}"
    # if the file still doesn't exist, lets try `wget` and cross our fingers
    [ ! -f "${local_tarball}" ] && [ $(command -v wget) ] && \
      echo "exec: wget ${wget_opts} ${remote_tarball}" 1>&2 && \
      wget ${wget_opts} -O "${local_tarball}" "${remote_tarball}"
    # if both were unsuccessful, exit
    [ ! -f "${local_tarball}" ] && \
      echo -n "ERROR: Cannot download $2 with cURL or wget; " && \
      echo "please install manually and try again." && \
      exit 2
........

需要用github的gitbush调用shell脚本

github客户端安装

鼠标右键会出现打开gitbush。。。

3.编译

在项目根目录下右键打开gitbush 输入命令

./build/mvn -DskipTests clean package

如果在有maven的报错可能是maven配置问题，但是shell脚本执行的操作已经完成

直接在根目录下maven打包就可以了

mvn install

编译时间很长