13.IPython Notebook 在Hadoop YARN-client模式運行驗證

前置準備Hadoop集羣搭建

1.啓動集羣

   start-all.sh

2.切換至ipynotebook工作目錄

   cd ~/pythonwork/ipynotebook

3.Hadoop YARN-client模式運行

   PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS="notebook" HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop  pyspark --master yarn  --deploy-mode client

  1.Master yarn-client is deprecated since 2.0. Please use master "yarn" with specified deploy mode instead