Spark YARN Cluster mode get this error
“Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster”
spark.yarn.jars
spark.yarn.archive
配置錯誤可能會導致此問題
/tmp/spark-80645695-04d9-4801-b119-bf526e903334/__spark_libs__5482901671928622840.zip -> hdfs:
正確配置:
spark.yarn.jar hdfs://udp02:8020/user//spark-lib/*.jar