問題詳情:
在使用Apache Livy中Batch模式提交Spark作業時,提交json如下:
{
"conf":{"spark.master":"local"},
"file":"/home/hadoop/Documents/spark/my_jars/SparkPlatform.jar",
"className":"com.application.SparkOperateKv"
}
其中file對應的是位於spark服務機器上的jar包。
這時候會出現如下異常
{"msg":"requirement failed: Local path /home/hadoop/Documents/spark/my_jars/SparkPlatform.jar cannot be added to user sessions."}
解決方法是在jar包路徑前添加 local:/
{
"conf":{"spark.master":"local"},
"file":"local:/home/hadoop/Documents/spark/my_jars/SparkPlatform.jar",
"className":"com.application.SparkOperateKv"
}