Eclipse Maven 非本地Spark

本地Spark跑不會出現什麼問題，但是切換到服務器上各種問題。

異常

java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.f$3 of type org.apache.spark.api.java.function.FlatMapFunction in instance of org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1

修改pom.xml文件在project中添加下方xml，然後右擊項目執行-> Run as -> maven claen && maven install

<build> 
  <plugins> 
    <plugin> 
      <groupId>org.apache.maven.plugins</groupId>  
      <artifactId>maven-compiler-plugin</artifactId>  
      <version>3.1</version>  
      <configuration> 
        <source>1.8</source>  
        <target>1.8</target> 
      </configuration> 
    </plugin>  
    <plugin> 
      <artifactId>maven-assembly-plugin</artifactId>  
      <configuration> 
        <descriptorRefs> 
          <descriptorRef>jar-with-dependencies</descriptorRef> 
        </descriptorRefs> 
      </configuration>  
      <executions> 
        <execution> 
          <id>make-assembly</id>  
          <phase>package</phase>  
          <goals> 
            <goal>single</goal> 
          </goals> 
        </execution> 
      </executions> 
    </plugin> 
  </plugins> 
</build>

添加jars路徑

String jarPath = "打包成功之後的項目jar文件"
JavaSparkContext jsc = new JavaSparkContext(new SparkConf()
				.setMaster("spark://服務器地址:端口")
				.setJars(new String[] {jarPath})
				.setAppName("Classification"));

然後在執行

END：