遇到這種問題首先考慮是不是jar包衝突 也就是依賴中存在多個版本的 netty導致程序出現NoSuchMethodError異常。
接下來使用命令mvn dependency:tree >> log/dependency.log用於分析現在程序中已有的jar包依賴，通過搜索日誌文件中發現：
org.apache.hbase:hbase-client:jar:1.4.1:compile中包含有io.netty:netty-all:jar:4.1.8.Final:compile jar包
org.apache.spark:spark-core_2.11:jar:2.3.0:compile中包含有io.netty:netty:jar:3.9.9.Final:compile
初步定位就是這兩個jar包導致的衝突。
我首先採用的方案是使用 <exclusions></exclusions>標籤剔除 spark-core 中所有的低版本的netty，無果。
後來發現在 stackoverflow 中有人發現類似的問題：https://stackoverflow.com/questions/49137397/spark-2-3-0-netty-version-issue-nosuchmethod-io-netty-buffer-pooledbytebufalloc?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
在這裏面主要強調了一遍maven的傳遞依賴問題，https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html。
並且評論中的解決方法是使用<dependencyManagement></dependencyManagement>對於項目中的多版本進行管理，添加依賴如下：
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.18.Final</version>
</dependency>
</dependencies>
</dependencyManagement>
完整依賴
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<properties>
<java.version>1.8</java.version>
<scala.version>2.11.12</scala.version>
<hbase.version>1.4.1</hbase.version>
<kafka.version>1.1.0</kafka.version>
<spark.version>2.3.0</spark.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.18.Final</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<!-- 引入hbase -->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>${hbase.version}</version>
</dependency>
<!-- 引入kafka -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>${kafka.version}</version>
</dependency>
<!-- 引入spark -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.11</artifactId>
<version>${spark.version}</version>
</dependency>
<!-- 引入日誌系統 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.21</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.21</version>
</dependency>
<!-- 簡化配置lombok -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.16.20</version>
</dependency>
<!-- 引入測試 -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
</plugins>
</build>
</project>