Sqoop抽數報MySQL錯誤CommunicationsException

報錯信息如下:

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:342)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2197)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2230)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2025)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:778)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:300)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:213)
... 11 more
Caused by: java.net.UnknownHostException: rr-xx.mysql.rds.aliyuncs.com: Name or service not known
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:188)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:301)
... 27 more

19/08/26 00:19:21 INFO mapreduce.Job: Task Id : attempt_1566550561110_12899_m_000011_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

 

參考:

http://www.sunyaozong.com/mysql-communications-link-failure/

https://issues.apache.org/jira/browse/SQOOP-509

由於抽數處理時間比較久,導致數據庫連接時間過長,數據庫將連接關閉,而程序以爲還在連接,繼續處理,才發現鏈接已經關閉導致該問題發生。

 

解決方式:

1、sqoop併發不要設置的太大(split參數)

2、修改MySQL配置

在mysql的my.ini裏面加上這兩個參數

wait_timeout=2147483 

interactive_timeout=2147483 

3、JDBC的連接串加上如下配置即可

interactiveClient=true

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章