sqoop error xxx is greater than xxx

 sqoop導數失敗,異常日誌:

java.io.IOException: 0.00149538554284945409038360094760876373868438891233028512477697236907921983118594701374880207023210232364363037049770355224609375 is greater than 0.001281757140869648262434488801267003365970469023032400173426768338316728763538149640267838069718209226266480982303619384765625
20/03/27 03:30:34 ERROR tool.MyImportTool: Encountered IOException running import job: java.io.IOException: 0.00149538554284945409038360094760876373868438891233028512477697236907921983118594701374880207023210232364363037049770355224609375 is greater than 0.001281757140869648262434488801267003365970469023032400173426768338316728763538149640267838069718209226266480982303619384765625
    at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:204)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:305)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:322)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:200)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325)
    at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:203)
    at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:176)
    at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:273)
    at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:747)
    at org.apache.sqoop.tool.MyImportTool.importTable(MyImportTool.java:499)
    at org.apache.sqoop.tool.MyImportTool.run(MyImportTool.java:605)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: 0.00149538554284945409038360094760876373868438891233028512477697236907921983118594701374880207023210232364363037049770355224609375 is greater than 0.001281757140869648262434488801267003365970469023032400173426768338316728763538149640267838069718209226266480982303619384765625
    at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:157)
    at org.apache.sqoop.mapreduce.db.TextSplitter.split(TextSplitter.java:116)
    at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:201)
    ... 22 more

這個問題是sqoop使用了非數字類型的字段值進行split 造成的,找到sqoop報錯的這斷源碼很快就能定位到。

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章