異常
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.LogManager
at org.apache.log4j.Logger.getLogger(Logger.java:105)
at org.apache.sqoop.util.LoggingUtils.setDebugLevel(LoggingUtils.java:69)
at org.apache.sqoop.tool.BaseSqoopTool.applyCommonOptions(BaseSqoopTool.java:633)
at org.apache.sqoop.tool.ImportTool.applyOptions(ImportTool.java:713)
at org.apache.sqoop.tool.SqoopTool.parseArguments(SqoopTool.java:433)
at org.apache.sqoop.Sqoop.run(Sqoop.java:129)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]
問題定位
單獨執行Sqoop腳本沒有問題,但通過oozie調用出現上述異常。
檢查classpath,都有log4j的jar包。
分析單獨執行和用oozie調用的腳本差別。
單獨執行時,有個關鍵參數 -Dhadoop.root.logger=INFO,console
用oozie調用度時,生成的腳本 -Dhadoop.root.logger=INFO,TLA
檢查log4j.properties,也有TLA的配置。
調試後發現是關鍵的參數-Dhadoop.tasklog.taskid缺少,這個與HADOOP_CLIENT_OPTS有關
問題解決
修改hadoop-env.sh
增加HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS"來解決。
參考
參考https://issues.apache.org/jira/browse/MAPREDUCE-3112