hive錯誤集錦

1.

Total jobs = 1

Stage-1 is selected by condition resolver.
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 999
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1441775634529_4716705, Tracking URL = http://sh02svr2882.hadoop.sh2.ctripcorp.com:8088/proxy/application_1441775634529_4716705/
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1441775634529_4716705
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2015-11-09 14:03:18,207 Stage-1 map = 0%,  reduce = 0%
Ended Job = job_1441775634529_4716705 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://sh02svr2882.hadoop.sh2.ctripcorp.com:8088/proxy/application_1441775634529_4716705/
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL

Total MapReduce CPU Time Spent: 0 msec


參數沒有設置,執行不了。

更改參考,設置參數:

SET hive.exec.dynamic.partition.mode=nonstrict;
SET hive.exec.parallel=true;
SET hive.exec.compress.intermediate=true;
SET mapred.map.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapred.job.reuse.jvm.num.tasks=10;
SET hive.auto.convert.join=true;
SET mapred.reduce.tasks=50;
SET hive.mapred.mode=nostrict;


2.

FAILED: SemanticException [Error 10041]: No partition predicate found for Alias "pageview" Table "pageview"

沒有分區,對於該表去查其分區字段,並在條件中引用分區


3.

mapreduce進程到reduce98%、99%->failed

數據傾斜

查詢主鍵出現null值,解決辦法:COALESCE(userid, '')<>'')



發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章