//1.3 指定分區類
job.setPartitionerClass(KpiPartitioner.class);
job.setNumReduceTasks(3);
static class KpiPartitioner extends HashPartitioner<Text, KpiWritable>{
@Override
public int getPartition(Text key, KpiWritable value, int numReduceTasks) {
if(key.toString().length()==11){
return 0;
}else if(key.toString().length()==10){
return 1;
}else{
return 2;
}
}
}
分區的例子必須打成jar運行
job.setJarByClass(KpiApp.class);
2.多個reduce任務在運行,提高整體job的運行效率
2. HashPartitioner是mapreduce的默認partitioner。計算方法是
which reducer=(key.hashCode() &Integer.MAX_VALUE) % numReduceTasks,得到當前的目的reducer。