TaskTracker是Map/Reduce中执行任务的服务。
1、有如下线程为其提供支撑:
1)提供一组RPC服务(TaskUmbilicalProtocol)的1个Listener线程与默认10个Handler线程;
2)提供另一组RPC服务(MapOutputProtocol)的1个Listener线程与默认2个Handler线程;
3)TaskTracker主线程:主线程运行offerService(),提供如下服务:
a)默认每隔10秒,向JobTracker发送1次心跳信息;
long waitTime = HEARTBEAT_INTERVAL - (now - lastHeartbeat);
if (waitTime > 0) {
try {
Thread.sleep(waitTime);
} catch (InterruptedException ie) {
}
continue;
}
//代码略
int resultCode = jobClient.emitHeartbeat(new TaskTrackerStatus(taskTrackerName, localHostname, mapOutputPort, taskReports), justStarted); //发送心跳
b)向JobTracker请求任务,并执行;
if (mapTotal < maxCurrentTasks || reduceTotal < maxCurrentTasks) {
Task t = jobClient.pollForNewTask(taskTrackerName); //请求任务
if (t != null) {
TaskInProgress tip = new TaskInProgress(t, this.fConf);
synchronized (this) {
tasks.put(t.getTaskId(), tip);
if (t.isMapTask()) {
mapTotal++;
} else {
reduceTotal++;
}
runningTasks.put(t.getTaskId(), tip);
}
tip.launchTask(); //执行任务
}
}
c)杀死超时不汇报进度的任务;
for (Iterator it = runningTasks.values().iterator(); it.hasNext(); ) {
TaskInProgress tip = (TaskInProgress) it.next();
if ((tip.getRunState() == TaskStatus.RUNNING) &&
(System.currentTimeMillis() - tip.getLastProgressReport() > this.taskTimeout)) {
LOG.info("Task " + tip.getTask().getTaskId() + " timed out. Killing.");
tip.reportDiagnosticInfo("Timed out.");
tip.killAndCleanup(); //杀死
}
}
d)向JobTracker询问可以终结的任务,并予以终结;
String toCloseId = jobClient.pollForTaskWithClosedJob(taskTrackerName);
if (toCloseId != null) {
synchronized (this) {
TaskInProgress tip = (TaskInProgress) tasks.get(toCloseId);
tip.jobHasFinished();
}
}
2、offerService中,请求到任务后,执行任务(tip.launchTask)的具体实现如下:
1)根据task是Map还是Reduce,构建MapTaskRunner或是ReduceTaskRunner线程;
2)不管是MapTaskRunner还是ReduceTaskRunner,都对抽象类TaskRunner进行了实现;
3)组装Java子进程命令行,入口类为TaskTracker的静态Child类,并执行,其中组装部件如下:
a)java执行文件;
b)classPath;
c)javaOpts,如堆内存大小;
d)TaskTracker.Child类名;
e)taskReport端口;
f)taskId;
3、执行任务的Java子进程实现如下:
1)获取TaskTracker的本地RPC代理;
TaskUmbilicalProtocol umbilical =
(TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
new InetSocketAddress(port), conf);
2)通过代理获取任务,并得到job信息;
Task task = umbilical.getTask(taskid);
JobConf job = new JobConf(task.getJobFile());
3)每隔1秒,发送ping信息到TaskTracker;
startPinging(umbilical, taskid); // start pinging parent
4)通过实现抽象类Task的MapTask或ReduceTask执行任务;
task.run(job, umbilical); // run the task
4、MapTask执行任务的具体实现如下:
1)根据Reduce任务的个数,将Map阶段的输出文件分为几块,并打开;
// open output files
final int partitions = job.getNumReduceTasks();
final SequenceFile.Writer[] outs = new SequenceFile.Writer[partitions];
2)根据Map阶段要输出文件的格式,Key、Value的类型,对Writer进行初始化;
for (int i = 0; i < partitions; i++) {
outs[i] =
new SequenceFile.Writer(FileSystem.getNamed("local", job),
this.mapOutputFile.getOutputFile(getTaskId(), i).toString(),
job.getOutputKeyClass(),
job.getOutputValueClass());
}
3)构建输出集合器,实现collect方法;
OutputCollector partCollector = new OutputCollector() { // make collector
public synchronized void collect(WritableComparable key,
Writable value)
throws IOException {
outs[partitioner.getPartition(key, value, partitions)]
.append(key, value);
reportProgress(umbilical);
}
};
OutputCollector collector = partCollector;
4)如果输入到Reduce前,需要组合归类的话,构建组合集合器;
boolean combining = job.getCombinerClass() != null;
if (combining) { // add combining collector
collector = new CombiningCollector(job, partCollector, reporter);
}
5)打开Map的输入文件;
final RecordReader rawIn = // open input
job.getInputFormat().getRecordReader
(FileSystem.get(job), split, job, reporter);
6)构建读入器,并实现每读入一对Key、Value,能够更新进度;
RecordReader in = new RecordReader() { // wrap in progress reporter
private float perByte = 1.0f /(float)split.getLength();
public synchronized boolean next(Writable key, Writable value)
throws IOException {
float progress = // compute progress
(float)Math.min((rawIn.getPos()-split.getStart())*perByte, 1.0f);
reportProgress(umbilical, progress);
return rawIn.next(key, value);
}
public long getPos() throws IOException { return rawIn.getPos(); }
public void close() throws IOException { rawIn.close(); }
};
7)构建MapRunner;
MapRunnable runner =
(MapRunnable)job.newInstance(job.getMapRunnerClass());
8)调用MapRunner的run方法;
runner.run(in, collector, reporter); // run the map
9)run方法内,先获取Key、Value,再循环调用用户定义的Mapper运行,通过集合器收集map的输出结果,直到所有数据对执行完毕。
public void run(RecordReader input, OutputCollector output,
Reporter reporter)
throws IOException {
try {
// allocate key & value instances that are re-used for all entries
WritableComparable key =
(WritableComparable)job.newInstance(inputKeyClass);
Writable value = (Writable)job.newInstance(inputValueClass);
while (input.next(key, value)) {
// map pair to output
mapper.map(key, value, output, reporter);
}
} finally {
mapper.close();
}
}
5、ReduceTask执行任务的具体实现如下:
1)准备收集Map阶段的所有输出文件;
// open a file to collect map output
String file = job.getLocalFile(getTaskId(), "all.1").toString();
SequenceFile.Writer writer =
new SequenceFile.Writer(lfs, file, keyClass, valueClass);
2)所有Map阶段的输出文件,合并汇总成一个文件,同时报告合并进度;
for (int i = 0; i < mapTaskIds.length; i++) {
appendPhase.addPhase(); // one per file
}
DataOutputBuffer buffer = new DataOutputBuffer();
for (int i = 0; i < mapTaskIds.length; i++) {
File partFile =
this.mapOutputFile.getInputFile(mapTaskIds[i], getTaskId());
float progPerByte = 1.0f / lfs.getLength(partFile);
Progress phase = appendPhase.phase();
phase.setStatus(partFile.toString());
SequenceFile.Reader in =
new SequenceFile.Reader(lfs, partFile.toString(), job);
try {
int keyLen;
while((keyLen = in.next(buffer)) > 0) {
writer.append(buffer.getData(), 0, buffer.getLength(), keyLen);
phase.set(in.getPosition()*progPerByte);
reportProgress(umbilical);
buffer.reset();
}
} finally {
in.close();
}
phase.complete();
}
} finally {
writer.close();
}
3)构建一个排序用的线程;
Thread sortProgress = new Thread() {
public void run() {
while (!sortComplete) {
try {
reportProgress(umbilical);
Thread.sleep(PROGRESS_INTERVAL);
} catch (InterruptedException e) {
continue;
} catch (Throwable e) {
return;
}
}
}
};
4)排序生成新文件,删除掉排序前文件,并报告进度;
String sortedFile = job.getLocalFile(getTaskId(), "all.2").toString();
WritableComparator comparator = job.getOutputKeyComparator();
try {
sortProgress.start();
// sort the input file
SequenceFile.Sorter sorter =
new SequenceFile.Sorter(lfs, comparator, valueClass, job);
sorter.sort(file, sortedFile); // sort
lfs.delete(new File(file)); // remove unsorted
} finally {
sortComplete = true;
}
5)构建Reduce的输出集合器;
// make output collector
String name = getOutputName(getPartition());
final RecordWriter out =
job.getOutputFormat().getRecordWriter(FileSystem.get(job), job, name);
OutputCollector collector = new OutputCollector() {
public void collect(WritableComparable key, Writable value)
throws IOException {
out.write(key, value);
reportProgress(umbilical);
}
};
6)构建读入器,循环调用用户定义的Reducer运行,通过集合器收集reduce的结果,直到所有数据对都执行完毕。
// apply reduce function
SequenceFile.Reader in = new SequenceFile.Reader(lfs, sortedFile, job);
Reporter reporter = getReporter(umbilical, getProgress());
long length = lfs.getLength(new File(sortedFile));
try {
ValuesIterator values = new ValuesIterator(in, length, comparator,
umbilical);
while (values.more()) {
reducer.reduce(values.getKey(), values, collector, reporter);
values.nextKey();
}
这就是TaskTracker一个完整的结构与运行流程。