Hadoop源码之TaskTracker

TaskTracker是Map/Reduce中执行任务的服务。

1、有如下线程为其提供支撑:

1)提供一组RPC服务(TaskUmbilicalProtocol)的1个Listener线程与默认10个Handler线程;

2)提供另一组RPC服务(MapOutputProtocol)的1个Listener线程与默认2个Handler线程;

3)TaskTracker主线程:主线程运行offerService(),提供如下服务:

a)默认每隔10秒,向JobTracker发送1次心跳信息;

long waitTime = HEARTBEAT_INTERVAL - (now - lastHeartbeat);
if (waitTime > 0) {
try {
    Thread.sleep(waitTime);
} catch (InterruptedException ie) {
}
continue;
}
//代码略
int resultCode = jobClient.emitHeartbeat(new TaskTrackerStatus(taskTrackerName, localHostname, mapOutputPort, taskReports), justStarted);                                          //发送心跳

b)向JobTracker请求任务,并执行;

if (mapTotal < maxCurrentTasks || reduceTotal < maxCurrentTasks) {
Task t = jobClient.pollForNewTask(taskTrackerName);                        //请求任务
if (t != null) {
    TaskInProgress tip = new TaskInProgress(t, this.fConf);
    synchronized (this) {
      tasks.put(t.getTaskId(), tip);
      if (t.isMapTask()) {
	  mapTotal++;
      } else {
	  reduceTotal++;
      }
      runningTasks.put(t.getTaskId(), tip);
    }
    tip.launchTask();                                                      //执行任务
}
}

 c)杀死超时不汇报进度的任务;

for (Iterator it = runningTasks.values().iterator(); it.hasNext(); ) {
    TaskInProgress tip = (TaskInProgress) it.next();
    if ((tip.getRunState() == TaskStatus.RUNNING) &&
	(System.currentTimeMillis() - tip.getLastProgressReport() > this.taskTimeout)) {
	LOG.info("Task " + tip.getTask().getTaskId() + " timed out.  Killing.");
	tip.reportDiagnosticInfo("Timed out.");
	tip.killAndCleanup();                                         //杀死
    }
}

 d)向JobTracker询问可以终结的任务,并予以终结;

String toCloseId = jobClient.pollForTaskWithClosedJob(taskTrackerName);
if (toCloseId != null) {
synchronized (this) {
TaskInProgress tip = (TaskInProgress) tasks.get(toCloseId);
tip.jobHasFinished();
}
}


2、offerService中,请求到任务后,执行任务(tip.launchTask)的具体实现如下:

1)根据task是Map还是Reduce,构建MapTaskRunner或是ReduceTaskRunner线程;

2)不管是MapTaskRunner还是ReduceTaskRunner,都对抽象类TaskRunner进行了实现;

3)组装Java子进程命令行,入口类为TaskTracker的静态Child类,并执行,其中组装部件如下:

a)java执行文件;

b)classPath;

c)javaOpts,如堆内存大小;

d)TaskTracker.Child类名;

e)taskReport端口;

f)taskId;

 

3、执行任务的Java子进程实现如下:

1)获取TaskTracker的本地RPC代理;

  TaskUmbilicalProtocol umbilical =
    (TaskUmbilicalProtocol)RPC.getProxy(TaskUmbilicalProtocol.class,
					new InetSocketAddress(port), conf);

2)通过代理获取任务,并得到job信息;

Task task = umbilical.getTask(taskid);
JobConf job = new JobConf(task.getJobFile());

3)每隔1秒,发送ping信息到TaskTracker;

startPinging(umbilical, taskid);        // start pinging parent

4)通过实现抽象类Task的MapTask或ReduceTask执行任务;

task.run(job, umbilical);           // run the task


4、MapTask执行任务的具体实现如下:

1)根据Reduce任务的个数,将Map阶段的输出文件分为几块,并打开;

    // open output files
    final int partitions = job.getNumReduceTasks();
    final SequenceFile.Writer[] outs = new SequenceFile.Writer[partitions];

2)根据Map阶段要输出文件的格式,Key、Value的类型,对Writer进行初始化;

      for (int i = 0; i < partitions; i++) {
        outs[i] =
          new SequenceFile.Writer(FileSystem.getNamed("local", job),
                                  this.mapOutputFile.getOutputFile(getTaskId(), i).toString(),
                                  job.getOutputKeyClass(),
                                  job.getOutputValueClass());
      }

3)构建输出集合器,实现collect方法;

      OutputCollector partCollector = new OutputCollector() { // make collector
          public synchronized void collect(WritableComparable key,
                                           Writable value)
            throws IOException {
            outs[partitioner.getPartition(key, value, partitions)]
              .append(key, value);
            reportProgress(umbilical);
          }
        };

      OutputCollector collector = partCollector;

4)如果输入到Reduce前,需要组合归类的话,构建组合集合器;

      boolean combining = job.getCombinerClass() != null;
      if (combining) {                            // add combining collector
        collector = new CombiningCollector(job, partCollector, reporter);
      }

5)打开Map的输入文件;

      final RecordReader rawIn =                  // open input
        job.getInputFormat().getRecordReader
        (FileSystem.get(job), split, job, reporter);

6)构建读入器,并实现每读入一对Key、Value,能够更新进度;

      RecordReader in = new RecordReader() {      // wrap in progress reporter
          private float perByte = 1.0f /(float)split.getLength();

          public synchronized boolean next(Writable key, Writable value)
            throws IOException {

            float progress =                        // compute progress
              (float)Math.min((rawIn.getPos()-split.getStart())*perByte, 1.0f);
            reportProgress(umbilical, progress);

            return rawIn.next(key, value);
          }
          public long getPos() throws IOException { return rawIn.getPos(); }
          public void close() throws IOException { rawIn.close(); }
        };

7)构建MapRunner;

      MapRunnable runner =
        (MapRunnable)job.newInstance(job.getMapRunnerClass());

8)调用MapRunner的run方法;

runner.run(in, collector, reporter);      // run the map

9)run方法内,先获取Key、Value,再循环调用用户定义的Mapper运行,通过集合器收集map的输出结果,直到所有数据对执行完毕。

  public void run(RecordReader input, OutputCollector output,
                  Reporter reporter)
    throws IOException {
    try {
      // allocate key & value instances that are re-used for all entries
      WritableComparable key =
        (WritableComparable)job.newInstance(inputKeyClass);
      Writable value = (Writable)job.newInstance(inputValueClass);
      while (input.next(key, value)) {
        // map pair to output
        mapper.map(key, value, output, reporter);
      }
    } finally {
        mapper.close();
    }
  }

 

5、ReduceTask执行任务的具体实现如下:

1)准备收集Map阶段的所有输出文件;

    // open a file to collect map output
    String file = job.getLocalFile(getTaskId(), "all.1").toString();
    SequenceFile.Writer writer =
      new SequenceFile.Writer(lfs, file, keyClass, valueClass);

2)所有Map阶段的输出文件,合并汇总成一个文件,同时报告合并进度;

      for (int i = 0; i < mapTaskIds.length; i++) {
        appendPhase.addPhase();                 // one per file
      }
      
      DataOutputBuffer buffer = new DataOutputBuffer();

      for (int i = 0; i < mapTaskIds.length; i++) {
        File partFile =
          this.mapOutputFile.getInputFile(mapTaskIds[i], getTaskId());
        float progPerByte = 1.0f / lfs.getLength(partFile);
        Progress phase = appendPhase.phase();
        phase.setStatus(partFile.toString());

        SequenceFile.Reader in =
          new SequenceFile.Reader(lfs, partFile.toString(), job);
        try {
          int keyLen;
          while((keyLen = in.next(buffer)) > 0) {
            writer.append(buffer.getData(), 0, buffer.getLength(), keyLen);
            phase.set(in.getPosition()*progPerByte);
            reportProgress(umbilical);
            buffer.reset();
          }
        } finally {
          in.close();
        }
        phase.complete();
      }
      
    } finally {
      writer.close();
    }

3)构建一个排序用的线程;

    Thread sortProgress = new Thread() {
        public void run() {
          while (!sortComplete) {
            try {
              reportProgress(umbilical);
              Thread.sleep(PROGRESS_INTERVAL);
            } catch (InterruptedException e) {
              continue;
            } catch (Throwable e) {
              return;
            }
          }
        }
      };

4)排序生成新文件,删除掉排序前文件,并报告进度;

    String sortedFile = job.getLocalFile(getTaskId(), "all.2").toString();

    WritableComparator comparator = job.getOutputKeyComparator();
    
    try {
      sortProgress.start();

      // sort the input file
      SequenceFile.Sorter sorter =
        new SequenceFile.Sorter(lfs, comparator, valueClass, job);
      sorter.sort(file, sortedFile);              // sort
      lfs.delete(new File(file));                 // remove unsorted

    } finally {
      sortComplete = true;
    }

5)构建Reduce的输出集合器;

    // make output collector
    String name = getOutputName(getPartition());
    final RecordWriter out =
      job.getOutputFormat().getRecordWriter(FileSystem.get(job), job, name);
    OutputCollector collector = new OutputCollector() {
        public void collect(WritableComparable key, Writable value)
          throws IOException {
          out.write(key, value);
          reportProgress(umbilical);
        }
      };

6)构建读入器,循环调用用户定义的Reducer运行,通过集合器收集reduce的结果,直到所有数据对都执行完毕。

    // apply reduce function
    SequenceFile.Reader in = new SequenceFile.Reader(lfs, sortedFile, job);
    Reporter reporter = getReporter(umbilical, getProgress());
    long length = lfs.getLength(new File(sortedFile));
    try {
      ValuesIterator values = new ValuesIterator(in, length, comparator,
                                                 umbilical);
      while (values.more()) {
        reducer.reduce(values.getKey(), values, collector, reporter);
        values.nextKey();
      }

 

这就是TaskTracker一个完整的结构与运行流程。

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章