YARN編程實例—distributedshell源碼分析

1.    概述

本文介紹YARN自帶的一個非常簡單的應用程序編程實例---distributedshell，他可以看做YARN編程中的“hello world”，它的主要功能是並行執行用戶提供的shell命令或者shell腳本。本文主要介紹distributedshell 的實現方法。

Distributedshell的源代碼在文件夾

src\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell下。

2.    Distributedshell客戶端源碼分析

Distributedshell Client的入口main函數如下：public static void main(String[] args) {

public static void main(String[] args) {
    boolean result = false;
    try {
      ApplicationMaster appMaster = new ApplicationMaster();
      LOG.info("Initializing ApplicationMaster");
      boolean doRun = appMaster.init(args);
      if (!doRun) {
        System.exit(0);
      }
      result = appMaster.run();
    } catch (Throwable t) {
      LOG.fatal("Error running ApplicationMaster", t);
      System.exit(1);
    }
    if (result) {
      LOG.info("Application Master completed successfully. exiting");
      System.exit(0);
    } else {
      LOG.info("Application Master failed. exiting");
      System.exit(2);
    }
  }

DistributedShell Client中最重要的是函數爲run()，該函數實現過程如下：

（1）構造RPC句柄。

利用Hadoop RPC接口創建一個可以直接與ResourceManager交互的RPC client句柄applicationsManager：private void connectToASM() throws IOException {

YarnConfiguration yarnConf = new YarnConfiguration(conf);

InetSocketAddress rmAddress = yarnConf.getSocketAddr(

YarnConfiguration.RM_ADDRESS,

YarnConfiguration.DEFAULT_RM_ADDRESS,

YarnConfiguration.DEFAULT_RM_PORT);

LOG.info(“Connecting to ResourceManager at ” + rmAddress);

applicationsManager = ((ClientRMProtocol) rpc.getProxy(

ClientRMProtocol.class, rmAddress, conf));

{color:#000000}}
（2）獲取application id。

與ResourceManager通信，請求application id：GetNewApplicationRequest request = Records.newRecord(GetNewApplicationRequest.class);

GetNewApplicationResponse response = applicationsManager.getNewApplication(request);
（3）構造ContainerLaunchContext。

構造一個用於運行ApplicationMaster的container，container相關信息被封裝到ContainerLaunchContext對象中：ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);

//添加本地資源

//填充localResources

amContainer.setLocalResources(localResources);

//添加運行ApplicationMaster所需的環境變量

Map<String, String> env = new HashMap<String, String>();

//填充env

amContainer.setEnvironment(env);

//添加啓動ApplicationMaster的命令

//填充commands;

amContainer.setCommands(commands);

//設置ApplicationMaster所需的資源

amContainer.setResource(capability);
（4）構造ApplicationSubmissionContext。

構造一個用於提交ApplicationMaster的ApplicationSubmissionContext：ApplicationSubmissionContext appContext =

Records.newRecord(ApplicationSubmissionContext.class);

//設置application id，調用GetNewApplicationResponse#getApplicationId()

appContext.setApplicationId(appId);

//設置Application名稱：“DistributedShell”

appContext.setApplicationName(appName);

//設置前面創建的container

appContext.setAMContainerSpec(amContainer);

//設置application的優先級，默認是0

pri.setPriority(amPriority);

//設置application的所在隊列，默認是”"

appContext.setQueue(amQueue);

//設置application的所屬用戶，默認是”"

appContext.setUser(amUser);
（5）提交ApplicationMaster。

將ApplicationMaster提交到ResourceManager上，從而完成作業提交功能：applicationsManager.submitApplication(appRequest);
（6） 顯示應用程序運行狀態。

爲了讓用戶知道應用程序進度，Client會每隔幾秒在shell終端上打印一次應用程序運行狀態：while (true) {

Thread.sleep(1000);

GetApplicationReportRequest reportRequest =

Records.newRecord(GetApplicationReportRequest.class);

reportRequest.setApplicationId(appId);

GetApplicationReportResponse reportResponse =

applicationsManager.getApplicationReport(reportRequest);

ApplicationReport report = reportResponse.getApplicationReport();

//打印report內容

…

YarnApplicationState state = report.getYarnApplicationState();

FinalApplicationStatus dsStatus = report.getFinalApplicationStatus();

if (YarnApplicationState.FINISHED == state) {

if (FinalApplicationStatus.SUCCEEDED == dsStatus) {

return true;

{color:#000000}} else {{color}

return false;

{color:#000000}}

{color:#000000}} else if (YarnApplicationState.KILLED == state

|| YarnApplicationState.FAILED == state) {

return false;

{color:#000000}}

}

3.    Distributedshell ApplicationMaster源碼分析

Distributedshell ApplicationMaster的實現方法與“如何編寫YARN應用程序”所描述的步驟完全一致，它的過程如下：

步驟1 ApplicationMaster由ResourceManager分配的一個container啓用，之後，它與ResourceManager通信，註冊自己，以告知自己所在的節點（host：port），trackingurl（客戶端可通過該url直接查詢AM運行狀態）等。RegisterApplicationMasterRequest appMasterRequest =

Records.newRecord(RegisterApplicationMasterRequest.class);

appMasterRequest.setApplicationAttemptId(appAttemptID);

appMasterRequest.setHost(appMasterHostname);

appMasterRequest.setRpcPort(appMasterRpcPort);

appMasterRequest.setTrackingUrl(appMasterTrackingUrl);

return resourceManager.registerApplicationMaster(appMasterRequest);
步驟2 ApplicationMaster週期性向ResourceManager發送心跳信息，以告知ResourceManager自己仍然活着，這是通過週期性調用AMRMProtocol#allocate實現的。

步驟3 爲了完成計算任務，ApplicationMaster需向ResourceManage發送一個ResourceRequest描述對資源的需求，包括container個數、期望資源所在的節點、需要的CPU和內存等，而ResourceManager則爲ApplicationMaster返回一個AllocateResponse結構以告知新分配到的container列表、運行完成的container列表和當前可用的資源量等信息。while (numCompletedContainers.get() < numTotalContainers

&& !appDone) {

Thread.sleep(1000);

List<ResourceRequest> resourceReq = new ArrayList<ResourceRequest>();

if (askCount > 0) {

ResourceRequest containerAsk = setupContainerAskForRM(askCount);

resourceReq.add(containerAsk);

{color:#000000}}

//如果resourceReq爲null，則可看做心跳信息，否則就是申請資源

AMResponse amResp =sendContainerAskToRM(resourceReq);

{color:#000000}}
步驟4 對於每個新分配到的container，ApplicationMaster將創建一個ContainerLaunchContext對象，該對象包含container id，啓動container所需環境、啓動container命令，然後與對應的節點通信，以啓動container。LaunchContainerRunnable runnableLaunchContainer =

new LaunchContainerRunnable(allocatedContainer);

//每個container由一個線程啓動

Thread launchThread = new Thread(runnableLaunchContainer);

launchThreads.add(launchThread);

launchThread.start();
步驟5 ApplicationMaster通過AMRMProtocol#allocate獲取各個container的運行狀況，一旦發現某個container失敗了，則會重新向ResourceManager發送資源請求，以重新運行失敗的container。

步驟6 作業運行失敗後，ApplicationMaster向ResourceManager發送FinishApplicationMasterRequest請求，以告知自己運行結束。FinishApplicationMasterRequest finishReq =

Records.newRecord(FinishApplicationMasterRequest.class);

finishReq.setAppAttemptId(appAttemptID);

boolean isSuccess = true;

if (numFailedContainers.get() == 0) {

finishReq.setFinishApplicationStatus(FinalApplicationStatus.SUCCEEDED);