#0. 先看一下測試demo的運行流程

a.創建一個YARN客戶端YarnClient，

並與ResourceManager建立連接

b.通過YARN客戶端創建一個應用並獲取到應用提交上下文對象、設置相關的屬性

特別是需要設置setAMContainerSpec，

防止後面getTokensConf時報空指針異常；

還需要設置setUnmanagedAM，不需要RM來管理AM（分配並啓動Container），置爲true之後就不再需要設置請求的資源大小了（UAM主要用來做測試）。

c.通過YARN客戶端提交應用

d.獲取並監控應用的狀態ApplicationReport

e.判斷應用狀態，如果爲ACCEPTED（已接受），則繼續監控應用嘗試的啓動狀態，並獲取到應用嘗試ID

f.跟據應用嘗試ID獲取到通信令牌Token

g.拿着Token去註冊ApplicationMaster

h.最後輸出一下響應信息

RegisterApplicationMasterResponse

測試代碼片段：


@Test
public void testAMRMClientAsync() throws Exception {
    YarnConfiguration yarnConfig = new YarnConfiguration();
    YarnClient yarnClient = YarnClient.createYarnClient();
    yarnClient.init(yarnConfig);
    // a.創建一個YARN客戶端YarnClient，並與ResourceManager建立連接
    yarnClient.start();

    // 應用提交上下文
    ApplicationSubmissionContext appContext = yarnClient.createApplication().getApplicationSubmissionContext();
    ApplicationId appId = appContext.getApplicationId();
    // 設置應用名稱
    appContext.setApplicationName("zlxiaoxiang_test");
    // 爲ApplicationMaster設置優先級
    Priority pri = Records.newRecord(Priority.class);
    pri.setPriority(1);
    appContext.setPriority(pri);
    // 設置調度隊列名稱 默認default
    appContext.setQueue("default");

    // 爲AM設置Container啓動的上下文
    ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);
    // 防止tokenConf報空指針異常 ByteBuffer tokenConf = submissionContext.getAMContainerSpec().getTokensConf();
    appContext.setAMContainerSpec(amContainer);
    // 不由RM管理，此處設置爲true；RM不會爲AM分配Container並啓動它；
    appContext.setUnmanagedAM(true);
    // 設置要請求的資源大小
//        appContext.setResource(Resource.newInstance(512, 1));

    // c.通過YARN客戶端提交應用
    yarnClient.submitApplication(appContext);
    // d.獲取並監控應用的狀態ApplicationReport
    ApplicationReport appReport = monitorApplication(yarnClient, appId, EnumSet.of(YarnApplicationState.ACCEPTED,
            YarnApplicationState.KILLED, YarnApplicationState.FAILED,
            YarnApplicationState.FINISHED));

    // e.判斷應用狀態，如果爲ACCEPTED（已接受），則繼續監控應用嘗試的啓動狀態，並獲取到應用嘗試ID
    if (appReport.getYarnApplicationState() == YarnApplicationState.ACCEPTED) {
        // 監控應用狀態
        ApplicationAttemptReport attemptReport = monitorCurrentAppAttempt(yarnClient, appId, YarnApplicationAttemptState.LAUNCHED);
        // 應用嘗試ID、用於啓動ApplicationMaster
        ApplicationAttemptId attemptId = attemptReport.getApplicationAttemptId();

        // f.跟據應用嘗試ID獲取到通信令牌Token
        Token<AMRMTokenIdentifier> token = yarnClient.getAMRMToken(attemptId.getApplicationId());

        UserGroupInformation userGroupInformation = UserGroupInformation.getCurrentUser();

        userGroupInformation.addToken(token);

        // g.拿着Token去註冊AM，否則跳轉不到RM服務端
        userGroupInformation.doAs((PrivilegedExceptionAction<Void>) () -> {
            // 創建一個AM-RM異步客戶端
            AMRMClientAsync<AMRMClient.ContainerRequest> resourceManagerClient = AMRMClientAsync.createAMRMClientAsync(
                    5000, new TestCallbackHandler());

            resourceManagerClient.init(yarnConfig);
            // 與RM建立連接
            resourceManagerClient.start();

            // 調用registerApplicationMaster
            final RegisterApplicationMasterResponse registerApplicationMasterResponse =
                    resourceManagerClient.registerApplicationMaster("localhost", 10088, "https://blog.icocoro.me/");

            LOG.info("registerApplicationMasterResponse: " + registerApplicationMasterResponse);

            return null;
        });

    }

    yarnClient.stop();

}

整個DEBUG過程：

a.在idea中DEBUG模式啓動ResourceManager，

在重點關注的位置打上斷點，也可以邊調試邊打斷點

b.在idea中DEBUG模式運行testAMRMClientAsync，

從主要關注的位置開始一步一步往下走

#1. DEBUG模式運行ResourceManager

#2. 瀏覽器訪問默認的8088端口

#3. 調試開始，跳到提交應用的斷點位置

#4. 會調用到RM端的ClientRMService，此處是可能發生NPE的位置

客戶端設置了相關屬性即可避免

#5. 調用到RM端的RMAppManager，進行應用註冊

#6. 應用註冊結束，返回到測試客戶端

判斷應用當前的狀態

#7. 觀察YARN的WEB界面

已經有一個應用了，狀態是ACCEPTED

#8. 需要取得通信Token

可以看到Token的屬性：


private byte[] identifier;
private byte[] password;
private Text kind;
private Text service;
private TokenRenewer renewer;

#9. 準備調用AMRMClientAsync的

registerApplicationMaster方法

#10. 進入AMRMClientAsyncImpl

#11. 進入AMRMClientImpl

這裏用到rmClient

是一個ApplicationMasterProtocolPBClientImpl對象

#12. 進入

ApplicationMasterProtocolPBClientImpl

這裏用到proxy

是一個ApplicationMasterProtocolPBServiceImpl對象

#13. 進入

ApplicationMasterProtocolPBServiceImpl

這裏用到real

是一個ApplicationMasterService對象

#14. 進入ApplicationMasterService

這裏用到amsProcessingChain

是一個AMSProcessingChain對象

實現了ApplicationMasterServiceProcessor接口

public interface ApplicationMasterServiceProcessor {

  /**
   * Initialize with and ApplicationMasterService Context as well as the
   * next processor in the chain.
   * @param amsContext AMSContext.
   * @param nextProcessor next ApplicationMasterServiceProcessor
   */
  void init(ApplicationMasterServiceContext amsContext,
      ApplicationMasterServiceProcessor nextProcessor);

  /**
   * Register AM attempt.
   * @param applicationAttemptId applicationAttemptId.
   * @param request Register Request.
   * @param response Register Response.
   * @throws IOException IOException.
   * @throws YarnException in critical situation where invalid
   *         profiles/resources are added.
   */
  void registerApplicationMaster(ApplicationAttemptId applicationAttemptId,
      RegisterApplicationMasterRequest request,
      RegisterApplicationMasterResponse response)
      throws IOException, YarnException;

  /**
   * Allocate call.
   * @param appAttemptId appAttemptId.
   * @param request Allocate Request.
   * @param response Allocate Response.
   * @throws YarnException YarnException.
   */
  void allocate(ApplicationAttemptId appAttemptId,
      AllocateRequest request, AllocateResponse response) throws YarnException;

  /**
   * Finish AM.
   * @param applicationAttemptId applicationAttemptId.
   * @param request Finish AM Request.
   * @param response Finish AM Response.
   */
  void finishApplicationMaster(
      ApplicationAttemptId applicationAttemptId,
      FinishApplicationMasterRequest request,
      FinishApplicationMasterResponse response);
}

#15. 進入AMSProcessingChain

這裏的head是個DisabledPlacementProcessor

當然也是ApplicationMasterServiceProcessor

#16. 最終會進入DefaultAMSProcessor

DefaultAMSProcessor是AMSProcessingChain裏最後一個processor

這裏面會有一塊核心代碼：

getRmContext().getDispatcher().getEventHandler()
        .handle(
            new RMAppAttemptRegistrationEvent(applicationAttemptId, request
                .getHost(), request.getRpcPort(), request.getTrackingUrl()));

#17. 返回到AMRMClientAsyncImpl的註冊方法裏面

#18. 查看YARN的WEB UI

顯示應用的狀態爲RUNNING。

Running Containers爲0，也就是並沒有分配實際的Container在運行。

返回的註冊響應信息：

INFO  test.MyTestAMRMClientAsync (MyTestAMRMClientAsync.java:lambda$testAMRMClientAsync$0(97)) - registerApplicationMasterResponse: maximumCapability { memory: 8192 virtual_cores: 4 resource_value_map { key: "memory-mb" value: 8192 units: "Mi" type: COUNTABLE } resource_value_map { key: "vcores" value: 4 units: "" type: COUNTABLE } } queue: "default" scheduler_resource_types: MEMORY resource_profiles { } resource_types { name: "memory-mb" units: "Mi" type: COUNTABLE } resource_types { name: "vcores" units: "" type: COUNTABLE }
Disconnected from the target VM, address: '127.0.0.1:55513', transport: 'socket'

以上就是AM註冊自己的整體宏觀過程。

【END】

往期推薦：