二十一、Flink源碼閱讀--ExecutionGraph生成過程

在上一篇服務端處理jobGraph提到，jobGraph是轉換爲executionGraph，最後被執行調度的，那麼本篇我們看下這一過程的源碼

源碼分析

從JobMaster的啓動開始講起
JobMaster.start—>JobMaster.startJobExecution—>resetAndScheduleExecutionGraph方法將JobGraph轉換爲 ExecutionGraph

private void resetAndScheduleExecutionGraph() throws Exception {
	validateRunsInMainThread();

	final CompletableFuture<Void> executionGraphAssignedFuture;

	if (executionGraph.getState() == JobStatus.CREATED) {
		executionGraphAssignedFuture = CompletableFuture.completedFuture(null);
	} else {
		suspendAndClearExecutionGraphFields(new FlinkException("ExecutionGraph is being reset in order to be rescheduled."));
		final JobManagerJobMetricGroup newJobManagerJobMetricGroup = jobMetricGroupFactory.create(jobGraph);
		final ExecutionGraph newExecutionGraph = createAndRestoreExecutionGraph(newJobManagerJobMetricGroup);//生成executionGraph

		executionGraphAssignedFuture = executionGraph.getTerminationFuture().handleAsync(
			(JobStatus ignored, Throwable throwable) -> {
				assignExecutionGraph(newExecutionGraph, newJobManagerJobMetricGroup);
				return null;
			},
			getMainThreadExecutor());
	}

	executionGraphAssignedFuture.thenRun(this::scheduleExecutionGraph);//執行executionGraph
}

詳細步驟進入createAndRestoreExecutionGraph方法

private ExecutionGraph createAndRestoreExecutionGraph(JobManagerJobMetricGroup currentJobManagerJobMetricGroup) throws Exception {
	ExecutionGraph newExecutionGraph = createExecutionGraph(currentJobManagerJobMetricGroup);
	...
	return newExecutionGraph;
}
-----> 
private ExecutionGraph createExecutionGraph(JobManagerJobMetricGroup currentJobManagerJobMetricGroup) throws JobExecutionException, JobException {
	return ExecutionGraphBuilder.buildGraph(
		null,
		jobGraph,
		jobMasterConfiguration.getConfiguration(),
		scheduledExecutorService,
		scheduledExecutorService,
		slotPool.getSlotProvider(),
		userCodeLoader,
		highAvailabilityServices.getCheckpointRecoveryFactory(),
		rpcTimeout,
		restartStrategy,
		currentJobManagerJobMetricGroup,
		blobServer,
		jobMasterConfiguration.getSlotRequestTimeout(),
		log);
}
-----> 
public static ExecutionGraph buildGraph(
		@Nullable ExecutionGraph prior,
		JobGraph jobGraph,
		Configuration jobManagerConfig,
		ScheduledExecutorService futureExecutor,
		Executor ioExecutor,
		SlotProvider slotProvider,
		ClassLoader classLoader,
		CheckpointRecoveryFactory recoveryFactory,
		Time rpcTimeout,
		RestartStrategy restartStrategy,
		MetricGroup metrics,
		int parallelismForAutoMax,
		BlobWriter blobWriter,
		Time allocationTimeout,
		Logger log)
	throws JobExecutionException, JobException {

	...

	for (JobVertex vertex : jobGraph.getVertices()) {
		String executableClass = vertex.getInvokableClassName();
		if (executableClass == null || executableClass.isEmpty()) {//檢查operator的class
			throw new JobSubmissionException(jobId,
					"The vertex " + vertex.getID() + " (" + vertex.getName() + ") has no invokable class.");
		}

		if (vertex.getParallelism() == ExecutionConfig.PARALLELISM_AUTO_MAX) {
			if (parallelismForAutoMax < 0) {
				throw new JobSubmissionException(
					jobId,
					PARALLELISM_AUTO_MAX_ERROR_MESSAGE);
			}
			else {
				vertex.setParallelism(parallelismForAutoMax);//設置最大併發
			}
		}

		try {
			vertex.initializeOnMaster(classLoader);
		}
		catch (Throwable t) {
				throw new JobExecutionException(jobId,
						"Cannot initialize task '" + vertex.getName() + "': " + t.getMessage(), t);
		}
	}

...

	executionGraph.attachJobGraph(sortedTopology);//核心代碼，根據JobVertex列表生成executionGraph

...

	//設置checkpoint相關

    //設置metric相關

	return executionGraph;
}

跟進到attachJobGraph方法，遍歷JobVertex列表,每個jobVertex生成對應的ExecutionJobVertex

ExecutionJobVertex ejv = new ExecutionJobVertex(//jobVertex生成ExecutionJobVertex
	this,
	jobVertex,
	1,
	rpcTimeout,
	globalModVersion,
	createTimestamp);

ejv.connectToPredecessors(this.intermediateResults);//將ExecutionJobVertex和IntermediateResult連接起來

核心方法就是ExecutionJobVertex的構造函數

// create the intermediate results
this.producedDataSets = new IntermediateResult[jobVertex.getNumberOfProducedIntermediateDataSets()];//根據
IntermediateDataSet個數創建IntermediateDataReult數組，數組大小一致

for (int i = 0; i < jobVertex.getProducedDataSets().size(); i++) {//將IntermediateDataSet 轉爲 IntermediateDataResult
	final IntermediateDataSet result = jobVertex.getProducedDataSets().get(i);

	this.producedDataSets[i] = new IntermediateResult(
			result.getId(),
			this,
			numTaskVertices,
			result.getResultType());
}

// create all task vertices，將每個ExecutionJobVertex 轉爲 ExecutionVertex
for (int i = 0; i < numTaskVertices; i++) {
	ExecutionVertex vertex = new ExecutionVertex(
			this,
			i,
			producedDataSets,
			timeout,
			initialGlobalModVersion,
			createTimestamp,
			maxPriorAttemptsHistoryLength);

	this.taskVertices[i] = vertex;
}

我們在看一看ExecutionVertex的構造函數

for (IntermediateResult result : producedDataSets) {//將IntermediateResult轉爲IntermediateResultPartition中，放到map集合
	IntermediateResultPartition irp = new IntermediateResultPartition(result, this, subTaskIndex);
	result.setPartition(subTaskIndex, irp);

	resultPartitions.put(irp.getPartitionId(), irp);
}

this.inputEdges = new ExecutionEdge[jobVertex.getJobVertex().getInputs().size()][];//二維數組

this.priorExecutions = new EvictingBoundedList<>(maxPriorExecutionHistoryLength);

this.currentExecution = new Execution(//創建物理Execution，即具體的物理執行
	getExecutionGraph().getFutureExecutor(),
	this,
	0,
	initialGlobalModVersion,
	createTimestamp,
	timeout);

具體的將ExecutionJobVertex和IntermediateResult連接起來的方法connectToPredecessors

public void connectToPredecessors(Map<IntermediateDataSetID, IntermediateResult> intermediateDataSets) throws JobException {

	List<JobEdge> inputs = jobVertex.getInputs();

	if (LOG.isDebugEnabled()) {
		LOG.debug(String.format("Connecting ExecutionJobVertex %s (%s) to %d predecessors.", jobVertex.getID(), jobVertex.getName(), inputs.size()));
	}

	for (int num = 0; num < inputs.size(); num++) {
		JobEdge edge = inputs.get(num);

		if (LOG.isDebugEnabled()) {
			if (edge.getSource() == null) {
				LOG.debug(String.format("Connecting input %d of vertex %s (%s) to intermediate result referenced via ID %s.",
						num, jobVertex.getID(), jobVertex.getName(), edge.getSourceId()));
			} else {
				LOG.debug(String.format("Connecting input %d of vertex %s (%s) to intermediate result referenced via predecessor %s (%s).",
						num, jobVertex.getID(), jobVertex.getName(), edge.getSource().getProducer().getID(), edge.getSource().getProducer().getName()));
			}
		}

		// fetch the intermediate result via ID. if it does not exist, then it either has not been created, or the order
		// in which this method is called for the job vertices is not a topological order
		IntermediateResult ires = intermediateDataSets.get(edge.getSourceId());
		if (ires == null) {
			throw new JobException("Cannot connect this job graph to the previous graph. No previous intermediate result found for ID "
					+ edge.getSourceId());
		}

		this.inputs.add(ires);

		int consumerIndex = ires.registerConsumer();

		for (int i = 0; i < parallelism; i++) {
			ExecutionVertex ev = taskVertices[i];
			ev.connectSource(num, ires, edge, consumerIndex);// 將ExecutionVertex與IntermediateResult關聯起來
		}
	}
}

在進入 connectAllToAll 方法 ，其實就是ExecutionEdge，IntermediateResultPartition關聯。

總結：

• 在 ExecutionGraph 中，節點對應的類是 ExecutionJobVertex，對應的就是 JobGraph 中的 JobVertex。每一個 ExexutionJobVertex 都是由一個 JobVertex 生成的
• 一個ExecutionJobVertex在底層等於並行度個ExecutionVertex
• 一個IntermediateDataset相當於一個IntermediateResult
• 一個ExecutionVertex相當於最後的一個Execution，Execution 是對 ExecutionVertex 的一次執行，通過 ExecutionAttemptId 來唯一標識
• 一個IntermediateResult分爲多個IntermediateResultPartition

JobGraph : JobVertex + IntermediateDateSet
IntermediateDataSet 的 producer是JobVertex，consumers是List
JobEdge的target 是JobVertex , source是IntermediateDataSet

ExecutionGraph: ExecutionJobVertex + IntermediateDateResult
IntermediateResult 的 producer 是ExecutionJobVertex

物理執行圖：ExecutionVertex + IntermediatePartition
IntermediatePartition的producer是ExecutionVertex，consumers是List<List>
ExecutionEdge的的source是IntermediateResultPartition， target 是ExecutionVertex