org.apache.flink.runtime.dispatcher

类 Dispatcher

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.FencedRpcEndpoint<DispatcherId>

org.apache.flink.runtime.dispatcher.Dispatcher

所有已实现的接口:

AutoCloseable, DispatcherGateway, org.apache.flink.runtime.rpc.FencedRpcGateway<DispatcherId>, org.apache.flink.runtime.rpc.RpcGateway, RestfulGateway, org.apache.flink.util.AutoCloseableAsync

直接已知子类:

MiniDispatcher, StandaloneDispatcher

public abstract class Dispatcher
extends org.apache.flink.runtime.rpc.FencedRpcEndpoint<DispatcherId>
implements DispatcherGateway

Base class for the Dispatcher component. The Dispatcher component is responsible for receiving job submissions, persisting them, spawning JobManagers to execute the jobs and to recover them in case of a master failure. Furthermore, it knows about the state of the Flink session cluster.

嵌套类概要

嵌套类

限定符和类型	类和说明
protected static class	Dispatcher.ExecutionType Enum to distinguish between initial job submission and re-submission for recovery.

从类继承的嵌套类/接口 org.apache.flink.runtime.rpc.RpcEndpoint

org.apache.flink.runtime.rpc.RpcEndpoint.MainThreadExecutor

字段概要

字段

限定符和类型	字段和说明
static String	DISPATCHER_NAME
protected CompletableFuture<ApplicationStatus>	shutDownFuture

从类继承的字段 org.apache.flink.runtime.rpc.RpcEndpoint

log, rpcServer

构造器概要

构造器

限定符	构造器和说明
	Dispatcher(org.apache.flink.runtime.rpc.RpcService rpcService, DispatcherId fencingToken, Collection<JobGraph> recoveredJobs, Collection<JobResult> recoveredDirtyJobs, DispatcherBootstrapFactory dispatcherBootstrapFactory, DispatcherServices dispatcherServices)
protected	Dispatcher(org.apache.flink.runtime.rpc.RpcService rpcService, DispatcherId fencingToken, Collection<JobGraph> recoveredJobs, Collection<JobResult> recoveredDirtyJobs, DispatcherBootstrapFactory dispatcherBootstrapFactory, DispatcherServices dispatcherServices, JobManagerRunnerRegistry jobManagerRunnerRegistry, ResourceCleanerFactory resourceCleanerFactory)

方法概要

限定符和类型	方法和说明
CompletableFuture<Acknowledge>	cancelJob(org.apache.flink.api.common.JobID jobId, org.apache.flink.api.common.time.Time timeout) Cancel the given job.
CompletableFuture<CoordinationResponse>	deliverCoordinationRequestToCoordinator(org.apache.flink.api.common.JobID jobId, OperatorID operatorId, org.apache.flink.util.SerializedValue<CoordinationRequest> serializedRequest, org.apache.flink.api.common.time.Time timeout) Deliver a coordination request to a specified coordinator and return the response.
CompletableFuture<Acknowledge>	disposeSavepoint(String savepointPath, org.apache.flink.api.common.time.Time timeout) Dispose the given savepoint.
CompletableFuture<Integer>	getBlobServerPort(org.apache.flink.api.common.time.Time timeout) Returns the port of the blob server.
CompletableFuture<ApplicationStatus>	getShutDownFuture()
CompletableFuture<OperationResult<String>>	getTriggeredSavepointStatus(AsynchronousJobOperationKey operationKey) Get the status of a savepoint triggered under the specified operation key.
protected CompletableFuture<org.apache.flink.runtime.dispatcher.Dispatcher.CleanupJobState>	jobReachedTerminalState(ExecutionGraphInfo executionGraphInfo)
CompletableFuture<Collection<org.apache.flink.api.common.JobID>>	listJobs(org.apache.flink.api.common.time.Time timeout) List the current set of submitted jobs.
protected void	onFatalError(Throwable throwable)
CompletableFuture<Void>	onRemovedJobGraph(org.apache.flink.api.common.JobID jobId)
void	onStart()
CompletableFuture<Void>	onStop()
CompletableFuture<ClusterOverview>	requestClusterOverview(org.apache.flink.api.common.time.Time timeout) Requests the cluster status overview.
CompletableFuture<ExecutionGraphInfo>	requestExecutionGraphInfo(org.apache.flink.api.common.JobID jobId, org.apache.flink.api.common.time.Time timeout) Requests the ExecutionGraphInfo containing additional information besides the ArchivedExecutionGraph.
CompletableFuture<JobResult>	requestJobResult(org.apache.flink.api.common.JobID jobId, org.apache.flink.api.common.time.Time timeout) Requests the JobResult of a job specified by the given jobId.
CompletableFuture<org.apache.flink.api.common.JobStatus>	requestJobStatus(org.apache.flink.api.common.JobID jobId, org.apache.flink.api.common.time.Time timeout) Request the JobStatus of the given job.
CompletableFuture<Collection<String>>	requestMetricQueryServiceAddresses(org.apache.flink.api.common.time.Time timeout) Requests the addresses of the MetricQueryService to query.
CompletableFuture<MultipleJobsDetails>	requestMultipleJobDetails(org.apache.flink.api.common.time.Time timeout) Requests job details currently being executed on the Flink cluster.
CompletableFuture<Collection<org.apache.flink.api.java.tuple.Tuple2<ResourceID,String>>>	requestTaskManagerMetricQueryServiceAddresses(org.apache.flink.api.common.time.Time timeout) Requests the addresses for the TaskManagers' MetricQueryService to query.
CompletableFuture<ThreadDumpInfo>	requestThreadDump(org.apache.flink.api.common.time.Time timeout) Requests the thread dump from the JobManager.
protected void	runPostJobGloballyTerminated(org.apache.flink.api.common.JobID jobId, org.apache.flink.api.common.JobStatus jobStatus)
CompletableFuture<Acknowledge>	shutDownCluster()
CompletableFuture<Acknowledge>	shutDownCluster(ApplicationStatus applicationStatus)
CompletableFuture<Acknowledge>	stopWithSavepoint(AsynchronousJobOperationKey operationKey, String targetDirectory, org.apache.flink.core.execution.SavepointFormatType formatType, TriggerSavepointMode savepointMode, org.apache.flink.api.common.time.Time timeout) Stops the job with a savepoint, returning a future that completes when the operation is started.
CompletableFuture<String>	stopWithSavepointAndGetLocation(org.apache.flink.api.common.JobID jobId, String targetDirectory, org.apache.flink.core.execution.SavepointFormatType formatType, TriggerSavepointMode savepointMode, org.apache.flink.api.common.time.Time timeout) Stops the job with a savepoint, returning a future that completes with the savepoint location when the savepoint is completed.
CompletableFuture<Acknowledge>	submitFailedJob(org.apache.flink.api.common.JobID jobId, String jobName, Throwable exception)
CompletableFuture<Acknowledge>	submitJob(JobGraph jobGraph, org.apache.flink.api.common.time.Time timeout) Submit a job to the dispatcher.
CompletableFuture<String>	triggerCheckpoint(org.apache.flink.api.common.JobID jobID, org.apache.flink.api.common.time.Time timeout)
CompletableFuture<Acknowledge>	triggerSavepoint(AsynchronousJobOperationKey operationKey, String targetDirectory, org.apache.flink.core.execution.SavepointFormatType formatType, TriggerSavepointMode savepointMode, org.apache.flink.api.common.time.Time timeout) Triggers a savepoint with the given savepoint directory as a target, returning a future that completes when the operation is started.
CompletableFuture<String>	triggerSavepointAndGetLocation(org.apache.flink.api.common.JobID jobId, String targetDirectory, org.apache.flink.core.execution.SavepointFormatType formatType, TriggerSavepointMode savepointMode, org.apache.flink.api.common.time.Time timeout) Triggers a savepoint with the given savepoint directory as a target, returning a future that completes with the savepoint location when it is complete.

从类继承的方法 org.apache.flink.runtime.rpc.FencedRpcEndpoint

getFencingToken

从类继承的方法 org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, closeAsync, getAddress, getEndpointId, getHostname, getMainThreadExecutor, getRpcService, getSelfGateway, getTerminationFuture, internalCallOnStart, internalCallOnStop, isRunning, registerResource, runAsync, scheduleRunAsync, scheduleRunAsync, start, stop, unregisterResource, validateRunsInMainThread

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

从接口继承的方法 org.apache.flink.runtime.webmonitor.RestfulGateway

requestJob

从接口继承的方法 org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

从接口继承的方法 org.apache.flink.util.AutoCloseableAsync

close

字段详细资料

DISPATCHER_NAME

public static final String DISPATCHER_NAME

另请参阅:

常量字段值

shutDownFuture

protected final CompletableFuture<ApplicationStatus> shutDownFuture

构造器详细资料

Dispatcher

public Dispatcher(org.apache.flink.runtime.rpc.RpcService rpcService,
                  DispatcherId fencingToken,
                  Collection<JobGraph> recoveredJobs,
                  Collection<JobResult> recoveredDirtyJobs,
                  DispatcherBootstrapFactory dispatcherBootstrapFactory,
                  DispatcherServices dispatcherServices)
           throws Exception

抛出:

Exception

Dispatcher

@VisibleForTesting
protected Dispatcher(org.apache.flink.runtime.rpc.RpcService rpcService,
                                        DispatcherId fencingToken,
                                        Collection<JobGraph> recoveredJobs,
                                        Collection<JobResult> recoveredDirtyJobs,
                                        DispatcherBootstrapFactory dispatcherBootstrapFactory,
                                        DispatcherServices dispatcherServices,
                                        JobManagerRunnerRegistry jobManagerRunnerRegistry,
                                        ResourceCleanerFactory resourceCleanerFactory)
                                 throws Exception

抛出:

Exception

方法详细资料

getShutDownFuture

public CompletableFuture<ApplicationStatus> getShutDownFuture()

onStart

public void onStart()
             throws Exception

覆盖:

onStart 在类中 org.apache.flink.runtime.rpc.RpcEndpoint

抛出:

Exception

onStop

public CompletableFuture<Void> onStop()

覆盖:

onStop 在类中 org.apache.flink.runtime.rpc.RpcEndpoint

submitJob

public CompletableFuture<Acknowledge> submitJob(JobGraph jobGraph,
                                                org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: DispatcherGateway

Submit a job to the dispatcher.

指定者:

submitJob 在接口中 DispatcherGateway

参数:

jobGraph - JobGraph to submit

timeout - RPC timeout

返回:

A future acknowledge if the submission succeeded

submitFailedJob

public CompletableFuture<Acknowledge> submitFailedJob(org.apache.flink.api.common.JobID jobId,
                                                      String jobName,
                                                      Throwable exception)

指定者:

submitFailedJob 在接口中 DispatcherGateway

listJobs

public CompletableFuture<Collection<org.apache.flink.api.common.JobID>> listJobs(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: DispatcherGateway

List the current set of submitted jobs.

指定者:

listJobs 在接口中 DispatcherGateway

参数:

timeout - RPC timeout

返回:

A future collection of currently submitted jobs

disposeSavepoint

public CompletableFuture<Acknowledge> disposeSavepoint(String savepointPath,
                                                       org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Dispose the given savepoint.

指定者:

disposeSavepoint 在接口中 RestfulGateway

参数:

savepointPath - identifying the savepoint to dispose

timeout - RPC timeout

返回:

A future acknowledge if the disposal succeeded

cancelJob

public CompletableFuture<Acknowledge> cancelJob(org.apache.flink.api.common.JobID jobId,
                                                org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Cancel the given job.

指定者:

cancelJob 在接口中 RestfulGateway

参数:

jobId - identifying the job to cancel

timeout - of the operation

返回:

A future acknowledge if the cancellation succeeded

requestClusterOverview

public CompletableFuture<ClusterOverview> requestClusterOverview(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the cluster status overview.

指定者:

requestClusterOverview 在接口中 RestfulGateway

参数:

timeout - for the asynchronous operation

返回:

Future containing the status overview

requestMultipleJobDetails

public CompletableFuture<MultipleJobsDetails> requestMultipleJobDetails(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests job details currently being executed on the Flink cluster.

指定者:

requestMultipleJobDetails 在接口中 RestfulGateway

参数:

timeout - for the asynchronous operation

返回:

Future containing the job details

requestJobStatus

public CompletableFuture<org.apache.flink.api.common.JobStatus> requestJobStatus(org.apache.flink.api.common.JobID jobId,
                                                                                 org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Request the JobStatus of the given job.

指定者:

requestJobStatus 在接口中 RestfulGateway

参数:

jobId - identifying the job for which to retrieve the JobStatus

timeout - for the asynchronous operation

返回:

A future to the JobStatus of the given job

requestExecutionGraphInfo

public CompletableFuture<ExecutionGraphInfo> requestExecutionGraphInfo(org.apache.flink.api.common.JobID jobId,
                                                                       org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the ExecutionGraphInfo containing additional information besides the ArchivedExecutionGraph. If there is no such graph, then the future is completed with a FlinkJobNotFoundException.

指定者:

requestExecutionGraphInfo 在接口中 RestfulGateway

参数:

jobId - identifying the job whose ExecutionGraphInfo is requested

timeout - for the asynchronous operation

返回:

Future containing the ExecutionGraphInfo for the given jobId, otherwise FlinkJobNotFoundException

requestJobResult

public CompletableFuture<JobResult> requestJobResult(org.apache.flink.api.common.JobID jobId,
                                                     org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the JobResult of a job specified by the given jobId.

指定者:

requestJobResult 在接口中 RestfulGateway

参数:

jobId - identifying the job for which to retrieve the JobResult.

timeout - for the asynchronous operation

返回:

Future which is completed with the job's JobResult once the job has finished

requestMetricQueryServiceAddresses

public CompletableFuture<Collection<String>> requestMetricQueryServiceAddresses(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the addresses of the MetricQueryService to query.

指定者:

requestMetricQueryServiceAddresses 在接口中 RestfulGateway

参数:

timeout - for the asynchronous operation

返回:

Future containing the collection of metric query service addresses to query

requestTaskManagerMetricQueryServiceAddresses

public CompletableFuture<Collection<org.apache.flink.api.java.tuple.Tuple2<ResourceID,String>>> requestTaskManagerMetricQueryServiceAddresses(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the addresses for the TaskManagers' MetricQueryService to query.

指定者:

requestTaskManagerMetricQueryServiceAddresses 在接口中 RestfulGateway

参数:

timeout - for the asynchronous operation

返回:

Future containing the collection of instance ids and the corresponding metric query service address

requestThreadDump

public CompletableFuture<ThreadDumpInfo> requestThreadDump(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Requests the thread dump from the JobManager.

指定者:

requestThreadDump 在接口中 RestfulGateway

参数:

timeout - timeout of the asynchronous operation

返回:

Future containing the thread dump information

getBlobServerPort

public CompletableFuture<Integer> getBlobServerPort(org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: DispatcherGateway

Returns the port of the blob server.

指定者:

getBlobServerPort 在接口中 DispatcherGateway

参数:

timeout - of the operation

返回:

A future integer of the blob server port

triggerCheckpoint

public CompletableFuture<String> triggerCheckpoint(org.apache.flink.api.common.JobID jobID,
                                                   org.apache.flink.api.common.time.Time timeout)

指定者:

triggerCheckpoint 在接口中 DispatcherGateway

triggerSavepoint

public CompletableFuture<Acknowledge> triggerSavepoint(AsynchronousJobOperationKey operationKey,
                                                       String targetDirectory,
                                                       org.apache.flink.core.execution.SavepointFormatType formatType,
                                                       TriggerSavepointMode savepointMode,
                                                       org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Triggers a savepoint with the given savepoint directory as a target, returning a future that completes when the operation is started.

指定者:

triggerSavepoint 在接口中 RestfulGateway

参数:

operationKey - the key of the operation, for deduplication purposes

targetDirectory - Target directory for the savepoint.

formatType - Binary format of the savepoint.

savepointMode - context of the savepoint operation

timeout - Timeout for the asynchronous operation

返回:

Future which is completed once the operation is triggered successfully

triggerSavepointAndGetLocation

public CompletableFuture<String> triggerSavepointAndGetLocation(org.apache.flink.api.common.JobID jobId,
                                                                String targetDirectory,
                                                                org.apache.flink.core.execution.SavepointFormatType formatType,
                                                                TriggerSavepointMode savepointMode,
                                                                org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: DispatcherGateway

Triggers a savepoint with the given savepoint directory as a target, returning a future that completes with the savepoint location when it is complete.

指定者:

triggerSavepointAndGetLocation 在接口中 DispatcherGateway

参数:

jobId - the job id

targetDirectory - Target directory for the savepoint.

formatType - Binary format of the savepoint.

savepointMode - context of the savepoint operation

timeout - Timeout for the asynchronous operation

返回:

Future which is completed once the operation is triggered successfully

getTriggeredSavepointStatus

public CompletableFuture<OperationResult<String>> getTriggeredSavepointStatus(AsynchronousJobOperationKey operationKey)

从接口复制的说明: RestfulGateway

Get the status of a savepoint triggered under the specified operation key.

指定者:

getTriggeredSavepointStatus 在接口中 RestfulGateway

参数:

operationKey - key of the operation

返回:

Future which completes immediately with the status, or fails if no operation is registered for the key

stopWithSavepoint

public CompletableFuture<Acknowledge> stopWithSavepoint(AsynchronousJobOperationKey operationKey,
                                                        String targetDirectory,
                                                        org.apache.flink.core.execution.SavepointFormatType formatType,
                                                        TriggerSavepointMode savepointMode,
                                                        org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Stops the job with a savepoint, returning a future that completes when the operation is started.

指定者:

stopWithSavepoint 在接口中 RestfulGateway

参数:

operationKey - key of the operation, for deduplication

targetDirectory - Target directory for the savepoint.

formatType - Binary format of the savepoint.

savepointMode - context of the savepoint operation

timeout - for the rpc call

返回:

Future which is completed once the operation is triggered successfully

stopWithSavepointAndGetLocation

public CompletableFuture<String> stopWithSavepointAndGetLocation(org.apache.flink.api.common.JobID jobId,
                                                                 String targetDirectory,
                                                                 org.apache.flink.core.execution.SavepointFormatType formatType,
                                                                 TriggerSavepointMode savepointMode,
                                                                 org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: DispatcherGateway

Stops the job with a savepoint, returning a future that completes with the savepoint location when the savepoint is completed.

指定者:

stopWithSavepointAndGetLocation 在接口中 DispatcherGateway

参数:

jobId - the job id

targetDirectory - Target directory for the savepoint.

savepointMode - context of the savepoint operation

timeout - for the rpc call

返回:

Future which is completed with the savepoint location once it is completed

shutDownCluster

public CompletableFuture<Acknowledge> shutDownCluster()

指定者:

shutDownCluster 在接口中 RestfulGateway

shutDownCluster

public CompletableFuture<Acknowledge> shutDownCluster(ApplicationStatus applicationStatus)

指定者:

shutDownCluster 在接口中 DispatcherGateway

deliverCoordinationRequestToCoordinator

public CompletableFuture<CoordinationResponse> deliverCoordinationRequestToCoordinator(org.apache.flink.api.common.JobID jobId,
                                                                                       OperatorID operatorId,
                                                                                       org.apache.flink.util.SerializedValue<CoordinationRequest> serializedRequest,
                                                                                       org.apache.flink.api.common.time.Time timeout)

从接口复制的说明: RestfulGateway

Deliver a coordination request to a specified coordinator and return the response.

指定者:

deliverCoordinationRequestToCoordinator 在接口中 RestfulGateway

参数:

jobId - identifying the job which the coordinator belongs to

operatorId - identifying the coordinator to receive the request

serializedRequest - serialized request to deliver

timeout - RPC timeout

返回:

A future containing the response. The response will fail with a FlinkException if the task is not running, or no operator/coordinator exists for the given ID, or the coordinator cannot handle client events.

runPostJobGloballyTerminated

protected void runPostJobGloballyTerminated(org.apache.flink.api.common.JobID jobId,
                                            org.apache.flink.api.common.JobStatus jobStatus)

onFatalError

protected void onFatalError(Throwable throwable)

jobReachedTerminalState

@VisibleForTesting
protected CompletableFuture<org.apache.flink.runtime.dispatcher.Dispatcher.CleanupJobState> jobReachedTerminalState(ExecutionGraphInfo executionGraphInfo)

onRemovedJobGraph

public CompletableFuture<Void> onRemovedJobGraph(org.apache.flink.api.common.JobID jobId)