接口的使用
org.apache.flink.runtime.shuffle.ShuffleMaster

使用ShuffleMaster的程序包

程序包	说明
org.apache.flink.runtime.executiongraph
org.apache.flink.runtime.io.network.partition
org.apache.flink.runtime.jobmaster
org.apache.flink.runtime.scheduler
org.apache.flink.runtime.scheduler.adaptive
org.apache.flink.runtime.scheduler.adaptivebatch
org.apache.flink.runtime.shuffle

org.apache.flink.runtime.executiongraph中ShuffleMaster的使用

返回ShuffleMaster的org.apache.flink.runtime.executiongraph中的方法

限定符和类型	方法和说明
ShuffleMaster<?>	DefaultExecutionGraph.getShuffleMaster()
ShuffleMaster<? extends ShuffleDescriptor>	InternalExecutionGraphAccessor.getShuffleMaster()

参数类型为ShuffleMaster的org.apache.flink.runtime.executiongraph中的方法

限定符和类型

方法和说明

static DefaultExecutionGraph

DefaultExecutionGraphBuilder.buildGraph(JobGraph jobGraph, org.apache.flink.configuration.Configuration jobManagerConfig, ScheduledExecutorService futureExecutor, Executor ioExecutor, ClassLoader classLoader, CompletedCheckpointStore completedCheckpointStore, CheckpointsCleaner checkpointsCleaner, CheckpointIDCounter checkpointIdCounter, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, org.slf4j.Logger log, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint, ExecutionDeploymentListener executionDeploymentListener, ExecutionStateUpdateListener executionStateUpdateListener, long initializationTimestamp, VertexAttemptNumberStore vertexAttemptNumberStore, VertexParallelismStore vertexParallelismStore, java.util.function.Supplier<CheckpointStatsTracker> checkpointStatsTrackerFactory, boolean isDynamicGraph, ExecutionJobVertex.Factory executionJobVertexFactory)

参数类型为ShuffleMaster的org.apache.flink.runtime.executiongraph中的构造器

构造器和说明

DefaultExecutionGraph(JobInformation jobInformation, ScheduledExecutorService futureExecutor, Executor ioExecutor, org.apache.flink.api.common.time.Time rpcTimeout, int executionHistorySizeLimit, ClassLoader userClassLoader, BlobWriter blobWriter, PartitionGroupReleaseStrategy.Factory partitionGroupReleaseStrategyFactory, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint, ExecutionDeploymentListener executionDeploymentListener, ExecutionStateUpdateListener executionStateUpdateListener, long initializationTimestamp, VertexAttemptNumberStore initialAttemptCounts, VertexParallelismStore vertexParallelismStore, boolean isDynamic, ExecutionJobVertex.Factory executionJobVertexFactory, List<JobStatusHook> jobStatusHooks)

org.apache.flink.runtime.io.network.partition中ShuffleMaster的使用

参数类型为ShuffleMaster的org.apache.flink.runtime.io.network.partition中的构造器

构造器和说明
JobMasterPartitionTrackerImpl(org.apache.flink.api.common.JobID jobId, ShuffleMaster<?> shuffleMaster, PartitionTrackerFactory.TaskExecutorGatewayLookup taskExecutorGatewayLookup)

org.apache.flink.runtime.jobmaster中ShuffleMaster的使用

返回ShuffleMaster的org.apache.flink.runtime.jobmaster中的方法

限定符和类型	方法和说明
ShuffleMaster<?>	JobManagerSharedServices.getShuffleMaster()

参数类型为ShuffleMaster的org.apache.flink.runtime.jobmaster中的方法

限定符和类型	方法和说明
SchedulerNG	DefaultSlotPoolServiceSchedulerFactory.createScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration configuration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations)
SchedulerNG	SlotPoolServiceSchedulerFactory.createScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration configuration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations) Creates a SchedulerNG.

参数类型为ShuffleMaster的org.apache.flink.runtime.jobmaster中的构造器

构造器和说明
JobManagerSharedServices(ScheduledExecutorService futureExecutor, ExecutorService ioExecutor, LibraryCacheManager libraryCacheManager, ShuffleMaster<?> shuffleMaster, BlobWriter blobWriter)
JobMaster(org.apache.flink.runtime.rpc.RpcService rpcService, JobMasterId jobMasterId, JobMasterConfiguration jobMasterConfiguration, ResourceID resourceId, JobGraph jobGraph, HighAvailabilityServices highAvailabilityService, SlotPoolServiceSchedulerFactory slotPoolServiceSchedulerFactory, JobManagerSharedServices jobManagerSharedServices, HeartbeatServices heartbeatServices, JobManagerJobMetricGroupFactory jobMetricGroupFactory, OnCompletionActions jobCompletionActions, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, ClassLoader userCodeLoader, ShuffleMaster<?> shuffleMaster, PartitionTrackerFactory partitionTrackerFactory, ExecutionDeploymentTracker executionDeploymentTracker, ExecutionDeploymentReconciler.Factory executionDeploymentReconcilerFactory, BlocklistHandler.Factory blocklistHandlerFactory, long initializationTimestamp)

org.apache.flink.runtime.scheduler中ShuffleMaster的使用

参数类型为ShuffleMaster的org.apache.flink.runtime.scheduler中的方法

限定符和类型	方法和说明
SchedulerNG	DefaultSchedulerFactory.createInstance(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations)
SchedulerNG	SchedulerNGFactory.createInstance(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations)
static void	SsgNetworkMemoryCalculationUtils.enrichNetworkMemory(SlotSharingGroup ssg, java.util.function.Function<JobVertexID,ExecutionJobVertex> ejvs, ShuffleMaster<?> shuffleMaster) Calculates network memory requirement of ExecutionJobVertex and update ResourceProfile of corresponding slot sharing group.

参数类型为ShuffleMaster的org.apache.flink.runtime.scheduler中的构造器

构造器和说明
DefaultExecutionGraphFactory(org.apache.flink.configuration.Configuration configuration, ClassLoader userCodeClassLoader, ExecutionDeploymentTracker executionDeploymentTracker, ScheduledExecutorService futureExecutor, Executor ioExecutor, org.apache.flink.api.common.time.Time rpcTimeout, JobManagerJobMetricGroup jobManagerJobMetricGroup, BlobWriter blobWriter, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker jobMasterPartitionTracker)
DefaultExecutionGraphFactory(org.apache.flink.configuration.Configuration configuration, ClassLoader userCodeClassLoader, ExecutionDeploymentTracker executionDeploymentTracker, ScheduledExecutorService futureExecutor, Executor ioExecutor, org.apache.flink.api.common.time.Time rpcTimeout, JobManagerJobMetricGroup jobManagerJobMetricGroup, BlobWriter blobWriter, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker jobMasterPartitionTracker, boolean isDynamicGraph, ExecutionJobVertex.Factory executionJobVertexFactory)
DefaultScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionOperations executionOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, org.apache.flink.api.common.time.Time rpcTimeout, VertexParallelismStore vertexParallelismStore, ExecutionDeployer.Factory executionDeployerFactory)

org.apache.flink.runtime.scheduler.adaptive中ShuffleMaster的使用

参数类型为ShuffleMaster的org.apache.flink.runtime.scheduler.adaptive中的方法

限定符和类型

方法和说明

SchedulerNG

AdaptiveSchedulerFactory.createInstance(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations)

org.apache.flink.runtime.scheduler.adaptivebatch中ShuffleMaster的使用

参数类型为ShuffleMaster的org.apache.flink.runtime.scheduler.adaptivebatch中的方法

限定符和类型

方法和说明

SchedulerNG

AdaptiveBatchSchedulerFactory.createInstance(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, SlotPoolService slotPoolService, ScheduledExecutorService futureExecutor, ClassLoader userCodeLoader, CheckpointRecoveryFactory checkpointRecoveryFactory, org.apache.flink.api.common.time.Time rpcTimeout, BlobWriter blobWriter, JobManagerJobMetricGroup jobManagerJobMetricGroup, org.apache.flink.api.common.time.Time slotRequestTimeout, ShuffleMaster<?> shuffleMaster, JobMasterPartitionTracker partitionTracker, ExecutionDeploymentTracker executionDeploymentTracker, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, org.apache.flink.runtime.rpc.FatalErrorHandler fatalErrorHandler, JobStatusListener jobStatusListener, BlocklistOperations blocklistOperations)

参数类型为ShuffleMaster的org.apache.flink.runtime.scheduler.adaptivebatch中的构造器

构造器和说明
AdaptiveBatchScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionOperations executionOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, org.apache.flink.api.common.time.Time rpcTimeout, VertexParallelismDecider vertexParallelismDecider, int defaultMaxParallelism)
SpeculativeScheduler(org.slf4j.Logger log, JobGraph jobGraph, Executor ioExecutor, org.apache.flink.configuration.Configuration jobMasterConfiguration, java.util.function.Consumer<org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor> startUpAction, org.apache.flink.util.concurrent.ScheduledExecutor delayExecutor, ClassLoader userCodeLoader, CheckpointsCleaner checkpointsCleaner, CheckpointRecoveryFactory checkpointRecoveryFactory, JobManagerJobMetricGroup jobManagerJobMetricGroup, SchedulingStrategyFactory schedulingStrategyFactory, FailoverStrategy.Factory failoverStrategyFactory, RestartBackoffTimeStrategy restartBackoffTimeStrategy, ExecutionOperations executionOperations, ExecutionVertexVersioner executionVertexVersioner, ExecutionSlotAllocatorFactory executionSlotAllocatorFactory, long initializationTimestamp, org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor mainThreadExecutor, JobStatusListener jobStatusListener, ExecutionGraphFactory executionGraphFactory, ShuffleMaster<?> shuffleMaster, org.apache.flink.api.common.time.Time rpcTimeout, VertexParallelismDecider vertexParallelismDecider, int defaultMaxParallelism, BlocklistOperations blocklistOperations)

org.apache.flink.runtime.shuffle中ShuffleMaster的使用

实现ShuffleMaster的org.apache.flink.runtime.shuffle中的类

限定符和类型	类和说明
class	NettyShuffleMaster Default ShuffleMaster for netty and local file based shuffle implementation.

返回ShuffleMaster的org.apache.flink.runtime.shuffle中的方法

限定符和类型	方法和说明
ShuffleMaster<SD>	ShuffleServiceFactory.createShuffleMaster(ShuffleMasterContext shuffleMasterContext) Factory method to create a specific ShuffleMaster implementation.