org.apache.flink.streaming.runtime.io.recovery

类 RescalingStreamTaskNetworkInput<T>

java.lang.Object

org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput<T,org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T>>

org.apache.flink.streaming.runtime.io.recovery.RescalingStreamTaskNetworkInput<T>

所有已实现的接口:

Closeable, AutoCloseable, org.apache.flink.runtime.io.AvailabilityProvider, PushingAsyncDataInput<T>, RecoverableStreamTaskInput<T>, StreamTaskInput<T>

@Internal
public final class RescalingStreamTaskNetworkInput<T>
extends AbstractStreamTaskNetworkInput<T,org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T>>
implements RecoverableStreamTaskInput<T>

A StreamTaskNetworkInput implementation that demultiplexes virtual channels.

The demultiplexing works in two dimensions for the following cases. *

• Subtasks of the current operator have been collapsed in a round-robin fashion.
• The connected output operator has been rescaled (up and down!) and there is an overlap of channels (mostly relevant to keyed exchanges).

In both cases, records from multiple old channels are received over one new physical channel, which need to demultiplex the record to correctly restore spanning records (similar to how StreamTaskNetworkInput works).

Note that when both cases occur at the same time (downscaling of several operators), there is the cross product of channels. So if two subtasks are collapsed and two channels overlap from the output side, there is a total of 4 virtual channels.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.streaming.runtime.io.PushingAsyncDataInput

PushingAsyncDataInput.DataOutput<T>

从接口继承的嵌套类/接口 org.apache.flink.runtime.io.AvailabilityProvider

org.apache.flink.runtime.io.AvailabilityProvider.AvailabilityHelper

字段概要

从类继承的字段 org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput

checkpointedInputGate, deserializationDelegate, flattenedChannelIndices, inputIndex, inputSerializer, recordDeserializers, statusWatermarkValve

从接口继承的字段 org.apache.flink.streaming.runtime.io.StreamTaskInput

UNSPECIFIED

从接口继承的字段 org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

构造器概要

构造器

构造器和说明

RescalingStreamTaskNetworkInput(CheckpointedInputGate checkpointedInputGate, org.apache.flink.api.common.typeutils.TypeSerializer<T> inputSerializer, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, StatusWatermarkValve statusWatermarkValve, int inputIndex, org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor inflightDataRescalingDescriptor, java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners, org.apache.flink.api.common.TaskInfo taskInfo)

方法概要

限定符和类型	方法和说明
StreamTaskInput<T>	finishRecovery()
protected org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T>	getActiveSerializer(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo)
CompletableFuture<Void>	prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter, long checkpointId) Prepares to spill the in-flight input buffers as checkpoint snapshot.
protected DataInputStatus	processEvent(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent bufferOrEvent)

从类继承的方法 org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput

close, emitNext, getAvailableFuture, getInputIndex, processBuffer, releaseDeserializer

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.runtime.io.StreamTaskInput

getInputIndex

从接口继承的方法 org.apache.flink.streaming.runtime.io.PushingAsyncDataInput

emitNext

从接口继承的方法 org.apache.flink.runtime.io.AvailabilityProvider

and, getAvailableFuture, isApproximatelyAvailable, isAvailable, or

从接口继承的方法 java.io.Closeable

close

构造器详细资料

RescalingStreamTaskNetworkInput

public RescalingStreamTaskNetworkInput(CheckpointedInputGate checkpointedInputGate,
                                       org.apache.flink.api.common.typeutils.TypeSerializer<T> inputSerializer,
                                       org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                                       StatusWatermarkValve statusWatermarkValve,
                                       int inputIndex,
                                       org.apache.flink.runtime.checkpoint.InflightDataRescalingDescriptor inflightDataRescalingDescriptor,
                                       java.util.function.Function<Integer,StreamPartitioner<?>> gatePartitioners,
                                       org.apache.flink.api.common.TaskInfo taskInfo)

方法详细资料

finishRecovery

public StreamTaskInput<T> finishRecovery()
                                  throws IOException

指定者:

finishRecovery 在接口中 RecoverableStreamTaskInput<T>

抛出:

IOException

getActiveSerializer

protected org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T> getActiveSerializer(org.apache.flink.runtime.checkpoint.channel.InputChannelInfo channelInfo)

覆盖:

getActiveSerializer 在类中 AbstractStreamTaskNetworkInput<T,org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T>>

processEvent

protected DataInputStatus processEvent(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent bufferOrEvent)

覆盖:

processEvent 在类中 AbstractStreamTaskNetworkInput<T,org.apache.flink.streaming.runtime.io.recovery.DemultiplexingRecordDeserializer<T>>

prepareSnapshot

public CompletableFuture<Void> prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter,
                                               long checkpointId)
                                        throws org.apache.flink.runtime.checkpoint.CheckpointException

从接口复制的说明: StreamTaskInput

Prepares to spill the in-flight input buffers as checkpoint snapshot.

指定者:

prepareSnapshot 在接口中 StreamTaskInput<T>

抛出:

org.apache.flink.runtime.checkpoint.CheckpointException