org.apache.flink.streaming.api.operators

类 SourceOperator<OUT,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.SourceOperator<OUT,SplitT>

类型参数:

OUT - The output type of the operator.

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.io.AvailabilityProvider, org.apache.flink.runtime.operators.coordination.OperatorEventHandler, KeyContext, SetupableStreamOperator<OUT>, StreamOperator<OUT>, StreamOperatorStateHandler.CheckpointedStreamOperator, PushingAsyncDataInput<OUT>

@Internal
public class SourceOperator<OUT,SplitT extends org.apache.flink.api.connector.source.SourceSplit>
extends AbstractStreamOperator<OUT>
implements org.apache.flink.runtime.operators.coordination.OperatorEventHandler, PushingAsyncDataInput<OUT>

Base source operator only used for integrating the source reader which is proposed by FLIP-27. It implements the interface of PushingAsyncDataInput which is naturally compatible with one input processing in runtime stack.

Important Note on Serialization: The SourceOperator inherits the Serializable interface from the StreamOperator, but is in fact NOT serializable. The operator must only be instantiated in the StreamTask from its factory.

另请参阅:

序列化表格

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.streaming.runtime.io.PushingAsyncDataInput

PushingAsyncDataInput.DataOutput<T>

从接口继承的嵌套类/接口 org.apache.flink.runtime.io.AvailabilityProvider

org.apache.flink.runtime.io.AvailabilityProvider.AvailabilityHelper

字段概要

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

从接口继承的字段 org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

构造器概要

构造器

构造器和说明

SourceOperator(org.apache.flink.util.function.FunctionWithException<org.apache.flink.api.connector.source.SourceReaderContext,org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>,Exception> readerFactory, org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway, org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer, org.apache.flink.api.common.eventtime.WatermarkStrategy<OUT> watermarkStrategy, ProcessingTimeService timeService, org.apache.flink.configuration.Configuration configuration, String localHostname, boolean emitProgressiveWatermarks)

方法概要

限定符和类型	方法和说明
void	close() This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.
DataInputStatus	emitNext(PushingAsyncDataInput.DataOutput<OUT> output) Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.
void	finish() This method is called at the end of data processing.
CompletableFuture<?>	getAvailableFuture()
org.apache.flink.runtime.metrics.groups.InternalSourceReaderMetricGroup	getSourceMetricGroup()
org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>	getSourceReader()
void	handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent event)
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context) Stream operators with state which can be restored need to override this hook method.
void	initReader() Initializes the reader.
protected void	initSourceMetricGroup()
void	notifyCheckpointAborted(long checkpointId)
void	notifyCheckpointComplete(long checkpointId)
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.
void	setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output) Initializes the operator.
void	snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context) Stream operators with state, which want to participate in a snapshot need to override this hook method.
CompletableFuture<Void>	stop(org.apache.flink.runtime.io.network.api.StopMode mode)

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, registerCounterOnOutput, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.io.AvailabilityProvider

and, isApproximatelyAvailable, isAvailable, or

构造器详细资料

SourceOperator

public SourceOperator(org.apache.flink.util.function.FunctionWithException<org.apache.flink.api.connector.source.SourceReaderContext,org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>,Exception> readerFactory,
                      org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway,
                      org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer,
                      org.apache.flink.api.common.eventtime.WatermarkStrategy<OUT> watermarkStrategy,
                      ProcessingTimeService timeService,
                      org.apache.flink.configuration.Configuration configuration,
                      String localHostname,
                      boolean emitProgressiveWatermarks)

方法详细资料

setup

public void setup(StreamTask<?,?> containingTask,
                  StreamConfig config,
                  Output<StreamRecord<OUT>> output)

从接口复制的说明: SetupableStreamOperator

Initializes the operator. Sets access to the context and the output.

指定者:

setup 在接口中 SetupableStreamOperator<OUT>

覆盖:

setup 在类中 AbstractStreamOperator<OUT>

initSourceMetricGroup

@VisibleForTesting
protected void initSourceMetricGroup()

initReader

public void initReader()
                throws Exception

Initializes the reader. The code from this method should ideally happen in the constructor or in the operator factory even. It has to happen here at a slightly later stage, because of the lazy metric initialization.

Calling this method explicitly is an optional way to have the reader initialization a bit earlier than in open(), as needed by the SourceOperatorStreamTask

This code should move to the constructor once the metric groups are available at task setup time.

抛出:

Exception

getSourceMetricGroup

public org.apache.flink.runtime.metrics.groups.InternalSourceReaderMetricGroup getSourceMetricGroup()

open

public void open()
          throws Exception

从类复制的说明: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

指定者:

open 在接口中 StreamOperator<OUT>

覆盖:

open 在类中 AbstractStreamOperator<OUT>

抛出:

Exception - An exception in this method causes the operator to fail.

finish

public void finish()
            throws Exception

从接口复制的说明: StreamOperator

This method is called at the end of data processing.

The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered data should be propagated, in order to cause the operation to be recognized as failed, because the last data items are not processed properly.

After this method is called, no more records can be produced for the downstream operators.

WARNING: It is not safe to use this method to commit any transactions or other side effects! You can use this method to flush any buffered data that can later on be committed e.g. in a CheckpointListener.notifyCheckpointComplete(long).

NOTE:This method does not need to close any resources. You should release external resources in the StreamOperator.close() method.

指定者:

finish 在接口中 StreamOperator<OUT>

覆盖:

finish 在类中 AbstractStreamOperator<OUT>

抛出:

Exception - An exception in this method causes the operator to fail.

stop

public CompletableFuture<Void> stop(org.apache.flink.runtime.io.network.api.StopMode mode)

close

public void close()
           throws Exception

从接口复制的说明: StreamOperator

This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.

This method is expected to make a thorough effort to release all resources that the operator has acquired.

NOTE:It can not emit any records! If you need to emit records at the end of processing, do so in the StreamOperator.finish() method.

指定者:

close 在接口中 StreamOperator<OUT>

覆盖:

close 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

emitNext

public DataInputStatus emitNext(PushingAsyncDataInput.DataOutput<OUT> output)
                         throws Exception

从接口复制的说明: PushingAsyncDataInput

Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.

This method should be non blocking.

指定者:

emitNext 在接口中 PushingAsyncDataInput<OUT>

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception

从类复制的说明: AbstractStreamOperator

Stream operators with state, which want to participate in a snapshot need to override this hook method.

指定者:

snapshotState 在接口中 StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

snapshotState 在类中 AbstractStreamOperator<OUT>

参数:

context - context that provides information and means required for taking a snapshot

抛出:

Exception

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

指定者:

getAvailableFuture 在接口中 org.apache.flink.runtime.io.AvailabilityProvider

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

从类复制的说明: AbstractStreamOperator

Stream operators with state which can be restored need to override this hook method.

指定者:

initializeState 在接口中 StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

initializeState 在类中 AbstractStreamOperator<OUT>

参数:

context - context that allows to register different states.

抛出:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.api.common.state.CheckpointListener

覆盖:

notifyCheckpointComplete 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)
                             throws Exception

指定者:

notifyCheckpointAborted 在接口中 org.apache.flink.api.common.state.CheckpointListener

覆盖:

notifyCheckpointAborted 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

handleOperatorEvent

public void handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent event)

指定者:

handleOperatorEvent 在接口中 org.apache.flink.runtime.operators.coordination.OperatorEventHandler

getSourceReader

@VisibleForTesting
public org.apache.flink.api.connector.source.SourceReader<OUT,SplitT> getSourceReader()