Package org.apache.hadoop.ozone.container.common.transport.server.ratis

Class ContainerStateMachine

java.lang.Object
org.apache.ratis.statemachine.impl.BaseStateMachine
org.apache.hadoop.ozone.container.common.transport.server.ratis.ContainerStateMachine
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.ratis.statemachine.StateMachine, org.apache.ratis.statemachine.StateMachine.DataApi, org.apache.ratis.statemachine.StateMachine.EventApi, org.apache.ratis.statemachine.StateMachine.FollowerEventApi, org.apache.ratis.statemachine.StateMachine.LeaderEventApi
public class ContainerStateMachine extends org.apache.ratis.statemachine.impl.BaseStateMachine
A StateMachine for containers, which is responsible for handling different types of container requests.

The container requests can be divided into readonly request, WriteChunk request and other write requests. - Read only requests (see HddsUtils.isReadOnly(org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandRequestProtoOrBuilder)) are handled by query(Message). - WriteChunk request contains user data - Other write request does not contain user data.

In order to optimize the write throughput, a WriteChunk request is processed : (1) startTransaction(RaftClientRequest) separate user data from the client request (2) the user data is written directly into the state machine via write(org.apache.ratis.proto.RaftProtos.LogEntryProto, org.apache.ratis.statemachine.TransactionContext) (3) transaction is committed via applyTransaction(TransactionContext)

For the other write requests, the transaction is directly committed via applyTransaction(TransactionContext).

There are 2 ordering operation which are enforced right now in the code, 1) WriteChunk must be executed after the CreateContainer; otherwise, WriteChunk will fail with container not found. 2) WriteChunk commit is executed after WriteChunk write. Then, WriteChunk commit and CreateContainer will be executed in the same order.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.ratis.statemachine.StateMachine

    org.apache.ratis.statemachine.StateMachine.DataApi, org.apache.ratis.statemachine.StateMachine.DataChannel, org.apache.ratis.statemachine.StateMachine.DataStream, org.apache.ratis.statemachine.StateMachine.EventApi, org.apache.ratis.statemachine.StateMachine.FollowerEventApi, org.apache.ratis.statemachine.StateMachine.LeaderEventApi, org.apache.ratis.statemachine.StateMachine.Registry

  • Field Summary

    Fields inherited from interface org.apache.ratis.statemachine.StateMachine.DataApi

    DEFAULT

    Fields inherited from interface org.apache.ratis.statemachine.StateMachine.EventApi

    DEFAULT

    Fields inherited from interface org.apache.ratis.statemachine.StateMachine.FollowerEventApi

    DEFAULT

    Fields inherited from interface org.apache.ratis.statemachine.StateMachine.LeaderEventApi

    DEFAULT

  • Constructor Summary

    Constructors
    Constructor
    Description
    ContainerStateMachine(HddsDatanodeService hddsDatanodeService, org.apache.ratis.protocol.RaftGroupId gid, ContainerDispatcher dispatcher, ContainerController containerController, List<ThreadPoolExecutor> chunkExecutors, XceiverServerRatis ratisServer, org.apache.hadoop.hdds.conf.ConfigurationSource conf, String threadNamePrefix)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    CompletableFuture<org.apache.ratis.protocol.Message>
    applyTransaction(org.apache.ratis.statemachine.TransactionContext trx)
     
    void
    buildMissingContainerSet(File snapshotFile)
     
    void
    close()
     
    void
    evictStateMachineCache()
     
    CompletableFuture<Void>
    flush(long index)
    Returns the combined future of all the writeChunks till the given log index.
    CSMMetrics
    getMetrics()
     
    org.apache.ratis.statemachine.StateMachineStorage
    getStateMachineStorage()
     
    void
    initialize(org.apache.ratis.server.RaftServer server, org.apache.ratis.protocol.RaftGroupId id, org.apache.ratis.server.storage.RaftStorage raftStorage)
     
    boolean
    isStateMachineHealthy()
     
    CompletableFuture<?>
    link(org.apache.ratis.statemachine.StateMachine.DataStream stream, org.apache.ratis.proto.RaftProtos.LogEntryProto entry)
     
    void
    notifyExtendedNoLeader(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto)
     
    void
    notifyFollowerSlowness(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto, org.apache.ratis.protocol.RaftPeer follower)
     
    void
    notifyGroupRemove()
     
    CompletableFuture<org.apache.ratis.server.protocol.TermIndex>
    notifyInstallSnapshotFromLeader(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto, org.apache.ratis.server.protocol.TermIndex firstTermIndexInLog)
     
    void
    notifyLeaderChanged(org.apache.ratis.protocol.RaftGroupMemberId groupMemberId, org.apache.ratis.protocol.RaftPeerId raftPeerId)
     
    void
    notifyLogFailed(Throwable t, org.apache.ratis.proto.RaftProtos.LogEntryProto failedEntry)
     
    void
    notifyNotLeader(Collection<org.apache.ratis.statemachine.TransactionContext> pendingEntries)
     
    void
    notifyServerShutdown(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfo, boolean allServer)
     
    void
    notifyTermIndexUpdated(long term, long index)
    Notifies the state machine about index updates because of entries which do not cause state machine update, i.e. conf entries, metadata entries
    void
    persistContainerSet(OutputStream out)
    As a part of taking snapshot with Ratis StateMachine, it will persist the existing container set in the snapshotFile.
    CompletableFuture<org.apache.ratis.protocol.Message>
    query(org.apache.ratis.protocol.Message request)
     
    CompletableFuture<org.apache.ratis.thirdparty.com.google.protobuf.ByteString>
    read(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.statemachine.TransactionContext trx)
    This method is used by the Leader to read state machine data for sending appendEntries to followers.
    static String
    smProtoToString(org.apache.ratis.protocol.RaftGroupId gid, ContainerController containerController, org.apache.ratis.proto.RaftProtos.StateMachineLogEntryProto proto)
     
    org.apache.ratis.statemachine.TransactionContext
    startTransaction(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.proto.RaftProtos.RaftPeerRole role)
    For applying log entry.
    org.apache.ratis.statemachine.TransactionContext
    startTransaction(org.apache.ratis.protocol.RaftClientRequest request)
    For the Leader to serve the given client request.
    CompletableFuture<org.apache.ratis.statemachine.StateMachine.DataStream>
    stream(org.apache.ratis.protocol.RaftClientRequest request)
     
    long
    takeSnapshot()
     
    String
    toStateMachineLogEntryString(org.apache.ratis.proto.RaftProtos.StateMachineLogEntryProto proto)
     
    CompletableFuture<Void>
    truncate(long index)
     
    CompletableFuture<org.apache.ratis.protocol.Message>
    write(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.statemachine.TransactionContext trx)
    writeStateMachineData(org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandRequestProto, long, long, long) calls are not synchronized with each other and also with applyTransaction(TransactionContext).

    Methods inherited from class org.apache.ratis.statemachine.impl.BaseStateMachine

    applyTransactionSerial, cancelTransaction, getGroupId, getId, getLastAppliedTermIndex, getLatestSnapshot, getLifeCycle, getLifeCycleState, getServer, pause, preAppendTransaction, queryStale, reinitialize, setLastAppliedTermIndex, toString, updateLastAppliedTermIndex, updateLastAppliedTermIndex

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.ratis.statemachine.StateMachine

    data, event, followerEvent, leaderEvent

    Methods inherited from interface org.apache.ratis.statemachine.StateMachine.DataApi

    read, write

    Methods inherited from interface org.apache.ratis.statemachine.StateMachine.EventApi

    notifyConfigurationChanged, notifyServerShutdown, notifySnapshotInstalled

    Methods inherited from interface org.apache.ratis.statemachine.StateMachine.LeaderEventApi

    notifyFollowerSlowness, notifyLeaderReady

  • Constructor Details

  • Method Details

    • getStateMachineStorage

      public org.apache.ratis.statemachine.StateMachineStorage getStateMachineStorage()
      Specified by:
      getStateMachineStorage in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      getStateMachineStorage in class org.apache.ratis.statemachine.impl.BaseStateMachine

    • getMetrics

      public CSMMetrics getMetrics()

    • initialize

      public void initialize(org.apache.ratis.server.RaftServer server, org.apache.ratis.protocol.RaftGroupId id, org.apache.ratis.server.storage.RaftStorage raftStorage) throws IOException
      Specified by:
      initialize in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      initialize in class org.apache.ratis.statemachine.impl.BaseStateMachine
      Throws:
      IOException

    • buildMissingContainerSet

      public void buildMissingContainerSet(File snapshotFile) throws IOException
      Throws:
      IOException

    • persistContainerSet

      public void persistContainerSet(OutputStream out) throws IOException
      As a part of taking snapshot with Ratis StateMachine, it will persist the existing container set in the snapshotFile.
      Parameters:
      out - OutputStream mapped to the Ratis snapshot file
      Throws:
      IOException

    • isStateMachineHealthy

      public boolean isStateMachineHealthy()

    • takeSnapshot

      public long takeSnapshot() throws IOException
      Specified by:
      takeSnapshot in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      takeSnapshot in class org.apache.ratis.statemachine.impl.BaseStateMachine
      Throws:
      IOException

    • startTransaction

      public org.apache.ratis.statemachine.TransactionContext startTransaction(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.proto.RaftProtos.RaftPeerRole role)
      For applying log entry.

    • startTransaction

      public org.apache.ratis.statemachine.TransactionContext startTransaction(org.apache.ratis.protocol.RaftClientRequest request) throws IOException
      For the Leader to serve the given client request.
      Specified by:
      startTransaction in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      startTransaction in class org.apache.ratis.statemachine.impl.BaseStateMachine
      Throws:
      IOException

    • stream

      public CompletableFuture<org.apache.ratis.statemachine.StateMachine.DataStream> stream(org.apache.ratis.protocol.RaftClientRequest request)

    • link

      public CompletableFuture<?> link(org.apache.ratis.statemachine.StateMachine.DataStream stream, org.apache.ratis.proto.RaftProtos.LogEntryProto entry)

    • write

      public CompletableFuture<org.apache.ratis.protocol.Message> write(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.statemachine.TransactionContext trx)
      writeStateMachineData(org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerCommandRequestProto, long, long, long) calls are not synchronized with each other and also with applyTransaction(TransactionContext).

    • query

      public CompletableFuture<org.apache.ratis.protocol.Message> query(org.apache.ratis.protocol.Message request)
      Specified by:
      query in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      query in class org.apache.ratis.statemachine.impl.BaseStateMachine

    • flush

      public CompletableFuture<Void> flush(long index)
      Returns the combined future of all the writeChunks till the given log index. The Raft log worker will wait for the stateMachineData to complete flush as well.
      Parameters:
      index - log index till which the stateMachine data needs to be flushed
      Returns:
      Combined future of all writeChunks till the log index given.

    • read

      public CompletableFuture<org.apache.ratis.thirdparty.com.google.protobuf.ByteString> read(org.apache.ratis.proto.RaftProtos.LogEntryProto entry, org.apache.ratis.statemachine.TransactionContext trx)
      This method is used by the Leader to read state machine data for sending appendEntries to followers. It will first get the data from stateMachineDataCache. If the data is not in the cache, it will read from the file by dispatching a command
      Parameters:
      trx - the transaction context, which can be null if this method is invoked after applyTransaction(TransactionContext).

    • notifyTermIndexUpdated

      public void notifyTermIndexUpdated(long term, long index)
      Notifies the state machine about index updates because of entries which do not cause state machine update, i.e. conf entries, metadata entries
      Specified by:
      notifyTermIndexUpdated in interface org.apache.ratis.statemachine.StateMachine.EventApi
      Overrides:
      notifyTermIndexUpdated in class org.apache.ratis.statemachine.impl.BaseStateMachine
      Parameters:
      term - term of the log entry
      index - index of the log entry

    • notifyServerShutdown

      public void notifyServerShutdown(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfo, boolean allServer)

    • applyTransaction

      public CompletableFuture<org.apache.ratis.protocol.Message> applyTransaction(org.apache.ratis.statemachine.TransactionContext trx)
      Specified by:
      applyTransaction in interface org.apache.ratis.statemachine.StateMachine
      Overrides:
      applyTransaction in class org.apache.ratis.statemachine.impl.BaseStateMachine

    • notifyNotLeader

      public void notifyNotLeader(Collection<org.apache.ratis.statemachine.TransactionContext> pendingEntries)

    • truncate

      public CompletableFuture<Void> truncate(long index)

    • evictStateMachineCache

      public void evictStateMachineCache()

    • notifyFollowerSlowness

      public void notifyFollowerSlowness(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto, org.apache.ratis.protocol.RaftPeer follower)

    • notifyExtendedNoLeader

      public void notifyExtendedNoLeader(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto)

    • notifyLogFailed

      public void notifyLogFailed(Throwable t, org.apache.ratis.proto.RaftProtos.LogEntryProto failedEntry)

    • notifyInstallSnapshotFromLeader

      public CompletableFuture<org.apache.ratis.server.protocol.TermIndex> notifyInstallSnapshotFromLeader(org.apache.ratis.proto.RaftProtos.RoleInfoProto roleInfoProto, org.apache.ratis.server.protocol.TermIndex firstTermIndexInLog)

    • notifyGroupRemove

      public void notifyGroupRemove()

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class org.apache.ratis.statemachine.impl.BaseStateMachine

    • notifyLeaderChanged

      public void notifyLeaderChanged(org.apache.ratis.protocol.RaftGroupMemberId groupMemberId, org.apache.ratis.protocol.RaftPeerId raftPeerId)

    • toStateMachineLogEntryString

      public String toStateMachineLogEntryString(org.apache.ratis.proto.RaftProtos.StateMachineLogEntryProto proto)

    • smProtoToString

      public static String smProtoToString(org.apache.ratis.protocol.RaftGroupId gid, ContainerController containerController, org.apache.ratis.proto.RaftProtos.StateMachineLogEntryProto proto)