Package org.apache.hadoop.ozone.om

Class KeyManagerImpl

java.lang.Object
org.apache.hadoop.ozone.om.KeyManagerImpl
All Implemented Interfaces:
OzoneManagerFS, IOzoneAcl, KeyManager
public class KeyManagerImpl extends Object implements KeyManager
Implementation of keyManager.

  • Constructor Summary

    Constructors
    Constructor
    Description
    KeyManagerImpl(OzoneManager om, ScmClient scmClient, org.apache.hadoop.hdds.conf.OzoneConfiguration conf, OMPerformanceMetrics metrics)
     
    KeyManagerImpl(OzoneManager om, ScmClient scmClient, org.apache.hadoop.ozone.om.OMMetadataManager metadataManager, org.apache.hadoop.hdds.conf.OzoneConfiguration conf, org.apache.hadoop.hdds.security.token.OzoneBlockTokenSecretManager secretManager, org.apache.hadoop.crypto.key.KeyProviderCryptoExtension kmsProvider, OMPerformanceMetrics metrics)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    checkAccess(org.apache.hadoop.ozone.security.acl.OzoneObj ozObject, org.apache.hadoop.ozone.security.acl.RequestContext context)
    Check access for given ozoneObject.
    List<org.apache.hadoop.ozone.OzoneAcl>
    getAcl(org.apache.hadoop.ozone.security.acl.OzoneObj obj)
    Returns list of ACLs for given Ozone object.
    CompactionService
    getCompactionService()
    Returns the instance of CompactionService.
    org.apache.hadoop.hdds.utils.db.TableIterator<String,? extends org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>>
    getDeletedDirEntries(String volume, String bucket)
    Returns an iterator for pending deleted directories for volume and bucket.
    List<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo>>>
    getDeletedKeyEntries(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo>,Boolean,IOException> filter, int size)
    Returns a list deleted entries from the deletedTable.
    KeyDeletingService
    getDeletingService()
    Returns the instance of Deleting Service.
    DirectoryDeletingService
    getDirDeletingService()
    Returns the instance of Directory Deleting Service.
    List<org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.ExpiredMultipartUploadsBucket>
    getExpiredMultipartUploads(Duration expireThreshold, int maxParts)
    Returns the MPU infos of up to count whose age is greater than or equal to expireThreshold.
    org.apache.hadoop.ozone.om.ExpiredOpenKeys
    getExpiredOpenKeys(Duration expireThreshold, int count, org.apache.hadoop.ozone.om.helpers.BucketLayout bucketLayout, Duration leaseThreshold)
    Returns the names of up to count open keys whose age is greater than or equal to expireThreshold.
    org.apache.hadoop.ozone.om.helpers.OzoneFileStatus
    getFileStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args)
    OzoneFS api to get file status for an entry.
    org.apache.hadoop.ozone.om.helpers.OzoneFileStatus
    getFileStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, String clientAddress)
    OzoneFS api to get file status for an entry.
    org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    getKeyInfo(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket, String clientAddress)
    Return info of an existing key to client side to access to data on datanodes.
    org.apache.hadoop.ozone.om.OMMetadataManager
    getMetadataManager()
    Returns the metadataManager.
    org.apache.hadoop.hdds.utils.BackgroundService
    getMultipartUploadCleanupService()
    Returns the instance of Multipart Upload Cleanup Service.
    Map<String,String>
    getObjectTagging(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket)
    Look up an existing key from the OM table and retrieve the tags from the key info.
    org.apache.hadoop.hdds.utils.BackgroundService
    getOpenKeyCleanupService()
    Returns the instance of Open Key Cleanup Service.
    PendingKeysDeletion
    getPendingDeletionKeys(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int count)
    Retrieves the keys that are pending deletion in a specified bucket and volume.
    PendingKeysDeletion
    getPendingDeletionKeys(org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int count)
    Retrieves pending deletion keys that match a given filter function.
    DeleteKeysResult
    getPendingDeletionSubDirs(long volumeId, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo parentInfo, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int remainingNum)
    Returns all sub directories under the given parent directory.
    DeleteKeysResult
    getPendingDeletionSubFiles(long volumeId, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo parentInfo, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int remainingNum)
    Returns all sub files under the given parent directory.
    org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo,IOException>
    getPreviousSnapshotOzoneDirInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo keyInfo)
    Returns the previous snapshot's ozone directorInfo corresponding for the object.
    org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo,IOException>
    getPreviousSnapshotOzoneDirInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo)
    Returns the previous snapshot's ozone directoryInfo corresponding for the object.
    org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmKeyInfo,IOException>
    getPreviousSnapshotOzoneKeyInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo)
    Returns the previous snapshot's ozone keyInfo corresponding for the object.
    List<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,String>>
    getRenamesKeyEntries(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,String>,Boolean,IOException> filter, int size)
    Returns a list rename entries from the snapshotRenamedTable.
    SnapshotDefragService
    getSnapshotDefragService()
    Get the SnapshotDefragService instance.
    SnapshotDeletingService
    getSnapshotDeletingService()
    Returns the instance of Snapshot Deleting service.
    SstFilteringService
    getSnapshotSstFilteringService()
    Returns the instance of Snapshot SST Filtering service.
    boolean
    isBucketFSOptimized(String volName, String buckName)
     
    boolean
    isDefragSvcEnabled()
     
    boolean
    isSstFilteringSvcEnabled()
     
    org.apache.hadoop.ozone.om.helpers.ListKeysResult
    listKeys(String volumeName, String bucketName, String startKey, String keyPrefix, int maxKeys)
    Returns a list of keys represented by OmKeyInfo in the given bucket.
    org.apache.hadoop.ozone.om.helpers.OmMultipartUploadList
    listMultipartUploads(String volumeName, String bucketName, String prefix, String keyMarker, String uploadIdMarker, int maxUploads, boolean withPagination)
     
    org.apache.hadoop.ozone.om.helpers.OmMultipartUploadListParts
    listParts(String volumeName, String bucketName, String keyName, String uploadID, int partNumberMarker, int maxParts)
    Returns list of parts of a multipart upload key.
    List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus>
    listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries)
    List the status for a file or a directory and its contents.
    List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus>
    listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries, String clientAddress)
    List the status for a file or a directory and its contents.
    List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus>
    listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries, String clientAddress, boolean allowPartialPrefixes)
    List the status for a file or a directory and its contents.
    org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    lookupFile(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, String clientAddress)
    OzoneFS api to lookup for a file.
    org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    lookupKey(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket, String clientAddress)
    Look up an existing key.
    void
    refresh(org.apache.hadoop.ozone.om.helpers.OmKeyInfo key)
    Refresh the key block location information by get latest info from SCM.
    protected void
    refreshPipeline(List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo> keyList)
    Refresh pipeline info in OM by asking SCM.
    protected Map<Long,org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline>
    refreshPipeline(Set<Long> containerIDs)
    Refresh pipeline info in OM by asking SCM.
    protected void
    refreshPipelineFromCache(org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo, boolean forceRefresh)
     
    List<? extends org.apache.hadoop.hdds.protocol.DatanodeDetails>
    sortDatanodes(List<? extends org.apache.hadoop.hdds.protocol.DatanodeDetails> nodes, String clientMachine)
     
    void
    start(org.apache.hadoop.hdds.conf.OzoneConfiguration configuration)
    Start key manager.
    void
    startSnapshotDefragService(org.apache.hadoop.hdds.conf.OzoneConfiguration conf)
    Start the snapshot defrag service if interval is not set to disabled value.
    void
    startSnapshotSstFilteringService(org.apache.hadoop.hdds.conf.OzoneConfiguration conf)
    Start the snapshot SST filtering service if interval is not set to disabled value.
    void
    stop()
    Stop key manager.
    void
    stopSnapshotDefragService()
    Stop the snapshot defrag service if it is running.
    void
    stopSnapshotSstFilteringService()
    Stop the snapshot SST filtering service if it is running.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.ozone.om.KeyManager

    getDeletedDirEntries, getDeletedDirEntries

  • Constructor Details

    • KeyManagerImpl

      public KeyManagerImpl(OzoneManager om, ScmClient scmClient, org.apache.hadoop.hdds.conf.OzoneConfiguration conf, OMPerformanceMetrics metrics)

    • KeyManagerImpl

      public KeyManagerImpl(OzoneManager om, ScmClient scmClient, org.apache.hadoop.ozone.om.OMMetadataManager metadataManager, org.apache.hadoop.hdds.conf.OzoneConfiguration conf, org.apache.hadoop.hdds.security.token.OzoneBlockTokenSecretManager secretManager, org.apache.hadoop.crypto.key.KeyProviderCryptoExtension kmsProvider, OMPerformanceMetrics metrics)

  • Method Details

    • start

      public void start(org.apache.hadoop.hdds.conf.OzoneConfiguration configuration)
      Description copied from interface: KeyManager
      Start key manager.
      Specified by:
      start in interface KeyManager
      Parameters:
      configuration -

    • startSnapshotSstFilteringService

      public void startSnapshotSstFilteringService(org.apache.hadoop.hdds.conf.OzoneConfiguration conf)
      Start the snapshot SST filtering service if interval is not set to disabled value.
      Parameters:
      conf -

    • stopSnapshotSstFilteringService

      public void stopSnapshotSstFilteringService()
      Stop the snapshot SST filtering service if it is running.

    • startSnapshotDefragService

      public void startSnapshotDefragService(org.apache.hadoop.hdds.conf.OzoneConfiguration conf)
      Start the snapshot defrag service if interval is not set to disabled value.
      Parameters:
      conf -

    • stopSnapshotDefragService

      public void stopSnapshotDefragService()
      Stop the snapshot defrag service if it is running.

    • stop

      public void stop()
      Description copied from interface: KeyManager
      Stop key manager.
      Specified by:
      stop in interface KeyManager

    • getSnapshotDefragService

      public SnapshotDefragService getSnapshotDefragService()
      Get the SnapshotDefragService instance.
      Specified by:
      getSnapshotDefragService in interface KeyManager
      Returns:
      SnapshotDefragService instance, or null if not initialized

    • lookupKey

      public org.apache.hadoop.ozone.om.helpers.OmKeyInfo lookupKey(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket, String clientAddress) throws IOException
      Description copied from interface: KeyManager
      Look up an existing key. Return the info of the key to client side, which DistributedStorageHandler will use to access the data on datanode.
      Specified by:
      lookupKey in interface KeyManager
      Parameters:
      args - the args of the key provided by client.
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      Returns:
      a OmKeyInfo instance client uses to talk to container.
      Throws:
      IOException

    • refreshPipeline

      protected void refreshPipeline(List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo> keyList) throws IOException
      Refresh pipeline info in OM by asking SCM.
      Parameters:
      keyList - a list of OmKeyInfo
      Throws:
      IOException

    • refreshPipeline

      protected Map<Long,org.apache.hadoop.hdds.scm.container.common.helpers.ContainerWithPipeline> refreshPipeline(Set<Long> containerIDs) throws IOException
      Refresh pipeline info in OM by asking SCM.
      Parameters:
      containerIDs - a set of containerIDs
      Throws:
      IOException

    • listKeys

      public org.apache.hadoop.ozone.om.helpers.ListKeysResult listKeys(String volumeName, String bucketName, String startKey, String keyPrefix, int maxKeys) throws IOException
      Description copied from interface: KeyManager
      Returns a list of keys represented by OmKeyInfo in the given bucket.
      Specified by:
      listKeys in interface KeyManager
      Parameters:
      volumeName - the name of the volume.
      bucketName - the name of the bucket.
      startKey - the start key name, only the keys whose name is after this value will be included in the result. This key is excluded from the result.
      keyPrefix - key name prefix, only the keys whose name has this prefix will be included in the result.
      maxKeys - the maximum number of keys to return. It ensures the size of the result will not exceed this limit.
      Returns:
      a list of keys.
      Throws:
      IOException

    • getPendingDeletionKeys

      public PendingKeysDeletion getPendingDeletionKeys(org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int count) throws IOException
      Description copied from interface: KeyManager
      Retrieves pending deletion keys that match a given filter function.
      Specified by:
      getPendingDeletionKeys in interface KeyManager
      Parameters:
      filter - a functional interface specifying the filter condition to apply to the keys. It takes a KeyValue pair containing a string key and an OmKeyInfo object, and returns a boolean value indicating whether the key meets the filter criteria.
      count - the maximum number of keys to retrieve.
      Returns:
      a PendingKeysDeletion object containing the keys that satisfy the filter criteria, up to the specified count.
      Throws:
      IOException - if an I/O error occurs while fetching the keys.

    • getPendingDeletionKeys

      public PendingKeysDeletion getPendingDeletionKeys(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int count) throws IOException
      Description copied from interface: KeyManager
      Retrieves the keys that are pending deletion in a specified bucket and volume.
      Specified by:
      getPendingDeletionKeys in interface KeyManager
      Parameters:
      volume - the name of the volume that contains the bucket.
      bucket - the name of the bucket within the volume where keys are located.
      startKey - the key from which to start retrieving pending deletions.
      filter - a filter function to determine which keys should be included in the pending deletion list.
      count - the maximum number of keys to retrieve that are pending deletion.
      Returns:
      a PendingKeysDeletion object containing the list of keys pending deletion based on the specified parameters.
      Throws:
      IOException - if an I/O error occurs during the operation.

    • getRenamesKeyEntries

      public List<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,String>> getRenamesKeyEntries(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,String>,Boolean,IOException> filter, int size) throws IOException
      Description copied from interface: KeyManager
      Returns a list rename entries from the snapshotRenamedTable.
      Specified by:
      getRenamesKeyEntries in interface KeyManager
      Parameters:
      filter - filter to apply on the entries.
      size - max number of keys to return.
      Returns:
      a Pair of list of Table.KeyValue representing the keys in the underlying metadataManager.
      Throws:
      IOException

    • getPreviousSnapshotOzoneDirInfo

      public org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo,IOException> getPreviousSnapshotOzoneDirInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo keyInfo) throws IOException
      Description copied from interface: KeyManager
      Returns the previous snapshot's ozone directorInfo corresponding for the object.
      Specified by:
      getPreviousSnapshotOzoneDirInfo in interface KeyManager
      Throws:
      IOException

    • getPreviousSnapshotOzoneDirInfo

      public org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo,IOException> getPreviousSnapshotOzoneDirInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo) throws IOException
      Description copied from interface: KeyManager
      Returns the previous snapshot's ozone directoryInfo corresponding for the object.
      Specified by:
      getPreviousSnapshotOzoneDirInfo in interface KeyManager
      Throws:
      IOException

    • getPreviousSnapshotOzoneKeyInfo

      public org.apache.ratis.util.function.CheckedFunction<KeyManager,org.apache.hadoop.ozone.om.helpers.OmKeyInfo,IOException> getPreviousSnapshotOzoneKeyInfo(long volumeId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo) throws IOException
      Description copied from interface: KeyManager
      Returns the previous snapshot's ozone keyInfo corresponding for the object.
      Specified by:
      getPreviousSnapshotOzoneKeyInfo in interface KeyManager
      Throws:
      IOException

    • getDeletedKeyEntries

      public List<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo>>> getDeletedKeyEntries(String volume, String bucket, String startKey, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo>,Boolean,IOException> filter, int size) throws IOException
      Description copied from interface: KeyManager
      Returns a list deleted entries from the deletedTable.
      Specified by:
      getDeletedKeyEntries in interface KeyManager
      Parameters:
      filter - filter to apply on the entries.
      size - max number of keys to return.
      Returns:
      a Pair of list of Table.KeyValue representing the keys in the underlying metadataManager.
      Throws:
      IOException

    • getExpiredOpenKeys

      public org.apache.hadoop.ozone.om.ExpiredOpenKeys getExpiredOpenKeys(Duration expireThreshold, int count, org.apache.hadoop.ozone.om.helpers.BucketLayout bucketLayout, Duration leaseThreshold) throws IOException
      Description copied from interface: KeyManager
      Returns the names of up to count open keys whose age is greater than or equal to expireThreshold.
      Specified by:
      getExpiredOpenKeys in interface KeyManager
      Parameters:
      expireThreshold - The threshold of open key expiration age.
      count - The maximum number of expired open keys to return.
      bucketLayout - The type of open keys to get (e.g. DEFAULT or FSO).
      leaseThreshold - The threshold of hsync key.
      Returns:
      the expired open keys.
      Throws:
      IOException

    • getExpiredMultipartUploads

      public List<org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.ExpiredMultipartUploadsBucket> getExpiredMultipartUploads(Duration expireThreshold, int maxParts) throws IOException
      Description copied from interface: KeyManager
      Returns the MPU infos of up to count whose age is greater than or equal to expireThreshold.
      Specified by:
      getExpiredMultipartUploads in interface KeyManager
      Parameters:
      expireThreshold - The threshold of expired multipart uploads to return.
      maxParts - The threshold of number of MPU parts to return. This is a soft limit, which means if the last MPU has parts that exceed this limit, it will be included in the returned expired MPUs. This is to handle situation where MPU that has more parts than maxParts never get deleted.
      Returns:
      a List of the expired OzoneManagerProtocolProtos.ExpiredMultipartUploadsBucket, the expired multipart uploads, grouped by volume and bucket.
      Throws:
      IOException

    • getObjectTagging

      public Map<String,String> getObjectTagging(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket) throws IOException
      Description copied from interface: KeyManager
      Look up an existing key from the OM table and retrieve the tags from the key info.
      Specified by:
      getObjectTagging in interface KeyManager
      Parameters:
      args - the args of the key provided by client.
      bucket - the resolved parent bucket of the key.
      Returns:
      Map of the tag set associated with the key.
      Throws:
      IOException

    • getMetadataManager

      public org.apache.hadoop.ozone.om.OMMetadataManager getMetadataManager()
      Description copied from interface: KeyManager
      Returns the metadataManager.
      Specified by:
      getMetadataManager in interface KeyManager
      Returns:
      OMMetadataManager.

    • getDeletingService

      public KeyDeletingService getDeletingService()
      Description copied from interface: KeyManager
      Returns the instance of Deleting Service.
      Specified by:
      getDeletingService in interface KeyManager
      Returns:
      Background service.

    • getDirDeletingService

      public DirectoryDeletingService getDirDeletingService()
      Description copied from interface: KeyManager
      Returns the instance of Directory Deleting Service.
      Specified by:
      getDirDeletingService in interface KeyManager
      Returns:
      Background service.

    • getOpenKeyCleanupService

      public org.apache.hadoop.hdds.utils.BackgroundService getOpenKeyCleanupService()
      Description copied from interface: KeyManager
      Returns the instance of Open Key Cleanup Service.
      Specified by:
      getOpenKeyCleanupService in interface KeyManager
      Returns:
      Background service.

    • getMultipartUploadCleanupService

      public org.apache.hadoop.hdds.utils.BackgroundService getMultipartUploadCleanupService()
      Description copied from interface: KeyManager
      Returns the instance of Multipart Upload Cleanup Service.
      Specified by:
      getMultipartUploadCleanupService in interface KeyManager
      Returns:
      Background service.

    • getSnapshotSstFilteringService

      public SstFilteringService getSnapshotSstFilteringService()
      Description copied from interface: KeyManager
      Returns the instance of Snapshot SST Filtering service.
      Specified by:
      getSnapshotSstFilteringService in interface KeyManager
      Returns:
      Background service.

    • getSnapshotDeletingService

      public SnapshotDeletingService getSnapshotDeletingService()
      Description copied from interface: KeyManager
      Returns the instance of Snapshot Deleting service.
      Specified by:
      getSnapshotDeletingService in interface KeyManager
      Returns:
      Background service.

    • getCompactionService

      public CompactionService getCompactionService()
      Description copied from interface: KeyManager
      Returns the instance of CompactionService.
      Specified by:
      getCompactionService in interface KeyManager
      Returns:
      BackgroundService

    • isSstFilteringSvcEnabled

      public boolean isSstFilteringSvcEnabled()

    • isDefragSvcEnabled

      public boolean isDefragSvcEnabled()

    • listMultipartUploads

      public org.apache.hadoop.ozone.om.helpers.OmMultipartUploadList listMultipartUploads(String volumeName, String bucketName, String prefix, String keyMarker, String uploadIdMarker, int maxUploads, boolean withPagination) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Specified by:
      listMultipartUploads in interface KeyManager
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • listParts

      public org.apache.hadoop.ozone.om.helpers.OmMultipartUploadListParts listParts(String volumeName, String bucketName, String keyName, String uploadID, int partNumberMarker, int maxParts) throws IOException
      Description copied from interface: KeyManager
      Returns list of parts of a multipart upload key.
      Specified by:
      listParts in interface KeyManager
      Parameters:
      volumeName -
      bucketName -
      keyName -
      uploadID -
      partNumberMarker -
      maxParts -
      Returns:
      OmMultipartUploadListParts
      Throws:
      IOException

    • getAcl

      public List<org.apache.hadoop.ozone.OzoneAcl> getAcl(org.apache.hadoop.ozone.security.acl.OzoneObj obj) throws IOException
      Description copied from interface: IOzoneAcl
      Returns list of ACLs for given Ozone object.
      Specified by:
      getAcl in interface IOzoneAcl
      Parameters:
      obj - Ozone object.
      Throws:
      IOException - if there is error.

    • checkAccess

      public boolean checkAccess(org.apache.hadoop.ozone.security.acl.OzoneObj ozObject, org.apache.hadoop.ozone.security.acl.RequestContext context) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Check access for given ozoneObject.
      Specified by:
      checkAccess in interface IOzoneAcl
      Parameters:
      ozObject - object for which access needs to be checked.
      context - Context object encapsulating all user related information.
      Returns:
      true if user has access else false.
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • getFileStatus

      public org.apache.hadoop.ozone.om.helpers.OzoneFileStatus getFileStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args) throws IOException
      OzoneFS api to get file status for an entry.
      Specified by:
      getFileStatus in interface OzoneManagerFS
      Parameters:
      args - Key args
      Returns:
      file status.
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException - if file does not exist if bucket does not exist if volume does not exist
      IOException - if there is error in the db invalid arguments

    • getFileStatus

      public org.apache.hadoop.ozone.om.helpers.OzoneFileStatus getFileStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, String clientAddress) throws IOException
      OzoneFS api to get file status for an entry.
      Specified by:
      getFileStatus in interface OzoneManagerFS
      Parameters:
      args - Key args
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      Returns:
      file status.
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException - if file does not exist if bucket does not exist if volume does not exist
      IOException - if there is error in the db invalid arguments

    • lookupFile

      public org.apache.hadoop.ozone.om.helpers.OmKeyInfo lookupFile(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, String clientAddress) throws IOException
      OzoneFS api to lookup for a file.
      Specified by:
      lookupFile in interface OzoneManagerFS
      Parameters:
      args - Key args
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      Returns:
      a OmKeyInfo instance client uses to talk to container.
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException - if given key is not found or it is not a file if bucket does not exist
      IOException - if there is error in the db invalid arguments

    • refresh

      public void refresh(org.apache.hadoop.ozone.om.helpers.OmKeyInfo key) throws IOException
      Refresh the key block location information by get latest info from SCM.
      Specified by:
      refresh in interface KeyManager
      Parameters:
      key -
      Throws:
      IOException

    • listStatus

      public List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus> listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries) throws IOException
      List the status for a file or a directory and its contents.
      Specified by:
      listStatus in interface OzoneManagerFS
      Parameters:
      args - Key args
      recursive - For a directory if true all the descendants of a particular directory are listed
      startKey - Key from which listing needs to start. If startKey exists its status is included in the final list.
      numEntries - Number of entries to list from the start key
      Returns:
      list of file status
      Throws:
      IOException - if file or bucket or volume does not exist

    • listStatus

      public List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus> listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries, String clientAddress) throws IOException
      Description copied from interface: OzoneManagerFS
      List the status for a file or a directory and its contents.
      Specified by:
      listStatus in interface OzoneManagerFS
      Parameters:
      args - the args of the key provided by client.
      recursive - For a directory if true all the descendants of a particular directory are listed
      startKey - Key from which listing needs to start. If startKey exists its status is included in the final list.
      numEntries - Number of entries to list from the start key
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      Returns:
      list of file status
      Throws:
      IOException - if file or bucket or volume does not exist

    • listStatus

      public List<org.apache.hadoop.ozone.om.helpers.OzoneFileStatus> listStatus(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, boolean recursive, String startKey, long numEntries, String clientAddress, boolean allowPartialPrefixes) throws IOException
      List the status for a file or a directory and its contents.
      Specified by:
      listStatus in interface OzoneManagerFS
      Parameters:
      args - Key args
      recursive - For a directory if true all the descendants of a particular directory are listed
      startKey - Key from which listing needs to start. If startKey exists its status is included in the final list.
      numEntries - Number of entries to list from the start key
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      allowPartialPrefixes - if partial prefixes should be allowed, this is needed in context of ListKeys
      Returns:
      list of file status
      Throws:
      IOException - if file or bucket or volume does not exist

    • sortDatanodes

      public List<? extends org.apache.hadoop.hdds.protocol.DatanodeDetails> sortDatanodes(List<? extends org.apache.hadoop.hdds.protocol.DatanodeDetails> nodes, String clientMachine)

    • getDeletedDirEntries

      public org.apache.hadoop.hdds.utils.db.TableIterator<String,? extends org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>> getDeletedDirEntries(String volume, String bucket) throws IOException
      Description copied from interface: KeyManager
      Returns an iterator for pending deleted directories for volume and bucket.
      Specified by:
      getDeletedDirEntries in interface KeyManager
      Throws:
      IOException

    • getPendingDeletionSubDirs

      public DeleteKeysResult getPendingDeletionSubDirs(long volumeId, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo parentInfo, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int remainingNum) throws IOException
      Description copied from interface: KeyManager
      Returns all sub directories under the given parent directory.
      Specified by:
      getPendingDeletionSubDirs in interface KeyManager
      Parameters:
      parentInfo -
      Returns:
      list of dirs
      Throws:
      IOException

    • getPendingDeletionSubFiles

      public DeleteKeysResult getPendingDeletionSubFiles(long volumeId, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo parentInfo, org.apache.ratis.util.function.CheckedFunction<org.apache.hadoop.hdds.utils.db.Table.KeyValue<String,org.apache.hadoop.ozone.om.helpers.OmKeyInfo>,Boolean,IOException> filter, int remainingNum) throws IOException
      Description copied from interface: KeyManager
      Returns all sub files under the given parent directory.
      Specified by:
      getPendingDeletionSubFiles in interface KeyManager
      Parameters:
      parentInfo -
      Returns:
      list of files
      Throws:
      IOException

    • isBucketFSOptimized

      public boolean isBucketFSOptimized(String volName, String buckName) throws IOException
      Throws:
      IOException

    • getKeyInfo

      public org.apache.hadoop.ozone.om.helpers.OmKeyInfo getKeyInfo(org.apache.hadoop.ozone.om.helpers.OmKeyArgs args, ResolvedBucket bucket, String clientAddress) throws IOException
      Description copied from interface: KeyManager
      Return info of an existing key to client side to access to data on datanodes.
      Specified by:
      getKeyInfo in interface KeyManager
      Parameters:
      args - the args of the key provided by client.
      clientAddress - a hint to key manager, order the datanode in returned pipeline by distance between client and datanode.
      Returns:
      a OmKeyInfo instance client uses to talk to container.
      Throws:
      IOException

    • refreshPipelineFromCache

      protected void refreshPipelineFromCache(org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo, boolean forceRefresh) throws IOException
      Throws:
      IOException