Package org.apache.hadoop.ozone.om.request.key

Class OMKeyRequest

java.lang.Object
org.apache.hadoop.ozone.om.request.OMClientRequest
org.apache.hadoop.ozone.om.request.key.OMKeyRequest
All Implemented Interfaces:
RequestAuditor
Direct Known Subclasses:
OMAllocateBlockRequest, OMDirectoriesPurgeRequestWithFSO, OMDirectoryCreateRequest, OMFileCreateRequest, OMKeyCommitRequest, OMKeyCreateRequest, OMKeyDeleteRequest, OMKeyPurgeRequest, OMKeyRenameRequest, OMKeysDeleteRequest, OMKeySetTimesRequest, OMKeysRenameRequest, OMOpenKeysDeleteRequest, OMRecoverLeaseRequest, S3DeleteObjectTaggingRequest, S3ExpiredMultipartUploadsAbortRequest, S3InitiateMultipartUploadRequest, S3MultipartUploadAbortRequest, S3MultipartUploadCommitPartRequest, S3MultipartUploadCompleteRequest, S3PutObjectTaggingRequest
public abstract class OMKeyRequest extends OMClientRequest
Interface for key write requests.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.hadoop.ozone.om.request.OMClientRequest

    OMClientRequest.Result

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected static final org.slf4j.Logger
    LOG
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    OMKeyRequest(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest omRequest)
     
    OMKeyRequest(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest omRequest, org.apache.hadoop.ozone.om.helpers.BucketLayout bucketLayoutArg)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected static Map<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo>
    addKeyInfoToDeleteMap(OzoneManager om, long trxnLogIndex, String ozoneKey, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo, Map<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo> deleteMap)
     
    protected List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo>
    allocateBlock(ScmClient scmClient, org.apache.hadoop.hdds.security.token.OzoneBlockTokenSecretManager secretManager, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.hdds.scm.container.common.helpers.ExcludeList excludeList, long requestedSize, long scmBlockSize, int preallocateBlocksMax, boolean grpcBlockTokenEnabled, String serviceID, OMMetrics omMetrics, boolean shouldSortDatanodes, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.UserInfo userInfo)
    This methods avoids multiple rpc calls to SCM by allocating multiple blocks in one rpc call.
    protected void
    checkBucketAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType)
    Check Acls for the ozone bucket.
    protected void
    checkBucketQuotaInBytes(org.apache.hadoop.ozone.om.OMMetadataManager metadataManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, long allocateSize)
    Check bucket quota in bytes.
    protected void
    checkBucketQuotaInNamespace(org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, long allocatedNamespace)
    Check namespace quota.
    protected boolean
    checkDirectoryAlreadyExists(String volumeName, String bucketName, String keyName, org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager)
    Check directory exists.
    protected void
    checkKeyAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, org.apache.hadoop.ozone.security.acl.OzoneObj.ResourceType resourceType)
    Check Acls for the ozone key.
    protected void
    checkKeyAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, org.apache.hadoop.ozone.security.acl.OzoneObj.ResourceType resourceType, String volumeOwner)
    Check Acls for the ozone key with volumeOwner.
    protected void
    checkKeyAclsInOpenKeyTable(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, long clientId)
    Check ACLs for Ozone Key in OpenKey table if ozone native authorizer is enabled.
    protected org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo
    createDirectoryInfoWithACL(String dirName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, long parentObjectId, long transactionIndex, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, org.apache.hadoop.ozone.om.OmConfig config)
    Fill in a DirectoryInfo for a new directory entry in OM database.
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    createDirectoryKeyInfoWithACL(String keyName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionIndex, org.apache.hadoop.hdds.client.ReplicationConfig serverDefaultReplConfig, org.apache.hadoop.ozone.om.OmConfig config)
    fill in a KeyInfo for a new directory entry in OM database.
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    createFileInfo(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, long size, org.apache.hadoop.fs.FileEncryptionInfo encInfo, PrefixManager prefixManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.ozone.om.OmConfig config)
    Create OmKeyInfo object.
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo.Builder
    dirKeyInfoBuilderNoACL(String keyName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, org.apache.hadoop.hdds.client.ReplicationConfig serverDefaultReplConfig)
     
    protected org.apache.commons.lang3.tuple.Pair<Map<org.apache.hadoop.ozone.om.helpers.OmKeyInfo,List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo>>,Integer>
    filterOutBlocksStillInUse(org.apache.hadoop.ozone.om.helpers.OmKeyInfo referenceKey, org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo keysToBeFiltered)
    Remove blocks in-place from keysToBeFiltered that exist in referenceKey.
    protected void
    generateRequiredEncryptionInfo(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs, OzoneManager ozoneManager)
    Generate EncryptionInfo and set in to newKeyArgs.
    protected List<org.apache.hadoop.ozone.OzoneAcl>
    getAclsForDir(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, org.apache.hadoop.ozone.om.OmConfig config)
    Inherit parent DEFAULT acls and generate its own ACCESS acls.
    protected List<org.apache.hadoop.ozone.OzoneAcl>
    getAclsForKey(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, PrefixManager prefixManager, org.apache.hadoop.ozone.om.OmConfig config)
     
    protected List<org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo>
    getAllMissingParentDirInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfoWithFSO pathInfo, long trxnLogIndex)
    Construct OmDirectoryInfo for every parent directory in missing list.
    protected List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo>
    getAllParentInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, List<String> missingParents, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, long trxnLogIndex)
    Construct OmKeyInfo for every parent directory in missing list.
    static org.apache.hadoop.ozone.om.helpers.OmBucketInfo
    getBucketInfo(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volume, String bucket)
    Return bucket info for the specified bucket.
    org.apache.hadoop.ozone.om.helpers.BucketLayout
    getBucketLayout()
     
    protected String
    getDBMultipartOpenKey(String volumeName, String bucketName, String keyName, String uploadID, org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager)
    Returns the DB key name of a multipart open key in OM metadata store.
    protected static Optional<org.apache.hadoop.fs.FileEncryptionInfo>
    getFileEncryptionInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo)
     
    protected org.apache.hadoop.fs.FileEncryptionInfo
    getFileEncryptionInfo(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs)
    Get FileEncryptionInfoProto from KeyArgs.
    protected void
    getFileEncryptionInfoForMpuKey(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs, OzoneManager ozoneManager)
     
    protected org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo
    getOldVersionsToCleanUp(org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyToDelete, long bucketId, long trxnLogIndex)
    Prepare key for deletion service on overwrite.
    protected OzoneLockStrategy
    getOzoneLockStrategy(OzoneManager ozoneManager)
     
    protected org.apache.hadoop.security.UserGroupInformation
    getRemoteUser()
     
    protected String
    getVolumeOwner(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volumeName)
     
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    prepareFileInfo(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmKeyInfo dbKeyInfo, long size, List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, org.apache.hadoop.fs.FileEncryptionInfo encInfo, PrefixManager prefixManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.ozone.om.OmConfig config)
    Prepare OmKeyInfo which will be persisted to openKeyTable.
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    prepareKeyInfo(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmKeyInfo dbKeyInfo, long size, List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, org.apache.hadoop.fs.FileEncryptionInfo encInfo, PrefixManager prefixManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.ozone.om.OmConfig config)
    Prepare OmKeyInfo which will be persisted to openKeyTable.
    protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs
    resolveBucketAndCheckKeyAcls(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType)
     
    protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs
    resolveBucketAndCheckKeyAclsWithFSO(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType)
     
    protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs
    resolveBucketAndCheckOpenKeyAcls(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, long clientId)
     
    protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs
    resolveBucketLink(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs)
     
    protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs
    resolveBucketLink(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, Map<String,String> auditMap)
     
    static long
    sumBlockLengths(org.apache.hadoop.ozone.om.helpers.OmKeyInfo omKeyInfo)
     
    void
    validateBucketAndVolume(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volumeName, String bucketName)
    Validate bucket and volume exists or not.
    protected void
    validateEncryptionKeyInfo(org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs)
     
    protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo
    wrapUncommittedBlocksAsPseudoKey(List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> uncommitted, org.apache.hadoop.ozone.om.helpers.OmKeyInfo omKeyInfo)
    Wrap the uncommitted blocks as pseudoKeyInfo.

    Methods inherited from class org.apache.hadoop.ozone.om.request.OMClientRequest

    buildAuditMessage, buildVolumeAuditMap, checkAcls, checkAcls, checkAcls, checkACLsWithFSO, createErrorOMResponse, createUGI, createUGIForApi, getAuditBuilder, getHostName, getOmLockDetails, getOmRequest, getRemoteAddress, getUserIfNotExists, getUserInfo, handleRequestFailure, isValidKeyPath, markForAudit, mergeOmLockDetails, normalizeKeyPath, preExecute, setUGI, validateAndNormalizeKey, validateAndNormalizeKey, validateAndNormalizeKey, validateAndUpdateCache, validateAndUpdateCache

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.hadoop.ozone.om.request.RequestAuditor

    buildKeyArgsAuditMap, buildLightKeyArgsAuditMap

  • Field Details

    • LOG

      protected static final org.slf4j.Logger LOG

  • Constructor Details

    • OMKeyRequest

      public OMKeyRequest(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest omRequest)

    • OMKeyRequest

      public OMKeyRequest(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.OMRequest omRequest, org.apache.hadoop.ozone.om.helpers.BucketLayout bucketLayoutArg)

  • Method Details

    • getBucketLayout

      public org.apache.hadoop.ozone.om.helpers.BucketLayout getBucketLayout()

    • resolveBucketLink

      protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs resolveBucketLink(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, Map<String,String> auditMap) throws IOException
      Throws:
      IOException

    • resolveBucketLink

      protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs resolveBucketLink(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs) throws IOException
      Throws:
      IOException

    • resolveBucketAndCheckKeyAcls

      protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs resolveBucketAndCheckKeyAcls(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType) throws IOException
      Throws:
      IOException

    • resolveBucketAndCheckKeyAclsWithFSO

      protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs resolveBucketAndCheckKeyAclsWithFSO(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType) throws IOException
      Throws:
      IOException

    • resolveBucketAndCheckOpenKeyAcls

      protected org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs resolveBucketAndCheckOpenKeyAcls(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, OzoneManager ozoneManager, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, long clientId) throws IOException
      Throws:
      IOException

    • allocateBlock

      protected List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> allocateBlock(ScmClient scmClient, org.apache.hadoop.hdds.security.token.OzoneBlockTokenSecretManager secretManager, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.hdds.scm.container.common.helpers.ExcludeList excludeList, long requestedSize, long scmBlockSize, int preallocateBlocksMax, boolean grpcBlockTokenEnabled, String serviceID, OMMetrics omMetrics, boolean shouldSortDatanodes, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.UserInfo userInfo) throws IOException
      This methods avoids multiple rpc calls to SCM by allocating multiple blocks in one rpc call.
      Throws:
      IOException

    • getRemoteUser

      protected org.apache.hadoop.security.UserGroupInformation getRemoteUser() throws IOException
      Throws:
      IOException

    • validateBucketAndVolume

      public void validateBucketAndVolume(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volumeName, String bucketName) throws IOException
      Validate bucket and volume exists or not.
      Parameters:
      omMetadataManager -
      volumeName -
      bucketName -
      Throws:
      IOException

    • getVolumeOwner

      protected String getVolumeOwner(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volumeName) throws IOException
      Throws:
      IOException

    • getFileEncryptionInfo

      protected static Optional<org.apache.hadoop.fs.FileEncryptionInfo> getFileEncryptionInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo) throws IOException
      Throws:
      IOException

    • getAclsForKey

      protected List<org.apache.hadoop.ozone.OzoneAcl> getAclsForKey(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, PrefixManager prefixManager, org.apache.hadoop.ozone.om.OmConfig config) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • getAclsForDir

      protected List<org.apache.hadoop.ozone.OzoneAcl> getAclsForDir(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, org.apache.hadoop.ozone.om.OmConfig config) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Inherit parent DEFAULT acls and generate its own ACCESS acls.
      Parameters:
      keyArgs -
      bucketInfo -
      omPathInfo -
      config -
      Returns:
      Acls which inherited parent DEFAULT and keyArgs ACCESS acls.
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • getAllMissingParentDirInfo

      protected List<org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo> getAllMissingParentDirInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfoWithFSO pathInfo, long trxnLogIndex) throws IOException
      Construct OmDirectoryInfo for every parent directory in missing list.
      Parameters:
      keyArgs - key arguments
      pathInfo - list of parent directories to be created and its ACLs
      trxnLogIndex - transaction log index id
      Returns:
      list of missing parent directories
      Throws:
      IOException - DB failure

    • getAllParentInfo

      protected List<org.apache.hadoop.ozone.om.helpers.OmKeyInfo> getAllParentInfo(OzoneManager ozoneManager, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, List<String> missingParents, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, long trxnLogIndex) throws IOException
      Construct OmKeyInfo for every parent directory in missing list.
      Parameters:
      ozoneManager -
      keyArgs -
      missingParents - list of parent directories to be created
      bucketInfo -
      omPathInfo -
      trxnLogIndex -
      Returns:
      List<OmKeyInfo>
      Throws:
      IOException

    • createDirectoryInfoWithACL

      protected org.apache.hadoop.ozone.om.helpers.OmDirectoryInfo createDirectoryInfoWithACL(String dirName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, long parentObjectId, long transactionIndex, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, org.apache.hadoop.ozone.om.OmConfig config) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Fill in a DirectoryInfo for a new directory entry in OM database.
      Parameters:
      dirName -
      keyArgs -
      objectId -
      parentObjectId -
      bucketInfo -
      omPathInfo -
      config -
      Returns:
      the OmDirectoryInfo structure
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • createDirectoryKeyInfoWithACL

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo createDirectoryKeyInfoWithACL(String keyName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionIndex, org.apache.hadoop.hdds.client.ReplicationConfig serverDefaultReplConfig, org.apache.hadoop.ozone.om.OmConfig config) throws org.apache.hadoop.ozone.om.exceptions.OMException
      fill in a KeyInfo for a new directory entry in OM database. without initializing ACLs from the KeyArgs - used for intermediate directories which get created internally/recursively during file and directory create.
      Parameters:
      keyName -
      keyArgs -
      objectId -
      bucketInfo -
      omPathInfo -
      transactionIndex -
      serverDefaultReplConfig -
      config -
      Returns:
      the OmKeyInfo structure
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • dirKeyInfoBuilderNoACL

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo.Builder dirKeyInfoBuilderNoACL(String keyName, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, long objectId, org.apache.hadoop.hdds.client.ReplicationConfig serverDefaultReplConfig)

    • checkBucketAcls

      protected void checkBucketAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType) throws IOException
      Check Acls for the ozone bucket.
      Parameters:
      ozoneManager -
      volume -
      bucket -
      key -
      Throws:
      IOException

    • checkKeyAcls

      protected void checkKeyAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, org.apache.hadoop.ozone.security.acl.OzoneObj.ResourceType resourceType) throws IOException
      Check Acls for the ozone key.
      Parameters:
      ozoneManager -
      volume -
      bucket -
      key -
      aclType -
      resourceType -
      Throws:
      IOException

    • checkKeyAcls

      protected void checkKeyAcls(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, org.apache.hadoop.ozone.security.acl.OzoneObj.ResourceType resourceType, String volumeOwner) throws IOException
      Check Acls for the ozone key with volumeOwner.
      Parameters:
      ozoneManager -
      volume -
      bucket -
      key -
      aclType -
      resourceType -
      Throws:
      IOException

    • checkKeyAclsInOpenKeyTable

      protected void checkKeyAclsInOpenKeyTable(OzoneManager ozoneManager, String volume, String bucket, String key, org.apache.hadoop.ozone.security.acl.IAccessAuthorizer.ACLType aclType, long clientId) throws IOException
      Check ACLs for Ozone Key in OpenKey table if ozone native authorizer is enabled.
      Parameters:
      ozoneManager -
      volume -
      bucket -
      key -
      aclType -
      clientId -
      Throws:
      IOException

    • generateRequiredEncryptionInfo

      protected void generateRequiredEncryptionInfo(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs, OzoneManager ozoneManager) throws IOException
      Generate EncryptionInfo and set in to newKeyArgs.
      Parameters:
      keyArgs -
      newKeyArgs -
      ozoneManager -
      Throws:
      IOException

    • getFileEncryptionInfoForMpuKey

      protected void getFileEncryptionInfoForMpuKey(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs.Builder newKeyArgs, OzoneManager ozoneManager) throws IOException
      Throws:
      IOException

    • getFileEncryptionInfo

      protected org.apache.hadoop.fs.FileEncryptionInfo getFileEncryptionInfo(org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs)
      Get FileEncryptionInfoProto from KeyArgs.
      Parameters:
      keyArgs -
      Returns:
      FileEncryptionInfo

    • checkBucketQuotaInBytes

      protected void checkBucketQuotaInBytes(org.apache.hadoop.ozone.om.OMMetadataManager metadataManager, org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, long allocateSize) throws IOException
      Check bucket quota in bytes.
      Parameters:
      metadataManager -
      omBucketInfo -
      allocateSize -
      Throws:
      IOException

    • checkBucketQuotaInNamespace

      protected void checkBucketQuotaInNamespace(org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, long allocatedNamespace) throws IOException
      Check namespace quota.
      Throws:
      IOException

    • checkDirectoryAlreadyExists

      protected boolean checkDirectoryAlreadyExists(String volumeName, String bucketName, String keyName, org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager) throws IOException
      Check directory exists. If exists return true, else false.
      Parameters:
      volumeName -
      bucketName -
      keyName -
      omMetadataManager -
      Throws:
      IOException

    • sumBlockLengths

      public static long sumBlockLengths(org.apache.hadoop.ozone.om.helpers.OmKeyInfo omKeyInfo)
      Returns:
      the number of bytes (replicated size) used by blocks pointed to by omKeyInfo.

    • getBucketInfo

      @Nullable public static org.apache.hadoop.ozone.om.helpers.OmBucketInfo getBucketInfo(org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, String volume, String bucket)
      Return bucket info for the specified bucket.

    • prepareKeyInfo

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo prepareKeyInfo(@Nonnull org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, @Nonnull org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmKeyInfo dbKeyInfo, long size, @Nonnull List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, @Nullable org.apache.hadoop.fs.FileEncryptionInfo encInfo, @Nonnull PrefixManager prefixManager, @Nullable org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.ozone.om.OmConfig config) throws IOException
      Prepare OmKeyInfo which will be persisted to openKeyTable.
      Returns:
      OmKeyInfo
      Throws:
      IOException

    • prepareFileInfo

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo prepareFileInfo(@Nonnull org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager, @Nonnull org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, org.apache.hadoop.ozone.om.helpers.OmKeyInfo dbKeyInfo, long size, @Nonnull List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, @Nullable org.apache.hadoop.fs.FileEncryptionInfo encInfo, @Nonnull PrefixManager prefixManager, @Nullable org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, org.apache.hadoop.ozone.om.OmConfig config) throws IOException
      Prepare OmKeyInfo which will be persisted to openKeyTable.
      Returns:
      OmKeyInfo
      Throws:
      IOException

    • createFileInfo

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo createFileInfo(@Nonnull org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs, @Nonnull List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> locations, @Nonnull org.apache.hadoop.hdds.client.ReplicationConfig replicationConfig, long size, @Nullable org.apache.hadoop.fs.FileEncryptionInfo encInfo, @Nonnull PrefixManager prefixManager, @Nullable org.apache.hadoop.ozone.om.helpers.OmBucketInfo omBucketInfo, OMFileRequest.OMPathInfo omPathInfo, long transactionLogIndex, long objectID, org.apache.hadoop.ozone.om.OmConfig config) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Create OmKeyInfo object.
      Returns:
      OmKeyInfo
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException

    • getDBMultipartOpenKey

      protected String getDBMultipartOpenKey(String volumeName, String bucketName, String keyName, String uploadID, org.apache.hadoop.ozone.om.OMMetadataManager omMetadataManager) throws IOException
      Returns the DB key name of a multipart open key in OM metadata store.
      Parameters:
      volumeName - - volume name.
      bucketName - - bucket name.
      keyName - - key name.
      uploadID - - Multi part upload ID for this key.
      omMetadataManager -
      Returns:
      String
      Throws:
      IOException

    • getOldVersionsToCleanUp

      protected org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo getOldVersionsToCleanUp(@Nonnull org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyToDelete, long bucketId, long trxnLogIndex) throws IOException
      Prepare key for deletion service on overwrite.
      Parameters:
      keyToDelete - OmKeyInfo of a key to be in deleteTable
      bucketId -
      trxnLogIndex -
      Returns:
      Old keys eligible for deletion.
      Throws:
      IOException

    • getOzoneLockStrategy

      protected OzoneLockStrategy getOzoneLockStrategy(OzoneManager ozoneManager)

    • wrapUncommittedBlocksAsPseudoKey

      protected org.apache.hadoop.ozone.om.helpers.OmKeyInfo wrapUncommittedBlocksAsPseudoKey(List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo> uncommitted, org.apache.hadoop.ozone.om.helpers.OmKeyInfo omKeyInfo)
      Wrap the uncommitted blocks as pseudoKeyInfo.
      Parameters:
      uncommitted - Uncommitted OmKeyLocationInfo
      omKeyInfo - Args for key block
      Returns:
      pseudoKeyInfo

    • addKeyInfoToDeleteMap

      protected static Map<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo> addKeyInfoToDeleteMap(OzoneManager om, long trxnLogIndex, String ozoneKey, long bucketId, org.apache.hadoop.ozone.om.helpers.OmKeyInfo keyInfo, Map<String,org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo> deleteMap)

    • filterOutBlocksStillInUse

      protected org.apache.commons.lang3.tuple.Pair<Map<org.apache.hadoop.ozone.om.helpers.OmKeyInfo,List<org.apache.hadoop.ozone.om.helpers.OmKeyLocationInfo>>,Integer> filterOutBlocksStillInUse(org.apache.hadoop.ozone.om.helpers.OmKeyInfo referenceKey, org.apache.hadoop.ozone.om.helpers.RepeatedOmKeyInfo keysToBeFiltered)
      Remove blocks in-place from keysToBeFiltered that exist in referenceKey.

      keysToBeFiltered.getOmKeyInfoList() becomes an empty list when all blocks are filtered out.

      Parameters:
      referenceKey - OmKeyInfo
      keysToBeFiltered - RepeatedOmKeyInfo

    • validateEncryptionKeyInfo

      protected void validateEncryptionKeyInfo(org.apache.hadoop.ozone.om.helpers.OmBucketInfo bucketInfo, org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.KeyArgs keyArgs) throws org.apache.hadoop.ozone.om.exceptions.OMException
      Throws:
      org.apache.hadoop.ozone.om.exceptions.OMException