Package org.apache.hadoop.ozone.container.common.volume

Class HddsVolume

java.lang.Object
org.apache.hadoop.ozone.container.common.volume.StorageVolume
org.apache.hadoop.ozone.container.common.volume.HddsVolume
All Implemented Interfaces:
org.apache.hadoop.hdfs.server.datanode.checker.Checkable<Boolean,org.apache.hadoop.hdfs.server.datanode.checker.VolumeCheckResult>
@Private @Unstable public class HddsVolume extends StorageVolume
HddsVolume represents volume in a datanode. MutableVolumeSet maintains a list of HddsVolumes, one for each volume in the Datanode. VolumeUsage in encompassed by this class.

The disk layout per volume is as follows:

../hdds/VERSION

../hdds/<<clusterUuid>>/current/<<containerDir>>/<<containerID >>/metadata

../hdds/<<clusterUuid>>/current/<<containerDir>>/<<containerID >>/<<dataDir>>

Each hdds volume has its own VERSION file. The hdds volume will have one clusterUuid directory for each SCM it is a part of (currently only one SCM is supported).

During DN startup, if the VERSION file exists, we verify that the clusterID in the version file matches the clusterID from SCM.

  • Field Details

  • Method Details

    • createWorkingDir

      public void createWorkingDir(String dirName, MutableVolumeSet dbVolumeSet) throws IOException
      Description copied from class: StorageVolume
      Create the working directory for the volume at //. Creates necessary subdirectories of the working directory as well. This includes the tmp directory at ///tmp. Child classes may override this method to add volume specific subdirectories, but they should call the parent method first to make sure initial directories are constructed.
      Overrides:
      createWorkingDir in class StorageVolume
      Parameters:
      dirName - scmID or clusterID according to SCM HA layout feature upgrade finalization status.
      Throws:
      IOException

    • createTmpDirs

      public void createTmpDirs(String workDirName) throws IOException
      Overrides:
      createTmpDirs in class StorageVolume
      Throws:
      IOException

    • getHddsRootDir

      public File getHddsRootDir()

    • getType

      public StorageVolume.VolumeType getType()

    • getVolumeIOStats

      public VolumeIOStats getVolumeIOStats()

    • getVolumeInfoStats

      public VolumeInfoMetrics getVolumeInfoStats()

    • reportBuilder

      protected StorageLocationReport.Builder reportBuilder()
      Overrides:
      reportBuilder in class StorageVolume

    • failVolume

      public void failVolume()
      Overrides:
      failVolume in class StorageVolume

    • shutdown

      public void shutdown()
      Overrides:
      shutdown in class StorageVolume

    • cleanDeletedContainerDir

      public void cleanDeletedContainerDir()
      Delete all files under volume/hdds/cluster-id/tmp/deleted-containers. This is the directory where containers are moved when they are deleted from the system, but before being removed from the filesystem. This makes the deletion atomic.

    • check

      public org.apache.hadoop.hdfs.server.datanode.checker.VolumeCheckResult check(@Nullable Boolean unused) throws Exception
      Description copied from class: StorageVolume
      Run a check on the current volume to determine if it is healthy. The check consists of a directory check and an IO check. If the directory check fails, the volume check fails immediately. The IO check is allows to fail up to ioFailureTolerance times out of the last ioTestCount IO checks before this volume check is failed. Each call to this method runs one IO check.
      Specified by:
      check in interface org.apache.hadoop.hdfs.server.datanode.checker.Checkable<Boolean,org.apache.hadoop.hdfs.server.datanode.checker.VolumeCheckResult>
      Overrides:
      check in class StorageVolume
      Parameters:
      unused - context for the check, ignored.
      Returns:
      result of checking the volume.
      Throws:
      Exception - if an exception was encountered while running the volume check and the thread was not interrupted.

    • checkDbHealth

      public org.apache.hadoop.hdfs.server.datanode.checker.VolumeCheckResult checkDbHealth(File dbFile) throws InterruptedException
      Throws:
      InterruptedException

    • checkVolumeUsages

      public void checkVolumeUsages()

    • incCommittedBytes

      public long incCommittedBytes(long delta)
      add "delta" bytes to committed space in the volume.
      Parameters:
      delta - bytes to add to committed space counter
      Returns:
      bytes of committed space

    • getCommittedBytes

      public long getCommittedBytes()
      return the committed space in the volume.
      Returns:
      bytes of committed space

    • getFreeSpaceToSpare

      public long getFreeSpaceToSpare(long volumeCapacity)

    • setGatherContainerUsages

      public void setGatherContainerUsages(Function<HddsVolume,Long> gatherContainerUsages)
      Overrides:
      setGatherContainerUsages in class StorageVolume

    • containerUsedSpace

      protected long containerUsedSpace()
      Overrides:
      containerUsedSpace in class StorageVolume

    • getContainerDirsPath

      public File getContainerDirsPath()
      Overrides:
      getContainerDirsPath in class StorageVolume

    • setDbVolume

      public void setDbVolume(DbVolume dbVolume)

    • getDbVolume

      public DbVolume getDbVolume()

    • getDbParentDir

      public File getDbParentDir()

    • setDbParentDir

      public void setDbParentDir(File dbParentDir)

    • getDeletedContainerDir

      public File getDeletedContainerDir()

    • setDeletedContainerDir

      public void setDeletedContainerDir(File deletedContainerDir)

    • isDbLoaded

      public boolean isDbLoaded()

    • isDbLoadFailure

      public boolean isDbLoadFailure()

    • loadDbStore

      public void loadDbStore(boolean readOnly) throws IOException
      Throws:
      IOException

    • setController

      public void setController(ContainerController controller)

    • getContainers

      public long getContainers()

    • createDbStore

      public void createDbStore(MutableVolumeSet dbVolumeSet) throws IOException
      Pick a DbVolume for HddsVolume and init db instance. Use the HddsVolume directly if no DbVolume found.
      Parameters:
      dbVolumeSet -
      Throws:
      IOException

    • compactDb

      public void compactDb()