Package org.apache.hadoop.ozone.container.common.impl

Class ContainerSet

java.lang.Object
org.apache.hadoop.ozone.container.common.impl.ContainerSet
All Implemented Interfaces:
Iterable<Container<?>>
public class ContainerSet extends Object implements Iterable<Container<?>>
Class that manages Containers created on the datanode.

  • Method Details

    • newReadOnlyContainerSet

      public static ContainerSet newReadOnlyContainerSet(long recoveringTimeout)

    • newRwContainerSet

      public static ContainerSet newRwContainerSet(WitnessedContainerMetadataStore metadataStore, long recoveringTimeout)

    • getCurrentTime

      public long getCurrentTime()

    • getContainerMetadataStore

      @Nullable public WitnessedContainerMetadataStore getContainerMetadataStore()

    • setRecoveringTimeout

      public void setRecoveringTimeout(long recoveringTimeout)

    • addContainer

      public boolean addContainer(Container<?> container) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Add Container to container map. This would fail if the container is already present or has been marked as missing.
      Parameters:
      container - container to be added
      Returns:
      If container is added to containerMap returns true, otherwise false
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • addContainerByOverwriteMissingContainer

      public boolean addContainerByOverwriteMissingContainer(Container<?> container) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Add Container to container map. This would overwrite the container even if it is missing. But would fail if the container is already present.
      Parameters:
      container - container to be added
      Returns:
      If container is added to containerMap returns true, otherwise false
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • ensureContainerNotMissing

      public void ensureContainerNotMissing(long containerId, org.apache.hadoop.hdds.protocol.datanode.proto.ContainerProtos.ContainerDataProto.State state) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • registerOnDemandScanner

      public void registerOnDemandScanner(OnDemandContainerScanner scanner)
      Parameters:
      scanner - The scanner instance will be invoked when a scan of a container in this set is requested.

    • scanContainer

      public void scanContainer(long containerID, String reasonForScan)
      Triggers a scan of a container in this set. This is a no-op if no scanner is registered or the container does not exist in the set.
      Parameters:
      containerID - The container in this set to scan.

    • scanContainerWithoutGap

      public void scanContainerWithoutGap(long containerID, String reasonForScan)
      Triggers a scan of a container in this set regardless of whether it was recently scanned. This is a no-op if no scanner is registered or the container does not exist in the set.
      Parameters:
      containerID - The container in this set to scan.

    • getContainer

      public Container<?> getContainer(long containerId)
      Returns the Container with specified containerId.
      Parameters:
      containerId - ID of the container to get
      Returns:
      Container

    • removeContainer

      public boolean removeContainer(long containerId) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Removes container from both memory and database. This should be used when the containerData on disk has been removed completely from the node.
      Parameters:
      containerId -
      Returns:
      True if container is removed from containerMap.
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • removeContainerOnlyFromMemory

      public boolean removeContainerOnlyFromMemory(long containerId) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Removes containerId from memory. This needs to be used when the container is still present on disk, and the inmemory state of the container needs to be updated.
      Parameters:
      containerId -
      Returns:
      True if container is removed from containerMap.
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • removeMissingContainer

      public boolean removeMissingContainer(long containerId) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Marks a container to be missing, thus it removes the container from inmemory containerMap and marks the container as missing.
      Parameters:
      containerId -
      Returns:
      True if container is removed from containerMap.
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • removeRecoveringContainer

      public boolean removeRecoveringContainer(long containerId)
      Removes the Recovering Container matching with specified containerId.
      Parameters:
      containerId - ID of the container to remove.
      Returns:
      true If container is removed from containerMap returns true, otherwise false.

    • containerCount

      public int containerCount()
      Return number of containers in container map.
      Returns:
      container count

    • handleVolumeFailures

      public void handleVolumeFailures(StateContext context) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      Remove all containers belonging to failed volume. Send FCR which will not contain removed containers.
      Parameters:
      context - StateContext
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • iterator

      public Iterator<Container<?>> iterator()
      Specified by:
      iterator in interface Iterable<Container<?>>

    • getRecoveringContainerIterator

      public Iterator<Map.Entry<Long,Long>> getRecoveringContainerIterator()
      Return an container Iterator over recoveringContainerMap.
      Returns:
      Iterator<Container<?>>

    • getContainerIterator

      public Iterator<Container<?>> getContainerIterator(HddsVolume volume)
      Return an iterator of containers associated with the specified volume. The iterator is sorted by last data scan timestamp in increasing order.
      Parameters:
      volume - the HDDS volume which should be used to filter containers
      Returns:
      Iterator<Container<?>>

    • containerCount

      public long containerCount(HddsVolume volume)
      Get the number of containers based on the given volume.
      Parameters:
      volume - hdds volume.
      Returns:
      number of containers

    • getContainerMapIterator

      public Iterator<Map.Entry<Long,Container<?>>> getContainerMapIterator()
      Return an containerMap iterator over containerMap.
      Returns:
      containerMap Iterator

    • getContainerMapCopy

      public Map<Long,Container<?>> getContainerMapCopy()
      Return a copy of the containerMap.
      Returns:
      containerMap

    • getContainerMap

      public Map<Long,Container<?>> getContainerMap()

    • listContainer

      public void listContainer(long startContainerId, long count, List<ContainerData> data) throws org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException
      A simple interface for container Iterations.

      This call make no guarantees about consistency of the data between different list calls. It just returns the best known data at that point of time. It is possible that using this iteration you can miss certain container from the listing.

      Parameters:
      startContainerId - - Return containers with Id >= startContainerId.
      count - - how many to return
      data - - Actual containerData
      Throws:
      org.apache.hadoop.hdds.scm.container.common.helpers.StorageContainerException

    • getContainerReport

      public org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ContainerReportsProto getContainerReport() throws IOException
      Get container report.
      Returns:
      The container report.
      Throws:
      IOException

    • getMissingContainerSet

      public Set<Long> getMissingContainerSet()

    • buildMissingContainerSetAndValidate

      public <T> void buildMissingContainerSetAndValidate(Map<T,Long> container2BCSIDMap, ToLongFunction<T> getId)
      Builds the missing container set by taking a diff between total no containers actually found and number of containers which actually got created. It also validates the BCSID stored in the snapshot file for each container as against what is reported in containerScan. This will only be called during the initialization of Datanode Service when it still not a part of any write Pipeline.
      Parameters:
      container2BCSIDMap - Map of containerId to BCSID persisted in the Ratis snapshot