Package org.apache.hadoop.ozone.om.service

Class DirectoryDeletingService

java.lang.Object
org.apache.hadoop.hdds.utils.BackgroundService
org.apache.hadoop.ozone.om.service.AbstractKeyDeletingService
org.apache.hadoop.ozone.om.service.DirectoryDeletingService
All Implemented Interfaces:
org.apache.hadoop.ozone.lock.BootstrapStateHandler
public class DirectoryDeletingService extends AbstractKeyDeletingService
Background service responsible for purging deleted directories and files in the Ozone Manager (OM) and associated snapshots.

This service periodically scans the deleted directory table and submits purge requests for directories and their sub-entries (subdirectories and files). It operates in both the active object store (AOS) and across all deep-clean enabled snapshots. The service supports parallel processing using a thread pool and coordinates exclusive size calculations and cleanup status updates for snapshots.

Key Features

  • Processes deleted directories in both the active OM and all snapshots with deep cleaning enabled.
  • Uses a thread pool to parallelize deletion tasks within each store or snapshot.
  • Employs filters to determine reclaimability of directories and files, ensuring safety with respect to snapshot chains.
  • Tracks and updates exclusive size and replicated exclusive size for each snapshot as directories and files are reclaimed.
  • Updates the "deep cleaned" flag for snapshots after a successful run.
  • Handles error and race conditions gracefully, deferring work if necessary.

Constructor Parameters

  • interval - How often the service runs.
  • unit - Time unit for the interval.
  • serviceTimeout - Service timeout in the given time unit.
  • ozoneManager - The OzoneManager instance.
  • configuration - Ozone configuration object.
  • dirDeletingServiceCorePoolSize - Number of parallel threads for deletion per store or snapshot.
  • deepCleanSnapshots - Whether to enable deep cleaning for snapshots.

Threading and Parallelism

  • Uses a configurable thread pool for parallel deletion tasks within each store/snapshot.
  • Each snapshot and AOS get a separate background task for deletion.

Snapshot Integration

  • Iterates all snapshots in the chain if deep cleaning is enabled.
  • Skips snapshots that are already deep-cleaned or not yet flushed to disk.
  • Updates snapshot metadata to reflect size changes and cleaning status.

Usage

  • Should be scheduled as a background service in OM.
  • Intended to be run only on the OM leader node.
See Also:

  • Constructor Details

    • DirectoryDeletingService

      public DirectoryDeletingService(long interval, TimeUnit unit, long serviceTimeout, OzoneManager ozoneManager, org.apache.hadoop.hdds.conf.OzoneConfiguration configuration, int dirDeletingServiceCorePoolSize, boolean deepCleanSnapshots)

  • Method Details

    • registerReconfigCallbacks

      public void registerReconfigCallbacks(org.apache.hadoop.hdds.conf.ReconfigurationHandler handler)

    • getTasks

      public org.apache.hadoop.hdds.utils.BackgroundTaskQueue getTasks()
      Specified by:
      getTasks in class org.apache.hadoop.hdds.utils.BackgroundService

    • shutdown

      public void shutdown()
      Overrides:
      shutdown in class org.apache.hadoop.hdds.utils.BackgroundService

    • start

      public void start()
      Overrides:
      start in class org.apache.hadoop.hdds.utils.BackgroundService

    • getDeletedDirsCount

      public long getDeletedDirsCount()
      Returns the number of dirs deleted by the background service.
      Returns:
      Long count.

    • getMovedDirsCount

      public long getMovedDirsCount()
      Returns the number of sub-dirs deleted by the background service.
      Returns:
      Long count.

    • getMovedFilesCount

      public long getMovedFilesCount()
      Returns the number of files moved to DeletedTable by the background service.
      Returns:
      Long count.