org.apache.hadoop.hbase.mob.compactions

Class PartitionedMobCompactor

java.lang.Object

org.apache.hadoop.hbase.mob.compactions.MobCompactor

org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor

@InterfaceAudience.Private
public class PartitionedMobCompactor
extends MobCompactor

An implementation of MobCompactor that compacts the mob files in partitions.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	compactionBatchSize The number of files compacted in a batch
protected int	compactionKVMax
protected int	delFileMaxCount
protected long	mergeableSize

Fields inherited from class org.apache.hadoop.hbase.mob.compactions.MobCompactor

column, conf, fs, mobFamilyDir, mobTableDir, pool, tableName

Constructor Summary

Constructors

Constructor and Description
PartitionedMobCompactor(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, TableName tableName, ColumnFamilyDescriptor column, ExecutorService pool)

Method Summary

Modifier and Type	Method and Description
List<org.apache.hadoop.fs.Path>	compact(List<org.apache.hadoop.fs.FileStatus> files, boolean allFiles) Compacts the candidate mob files.
protected List<org.apache.hadoop.fs.Path>	compactDelFiles(PartitionedMobCompactionRequest request, List<org.apache.hadoop.fs.Path> delFilePaths) Compacts the del files in batches which avoids opening too many files.
protected List<org.apache.hadoop.fs.Path>	compactMobFiles(PartitionedMobCompactionRequest request) Compacts the selected small mob files and all the del files.
protected List<org.apache.hadoop.fs.Path>	performCompaction(PartitionedMobCompactionRequest request) Performs the compaction on the selected files.
protected PartitionedMobCompactionRequest	select(List<org.apache.hadoop.fs.FileStatus> candidates, boolean allFiles) Selects the compacted mob/del files.

Methods inherited from class org.apache.hadoop.hbase.mob.compactions.MobCompactor

compact, compact

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mergeableSize

protected long mergeableSize

delFileMaxCount

protected int delFileMaxCount

compactionBatchSize

protected int compactionBatchSize

The number of files compacted in a batch

compactionKVMax

protected int compactionKVMax

Constructor Detail

PartitionedMobCompactor

public PartitionedMobCompactor(org.apache.hadoop.conf.Configuration conf,
                               org.apache.hadoop.fs.FileSystem fs,
                               TableName tableName,
                               ColumnFamilyDescriptor column,
                               ExecutorService pool)
                        throws IOException

Throws:

IOException

Method Detail

compact

public List<org.apache.hadoop.fs.Path> compact(List<org.apache.hadoop.fs.FileStatus> files,
                                               boolean allFiles)
                                        throws IOException

Description copied from class: MobCompactor

Compacts the candidate mob files.

Specified by:

compact in class MobCompactor

Parameters:

files - The candidate mob files.

allFiles - Whether add all mob files into the compaction.

Returns:

The paths of new mob files generated in the compaction.

Throws:

IOException

select

protected PartitionedMobCompactionRequest select(List<org.apache.hadoop.fs.FileStatus> candidates,
                                                 boolean allFiles)
                                          throws IOException

Selects the compacted mob/del files. Iterates the candidates to find out all the del files and small mob files.

Parameters:

candidates - All the candidates.

allFiles - Whether add all mob files into the compaction.

Returns:

A compaction request.

Throws:

IOException - if IO failure is encountered

performCompaction

protected List<org.apache.hadoop.fs.Path> performCompaction(PartitionedMobCompactionRequest request)
                                                     throws IOException

Performs the compaction on the selected files.

1. Compacts the del files.
2. Compacts the selected small mob files and all the del files.
3. If all the candidates are selected, delete the del files.

Parameters:

request - The compaction request.

Returns:

The paths of new mob files generated in the compaction.

Throws:

IOException - if IO failure is encountered

compactMobFiles

protected List<org.apache.hadoop.fs.Path> compactMobFiles(PartitionedMobCompactionRequest request)
                                                   throws IOException

Compacts the selected small mob files and all the del files.

Parameters:

request - The compaction request.

Returns:

The paths of new mob files after compactions.

Throws:

IOException - if IO failure is encountered

compactDelFiles

protected List<org.apache.hadoop.fs.Path> compactDelFiles(PartitionedMobCompactionRequest request,
                                                          List<org.apache.hadoop.fs.Path> delFilePaths)
                                                   throws IOException

Compacts the del files in batches which avoids opening too many files.

Parameters:

request - The compaction request.

delFilePaths - Del file paths to compact

Returns:

The paths of new del files after merging or the original files if no merging is necessary.

Throws:

IOException - if IO failure is encountered