org.apache.hadoop.hbase.regionserver

Class HRegionFileSystem

java.lang.Object

org.apache.hadoop.hbase.regionserver.HRegionFileSystem

@InterfaceAudience.Private
public class HRegionFileSystem
extends Object

View to an on-disk Region. Provides the set of methods necessary to interact with the on-disk region data.

Field Summary

Fields

Modifier and Type	Field and Description
static String	REGION_INFO_FILE Name of the region info file that resides just under the region directory.
static String	REGION_MERGES_DIR Temporary subdirectory of the region directory used for merges.
static String	REGION_SPLITS_DIR Temporary subdirectory of the region directory used for splits.

Method Summary

Modifier and Type	Method and Description
void	cleanupMergedRegion(RegionInfo mergedRegion) Remove merged region
org.apache.hadoop.fs.Path	commitDaughterRegion(RegionInfo regionInfo) Commit a daughter region, moving it from the split temporary directory to the proper location in the filesystem.
void	commitMergedRegion(RegionInfo mergedRegionInfo) Commit a merged region, moving it from the merges temporary directory to the proper location in the filesystem.
org.apache.hadoop.fs.Path	commitStoreFile(String familyName, org.apache.hadoop.fs.Path buildPath) Move the file from a build/temp location to the main family store directory.
void	createMergesDir() Create the region merges directory.
static HRegionFileSystem	createRegionOnFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo) Create a new Region on file-system.
void	createSplitsDir() Create the region splits directory.
org.apache.hadoop.fs.Path	createTempName() Generate a unique temporary Path.
org.apache.hadoop.fs.Path	createTempName(String suffix) Generate a unique temporary Path.
void	deleteFamily(String familyName) Remove the region family from disk, archiving the store files.
static void	deleteRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo) Remove the region from the table directory, archiving the region's hfiles.
Collection<String>	getFamilies()
org.apache.hadoop.fs.FileSystem	getFileSystem()
org.apache.hadoop.fs.Path	getMergesDir()
org.apache.hadoop.fs.Path	getRegionDir()
RegionInfo	getRegionInfo()
RegionInfo	getRegionInfoForFS()
org.apache.hadoop.fs.Path	getSplitsDir(RegionInfo hri)
String	getStoragePolicyName(String familyName) Get the storage policy of the directory of CF.
org.apache.hadoop.fs.Path	getStoreDir(String familyName) Returns the directory path of the specified family
Collection<StoreFileInfo>	getStoreFiles(byte[] familyName) Returns the store files available for the family.
Collection<StoreFileInfo>	getStoreFiles(String familyName)
Collection<StoreFileInfo>	getStoreFiles(String familyName, boolean validate) Returns the store files available for the family.
static List<org.apache.hadoop.fs.LocatedFileStatus>	getStoreFilesLocatedStatus(HRegionFileSystem regionfs, String familyName, boolean validate) Returns the store files' LocatedFileStatus which available for the family.
org.apache.hadoop.fs.Path	getTableDir()
boolean	hasReferences(String familyName) Returns true if the specified family has reference files
boolean	hasReferences(TableDescriptor htd) Check whether region has Reference file
static RegionInfo	loadRegionInfoFileContent(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path regionDir) Create a RegionInfo from the serialized version on-disk.
org.apache.hadoop.fs.Path	mergeStoreFile(RegionInfo mergedRegion, String familyName, HStoreFile f, org.apache.hadoop.fs.Path mergedDir) Write out a merge reference under the given merges directory.
static HRegionFileSystem	openRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tableDir, RegionInfo regionInfo, boolean readOnly) Open Region from file-system.
void	removeStoreFile(String familyName, org.apache.hadoop.fs.Path filePath) Archives the specified store file from the specified family.
void	removeStoreFiles(String familyName, Collection<HStoreFile> storeFiles) Closes and archives the specified store files from the specified family.
void	setStoragePolicy(String familyName, String policyName) Set the directory of CF to the specified storage policy.
org.apache.hadoop.fs.Path	splitStoreFile(RegionInfo hri, String familyName, HStoreFile f, byte[] splitRow, boolean top, RegionSplitPolicy splitPolicy) Write out a split reference.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REGION_INFO_FILE

public static final String REGION_INFO_FILE

Name of the region info file that resides just under the region directory.

See Also:

Constant Field Values

REGION_MERGES_DIR

public static final String REGION_MERGES_DIR

Temporary subdirectory of the region directory used for merges.

See Also:

Constant Field Values

REGION_SPLITS_DIR

public static final String REGION_SPLITS_DIR

Temporary subdirectory of the region directory used for splits.

See Also:

Constant Field Values

Method Detail

getFileSystem

public org.apache.hadoop.fs.FileSystem getFileSystem()

Returns:

the underlying FileSystem

getRegionInfo

public RegionInfo getRegionInfo()

Returns:

the RegionInfo that describe this on-disk region view

getRegionInfoForFS

public RegionInfo getRegionInfoForFS()

getTableDir

public org.apache.hadoop.fs.Path getTableDir()

Returns:

Path to the region's root directory.

getRegionDir

public org.apache.hadoop.fs.Path getRegionDir()

Returns:

Path to the region directory.

getStoreDir

public org.apache.hadoop.fs.Path getStoreDir(String familyName)

Returns the directory path of the specified family

Parameters:

familyName - Column Family Name

Returns:

Path to the directory of the specified family

setStoragePolicy

public void setStoragePolicy(String familyName,
                             String policyName)

Set the directory of CF to the specified storage policy.
"LAZY_PERSIST", "ALL_SSD", "ONE_SSD", "HOT", "WARM", "COLD"

See HdfsConstants for more details.

Parameters:

familyName - The name of column family.

policyName - The name of the storage policy: 'HOT', 'COLD', etc. See see hadoop 2.6+ org.apache.hadoop.hdfs.protocol.HdfsConstants for possible list e.g 'COLD', 'WARM', 'HOT', 'ONE_SSD', 'ALL_SSD', 'LAZY_PERSIST'.

getStoragePolicyName

@Nullable
public String getStoragePolicyName(String familyName)

Get the storage policy of the directory of CF.

Parameters:

familyName - The name of column family.

Returns:

Storage policy name, or null if not using HFileSystem or exception thrown when trying to get policy

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(byte[] familyName)
                                        throws IOException

Returns the store files available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a set of StoreFileInfo for the specified family.

Throws:

IOException

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(String familyName)
                                        throws IOException

Throws:

IOException

getStoreFiles

public Collection<StoreFileInfo> getStoreFiles(String familyName,
                                               boolean validate)
                                        throws IOException

Returns the store files available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a set of StoreFileInfo for the specified family.

Throws:

IOException

getStoreFilesLocatedStatus

public static List<org.apache.hadoop.fs.LocatedFileStatus> getStoreFilesLocatedStatus(HRegionFileSystem regionfs,
                                                                                      String familyName,
                                                                                      boolean validate)
                                                                               throws IOException

Returns the store files' LocatedFileStatus which available for the family. This methods performs the filtering based on the valid store files.

Parameters:

familyName - Column Family Name

Returns:

a list of store files' LocatedFileStatus for the specified family.

Throws:

IOException

hasReferences

public boolean hasReferences(String familyName)
                      throws IOException

Returns true if the specified family has reference files

Parameters:

familyName - Column Family Name

Returns:

true if family contains reference files

Throws:

IOException

hasReferences

public boolean hasReferences(TableDescriptor htd)
                      throws IOException

Check whether region has Reference file

Parameters:

htd - table desciptor of the region

Returns:

true if region has reference file

Throws:

IOException

getFamilies

public Collection<String> getFamilies()
                               throws IOException

Returns:

the set of families present on disk

Throws:

IOException

deleteFamily

public void deleteFamily(String familyName)
                  throws IOException

Remove the region family from disk, archiving the store files.

Parameters:

familyName - Column Family Name

Throws:

IOException - if an error occours during the archiving

createTempName

public org.apache.hadoop.fs.Path createTempName()

Generate a unique temporary Path. Used in conjuction with commitStoreFile() to get a safer file creation. Path file = fs.createTempName(); ...StoreFile.Writer(file)... fs.commitStoreFile("family", file);

Returns:

Unique Path of the temporary file

createTempName

public org.apache.hadoop.fs.Path createTempName(String suffix)

Generate a unique temporary Path. Used in conjuction with commitStoreFile() to get a safer file creation. Path file = fs.createTempName(); ...StoreFile.Writer(file)... fs.commitStoreFile("family", file);

Parameters:

suffix - extra information to append to the generated name

Returns:

Unique Path of the temporary file

commitStoreFile

public org.apache.hadoop.fs.Path commitStoreFile(String familyName,
                                                 org.apache.hadoop.fs.Path buildPath)
                                          throws IOException

Move the file from a build/temp location to the main family store directory.

Parameters:

familyName - Family that will gain the file

buildPath - Path to the file to commit.

Returns:

The new Path of the committed file

Throws:

IOException

removeStoreFile

public void removeStoreFile(String familyName,
                            org.apache.hadoop.fs.Path filePath)
                     throws IOException

Archives the specified store file from the specified family.

Parameters:

familyName - Family that contains the store files

filePath - Path to the store file to remove

Throws:

IOException - if the archiving fails

removeStoreFiles

public void removeStoreFiles(String familyName,
                             Collection<HStoreFile> storeFiles)
                      throws IOException

Closes and archives the specified store files from the specified family.

Parameters:

familyName - Family that contains the store files

storeFiles - set of store files to remove

Throws:

IOException - if the archiving fails

getSplitsDir

public org.apache.hadoop.fs.Path getSplitsDir(RegionInfo hri)

commitDaughterRegion

public org.apache.hadoop.fs.Path commitDaughterRegion(RegionInfo regionInfo)
                                               throws IOException

Commit a daughter region, moving it from the split temporary directory to the proper location in the filesystem.

Parameters:

regionInfo - daughter RegionInfo

Throws:

IOException

createSplitsDir

public void createSplitsDir()
                     throws IOException

Create the region splits directory.

Throws:

IOException

splitStoreFile

public org.apache.hadoop.fs.Path splitStoreFile(RegionInfo hri,
                                                String familyName,
                                                HStoreFile f,
                                                byte[] splitRow,
                                                boolean top,
                                                RegionSplitPolicy splitPolicy)
                                         throws IOException

Write out a split reference. Package local so it doesnt leak out of regionserver.

Parameters:

hri - RegionInfo of the destination

familyName - Column Family Name

f - File to split.

splitRow - Split Row

top - True if we are referring to the top half of the hfile.

splitPolicy - A split policy instance; be careful! May not be full populated; e.g. if this method is invoked on the Master side, then the RegionSplitPolicy will NOT have a reference to a Region.

Returns:

Path to created reference.

Throws:

IOException

getMergesDir

public org.apache.hadoop.fs.Path getMergesDir()

Returns:

Path to the temp directory used during merge operations

cleanupMergedRegion

public void cleanupMergedRegion(RegionInfo mergedRegion)
                         throws IOException

Remove merged region

Parameters:

mergedRegion - RegionInfo

Throws:

IOException

createMergesDir

public void createMergesDir()
                     throws IOException

Create the region merges directory.

Throws:

IOException - If merges dir already exists or we fail to create it.

See Also:

cleanupMergesDir()

mergeStoreFile

public org.apache.hadoop.fs.Path mergeStoreFile(RegionInfo mergedRegion,
                                                String familyName,
                                                HStoreFile f,
                                                org.apache.hadoop.fs.Path mergedDir)
                                         throws IOException

Write out a merge reference under the given merges directory. Package local so it doesnt leak out of regionserver.

Parameters:

mergedRegion - RegionInfo of the merged region

familyName - Column Family Name

f - File to create reference.

mergedDir -

Returns:

Path to created reference.

Throws:

IOException

commitMergedRegion

public void commitMergedRegion(RegionInfo mergedRegionInfo)
                        throws IOException

Commit a merged region, moving it from the merges temporary directory to the proper location in the filesystem.

Parameters:

mergedRegionInfo - merged region RegionInfo

Throws:

IOException

loadRegionInfoFileContent

public static RegionInfo loadRegionInfoFileContent(org.apache.hadoop.fs.FileSystem fs,
                                                   org.apache.hadoop.fs.Path regionDir)
                                            throws IOException

Create a RegionInfo from the serialized version on-disk.

Parameters:

fs - FileSystem that contains the Region Info file

regionDir - Path to the Region Directory that contains the Info file

Returns:

An RegionInfo instance gotten from the Region Info file.

Throws:

IOException - if an error occurred during file open/read operation.

createRegionOnFileSystem

public static HRegionFileSystem createRegionOnFileSystem(org.apache.hadoop.conf.Configuration conf,
                                                         org.apache.hadoop.fs.FileSystem fs,
                                                         org.apache.hadoop.fs.Path tableDir,
                                                         RegionInfo regionInfo)
                                                  throws IOException

Create a new Region on file-system.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to add the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be added

Throws:

IOException - if the region creation fails due to a FileSystem exception.

openRegionFromFileSystem

public static HRegionFileSystem openRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf,
                                                         org.apache.hadoop.fs.FileSystem fs,
                                                         org.apache.hadoop.fs.Path tableDir,
                                                         RegionInfo regionInfo,
                                                         boolean readOnly)
                                                  throws IOException

Open Region from file-system.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to add the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be added

readOnly - True if you don't want to edit the region data

Throws:

IOException - if the region creation fails due to a FileSystem exception.

deleteRegionFromFileSystem

public static void deleteRegionFromFileSystem(org.apache.hadoop.conf.Configuration conf,
                                              org.apache.hadoop.fs.FileSystem fs,
                                              org.apache.hadoop.fs.Path tableDir,
                                              RegionInfo regionInfo)
                                       throws IOException

Remove the region from the table directory, archiving the region's hfiles.

Parameters:

conf - the Configuration to use

fs - FileSystem from which to remove the region

tableDir - Path to where the table is being stored

regionInfo - RegionInfo for region to be deleted

Throws:

IOException - if the request cannot be completed