org.apache.hadoop.hbase.coordination

Class ZkSplitLogWorkerCoordination

java.lang.Object

org.apache.hadoop.hbase.zookeeper.ZKListener

org.apache.hadoop.hbase.coordination.ZkSplitLogWorkerCoordination

All Implemented Interfaces:

SplitLogWorkerCoordination

@InterfaceAudience.Private
public class ZkSplitLogWorkerCoordination
extends ZKListener
implements SplitLogWorkerCoordination

ZooKeeper based implementation of SplitLogWorkerCoordination It listen for changes in ZooKeeper and

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ZkSplitLogWorkerCoordination.ZkSplitTaskDetails When ZK-based implementation wants to complete the task, it needs to know task znode and current znode cversion (needed for subsequent update operation).

Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination

SplitLogWorkerCoordination.SplitTaskDetails

Field Summary

Fields

Modifier and Type	Field and Description
protected AtomicInteger	tasksInProgress

Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZKListener

watcher

Fields inherited from interface org.apache.hadoop.hbase.coordination.SplitLogWorkerCoordination

DEFAULT_MAX_SPLITTERS

Constructor Summary

Constructors

Constructor and Description
ZkSplitLogWorkerCoordination(ServerName serverName, ZKWatcher watcher)

Method Summary

Modifier and Type	Method and Description
protected static int	attemptToOwnTask(boolean isFirstTime, ZKWatcher zkw, ServerName server, String task, int taskZKVersion) Try to own the task by transitioning the zk node data from UNASSIGNED to OWNED.
void	endTask(SplitLogTask slt, LongAdder ctr, SplitLogWorkerCoordination.SplitTaskDetails details) endTask() can fail and the only way to recover out of it is for the SplitLogManager to timeout the task node.
void	getDataSetWatchAsync()
int	getTaskReadySeq() Used by unit tests to check how many tasks were processed
void	init(RegionServerServices server, org.apache.hadoop.conf.Configuration conf, SplitLogWorker.TaskExecutor splitExecutor, SplitLogWorker worker) Override setter from SplitLogWorkerCoordination
boolean	isReady() Check whether the log splitter is ready to supply tasks
boolean	isStop()
void	markCorrupted(org.apache.hadoop.fs.Path rootDir, String name, org.apache.hadoop.fs.FileSystem fs) marks log file as corrupted
void	nodeChildrenChanged(String path) Override handler from ZKListener
void	nodeDataChanged(String path) Override handler from ZKListener
void	registerListener() set the listener for task changes.
void	removeListener() remove the listener for task changes.
void	stopProcessingTasks() called when Coordination should stop processing tasks and exit
void	taskLoop() Wait for tasks to become available at /hbase/splitlog zknode.

Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZKListener

getWatcher, nodeCreated, nodeDeleted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tasksInProgress

protected final AtomicInteger tasksInProgress

Constructor Detail

ZkSplitLogWorkerCoordination

public ZkSplitLogWorkerCoordination(ServerName serverName,
                                    ZKWatcher watcher)

Method Detail

nodeChildrenChanged

public void nodeChildrenChanged(String path)

Override handler from ZKListener

Overrides:

nodeChildrenChanged in class ZKListener

nodeDataChanged

public void nodeDataChanged(String path)

Override handler from ZKListener

Overrides:

nodeDataChanged in class ZKListener

init

public void init(RegionServerServices server,
                 org.apache.hadoop.conf.Configuration conf,
                 SplitLogWorker.TaskExecutor splitExecutor,
                 SplitLogWorker worker)

Override setter from SplitLogWorkerCoordination

Specified by:

init in interface SplitLogWorkerCoordination

Parameters:

server - instance of RegionServerServices to work with

conf - is current configuration.

splitExecutor - split executor from SplitLogWorker

worker - instance of SplitLogWorker

getDataSetWatchAsync

public void getDataSetWatchAsync()

attemptToOwnTask

protected static int attemptToOwnTask(boolean isFirstTime,
                                      ZKWatcher zkw,
                                      ServerName server,
                                      String task,
                                      int taskZKVersion)

Try to own the task by transitioning the zk node data from UNASSIGNED to OWNED.

This method is also used to periodically heartbeat the task progress by transitioning the node from OWNED to OWNED.

Parameters:

isFirstTime - shows whther it's the first attempt.

zkw - zk wathcer

server - name

task - to own

taskZKVersion - version of the task in zk

Returns:

non-negative integer value when task can be owned by current region server otherwise -1

taskLoop

public void taskLoop()
              throws InterruptedException

Wait for tasks to become available at /hbase/splitlog zknode. Grab a task one at a time. This policy puts an upper-limit on the number of simultaneous log splitting that could be happening in a cluster.

Synchronization using taskReadySeq ensures that it will try to grab every task that has been put up

Specified by:

taskLoop in interface SplitLogWorkerCoordination

Throws:

InterruptedException

markCorrupted

public void markCorrupted(org.apache.hadoop.fs.Path rootDir,
                          String name,
                          org.apache.hadoop.fs.FileSystem fs)

Description copied from interface: SplitLogWorkerCoordination

marks log file as corrupted

Specified by:

markCorrupted in interface SplitLogWorkerCoordination

Parameters:

rootDir - where to find the log

name - of the log

fs - file system

isReady

public boolean isReady()
                throws InterruptedException

Description copied from interface: SplitLogWorkerCoordination

Check whether the log splitter is ready to supply tasks

Specified by:

isReady in interface SplitLogWorkerCoordination

Returns:

false if there is no tasks

Throws:

InterruptedException - if the SplitLogWorker was stopped

getTaskReadySeq

public int getTaskReadySeq()

Description copied from interface: SplitLogWorkerCoordination

Used by unit tests to check how many tasks were processed

Specified by:

getTaskReadySeq in interface SplitLogWorkerCoordination

Returns:

number of tasks

registerListener

public void registerListener()

Description copied from interface: SplitLogWorkerCoordination

set the listener for task changes. Implementation specific

Specified by:

registerListener in interface SplitLogWorkerCoordination

removeListener

public void removeListener()

Description copied from interface: SplitLogWorkerCoordination

remove the listener for task changes. Implementation specific

Specified by:

removeListener in interface SplitLogWorkerCoordination

stopProcessingTasks

public void stopProcessingTasks()

Description copied from interface: SplitLogWorkerCoordination

called when Coordination should stop processing tasks and exit

Specified by:

stopProcessingTasks in interface SplitLogWorkerCoordination

isStop

public boolean isStop()

Specified by:

isStop in interface SplitLogWorkerCoordination

Returns:

the current value of exitWorker

endTask

public void endTask(SplitLogTask slt,
                    LongAdder ctr,
                    SplitLogWorkerCoordination.SplitTaskDetails details)

endTask() can fail and the only way to recover out of it is for the SplitLogManager to timeout the task node.

Specified by:

endTask in interface SplitLogWorkerCoordination

Parameters:

slt -

ctr -

details - details about log split task (specific to coordination engine being used).