Package com.crawljax.core

Class UnfiredCandidateActions

java.lang.Object

com.crawljax.core.UnfiredCandidateActions

@Singleton
public class UnfiredCandidateActions
extends Object

Contains all the CandidateCrawlActions that still have to be fired to get a result.

Field Summary

Fields

Modifier and Type	Field	Description
private Map<Integer,Queue<CandidateCrawlAction>>	cache
private com.codahale.metrics.Counter	crawlerLostCount
private com.google.common.util.concurrent.Striped<Lock>	locks
private static org.slf4j.Logger	LOG
private javax.inject.Provider<StateFlowGraph>	sfg
private BlockingQueue<Integer>	statesWithCandidates
private com.codahale.metrics.Counter	unfiredActionsCount

Constructor Summary

Constructors

Constructor	Description
UnfiredCandidateActions(BrowserConfiguration config, javax.inject.Provider<StateFlowGraph> sfg, com.codahale.metrics.MetricRegistry registry)

Method Summary

Modifier and Type	Method	Description
void	addActions(com.google.common.collect.ImmutableList<CandidateElement> extract, StateVertex currentState)
(package private) void	addActions(Collection<CandidateCrawlAction> actions, StateVertex state)
StateVertex	awaitNewTask()
StateVertex	awaitNewTaskPriority(CrawlRules.CrawlPriorityMode crawlPriorityMode, boolean crawlNearDuplicates, boolean delayNearDuplicateCrawling)
StateVertex	getNextNonDuplicate()
StateVertex	getOldest()
boolean	isEmpty()
(package private) CandidateCrawlAction	pollActionOrNull(StateVertex state)
void	purgeActionsForState(StateVertex crawlTask)
private void	removeStateFromQueue(int id)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

private static final org.slf4j.Logger LOG

cache

private final Map<Integer,Queue<CandidateCrawlAction>> cache

statesWithCandidates

private final BlockingQueue<Integer> statesWithCandidates

locks

private final com.google.common.util.concurrent.Striped<Lock> locks

sfg

private final javax.inject.Provider<StateFlowGraph> sfg

crawlerLostCount

private final com.codahale.metrics.Counter crawlerLostCount

unfiredActionsCount

private final com.codahale.metrics.Counter unfiredActionsCount

Constructor Detail

UnfiredCandidateActions

@Inject
UnfiredCandidateActions(BrowserConfiguration config,
                        javax.inject.Provider<StateFlowGraph> sfg,
                        com.codahale.metrics.MetricRegistry registry)

Method Detail

pollActionOrNull

CandidateCrawlAction pollActionOrNull(StateVertex state)

Parameters:

state - The state you want to poll an CandidateCrawlAction for.

Returns:

The next to-be-crawled action or null if none available.

removeStateFromQueue

private void removeStateFromQueue(int id)

addActions

public void addActions(com.google.common.collect.ImmutableList<CandidateElement> extract,
                       StateVertex currentState)

Parameters:

extract - The actions you want to add to a state.

currentState - The state you are in.

addActions

void addActions(Collection<CandidateCrawlAction> actions,
                StateVertex state)

Parameters:

actions - The actions you want to add to a state.

state - The state name. This should be unique per state.

isEmpty

public boolean isEmpty()

Returns:

If there are any pending actions to be crawled. This method is not threadsafe and might return a stale value.

awaitNewTask

public StateVertex awaitNewTask()
                         throws InterruptedException

Returns:

A new crawl task as soon as one is ready. Until then, it blocks.

Throws:

InterruptedException - when taking from the queue is interrupted.

getOldest

public StateVertex getOldest()

getNextNonDuplicate

public StateVertex getNextNonDuplicate()

awaitNewTaskPriority

public StateVertex awaitNewTaskPriority(CrawlRules.CrawlPriorityMode crawlPriorityMode,
                                        boolean crawlNearDuplicates,
                                        boolean delayNearDuplicateCrawling)
                                 throws InterruptedException

Parameters:

crawlPriorityMode - the crawling modality

crawlNearDuplicates - whether to consider near-duplicates

delayNearDuplicateCrawling - whether to postpone the crawl of near-duplicates

Returns:

A new crawl task as soon as one is ready. Until then, it blocks.

Throws:

InterruptedException - when taking from the queue is interrupted.

purgeActionsForState

public void purgeActionsForState(StateVertex crawlTask)