com.crawljax.core.configuration

Class CrawljaxConfiguration.CrawljaxConfigurationBuilder

java.lang.Object

com.crawljax.core.configuration.CrawljaxConfiguration.CrawljaxConfigurationBuilder

Enclosing class:

CrawljaxConfiguration

public static class CrawljaxConfiguration.CrawljaxConfigurationBuilder
extends Object

Method Summary

Methods

Modifier and Type	Method and Description
CrawljaxConfiguration.CrawljaxConfigurationBuilder	addPlugin(Plugin... plugins) Add plugins to Crawljax.
CrawljaxConfiguration	build()
CrawlController	buildControl()
CrawlRules.CrawlRulesBuilder	crawlRules()
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setBasicAuth(String username, String password) If the website uses Basic auth you can set the username and password here.
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setBrowserConfig(BrowserConfiguration configuration)
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setMaximumDepth(int depth)
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setMaximumRunTime(long time, TimeUnit unit)
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setMaximumStates(int states)
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setOutputDirectory(File output) Set the output folder for any Plugin you might configure.
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setProxyConfig(ProxyConfiguration configuration)
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setUnlimitedCrawlDepth() Set the crawl depth to unlimited.
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setUnlimitedRuntime() Set the maximum runtime to unlimited.
CrawljaxConfiguration.CrawljaxConfigurationBuilder	setUnlimitedStates() Crawl without a maximum state limit.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setBasicAuth

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setBasicAuth(String username,
                                                              String password)

If the website uses Basic auth you can set the username and password here.

Parameters:

username - The username for the website.

password - The password for the website.

Returns:

CrawljaxConfiguration.CrawljaxConfigurationBuilder for method chaining.

setMaximumStates

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setMaximumStates(int states)

Parameters:

states - The maximum number of states the Crawler should crawl. The default is unlimited.

setUnlimitedStates

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setUnlimitedStates()

Crawl without a maximum state limit.

setMaximumRunTime

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setMaximumRunTime(long time,
                                                                   TimeUnit unit)

Parameters:

time - The maximum time the crawler should run. Default is one hour.

setUnlimitedRuntime

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setUnlimitedRuntime()

Set the maximum runtime to unlimited.

setMaximumDepth

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setMaximumDepth(int depth)

Parameters:

depth - The maximum depth the crawler can reach. The default is 2.

setUnlimitedCrawlDepth

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setUnlimitedCrawlDepth()

Set the crawl depth to unlimited. The default depth is 2.

addPlugin

public CrawljaxConfiguration.CrawljaxConfigurationBuilder addPlugin(Plugin... plugins)

Add plugins to Crawljax. Note that without plugins, Crawljax won't give any ouput. For basic output at least enable the CrawlOverviewPlugin.

You can call this method several times to add multiple plugins

Parameters:

plugins - the plugins you would like to enable.

setProxyConfig

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setProxyConfig(ProxyConfiguration configuration)

Parameters:

configuration - The proxy configuration. Default is ProxyConfiguration.noProxy()

crawlRules

public CrawlRules.CrawlRulesBuilder crawlRules()

Returns:

The CrawlRules.CrawlRulesBuilder to define crawling rules.

setBrowserConfig

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setBrowserConfig(BrowserConfiguration configuration)

Parameters:

configuration - a custom BrowserConfiguration. The default is a single EmbeddedBrowser.BrowserType.FIREFOX browser.

setOutputDirectory

public CrawljaxConfiguration.CrawljaxConfigurationBuilder setOutputDirectory(File output)

Set the output folder for any Plugin you might configure. Crawljax itself doesn't need an output folder but many plug-ins do.

Parameters:

output - The output folder. If it does not exist it will be created.

Throws:

IllegalStateException - if the specified file is not writable or exists but isn't a folder.

build

public CrawljaxConfiguration build()

buildControl

public CrawlController buildControl()