com.cinchapi.concourse.importer

Class Importer

Object

Importer

Direct Known Subclasses:

DelimitedLineImporter, JsonImporter

@NotThreadSafe
public abstract class Importer
extends Object

This is the base class for bringing data from flat files into Concourse. Implementing classes must provide logic for importing data from both files and strings. A client connection to Concourse is provided so that the implementation can choose the specific methods for optimally importing the data from the file.

This framework does not mandate anything about the structure of the raw data so it is possible to use this implementation to handle things like CSV files, XML files, SQL dumps, email documents, etc in both generic and specific form.

An Importer is assumed to maintain state about the import process it manages (i.e. it is possible, an expected that the importer will keep track of things like which files it has imported or whether it has already processed header information to provide metadata for the import). As a result, importers are not safe for concurrent use by multiple threads.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ANNOTATE_DATA_SOURCE_OPTION_NAME The name of the key that contains the value for the --annotate-data-source CLI option.
protected Concourse	concourse The connection to Concourse.
protected static String	DATA_SOURCE_ANNOTATION_KEY The key that contains datasource annotations.
protected Map<String,String>	params A map of dynamic parameters that may be passed in from the ImportCli.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Importer(Concourse concourse) Construct a new instance.

Method Summary

Modifier and Type	Method and Description
void	close() Close the importer and release any resources that it has open.
abstract Set<Long>	importFile(String file) Import the data from file into Concourse and return the record ids where the data was imported.
abstract Set<Long>	importString(String data) Import data directly from a String, for instance input from stdin.
void	setParams(Map<String,String> params) Assign a collection of dynamic parameters to this Importer for additional configuration.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ANNOTATE_DATA_SOURCE_OPTION_NAME

public static final String ANNOTATE_DATA_SOURCE_OPTION_NAME

The name of the key that contains the value for the --annotate-data-source CLI option.

See Also:

Constant Field Values

DATA_SOURCE_ANNOTATION_KEY

protected static final String DATA_SOURCE_ANNOTATION_KEY

The key that contains datasource annotations.

See Also:

Constant Field Values

concourse

protected final Concourse concourse

The connection to Concourse.

params

protected Map<String,String> params

A map of dynamic parameters that may be passed in from the ImportCli.

Constructor Detail

Importer

protected Importer(Concourse concourse)

Construct a new instance.

Parameters:

concourse -

Method Detail

close

public final void close()

Close the importer and release any resources that it has open.

importFile

@Nullable
public abstract Set<Long> importFile(String file)

Import the data from file into Concourse and return the record ids where the data was imported.

Parameters:

file - the path of the file to import; relative paths will be resolved against the user.dir system property. When using the ImportCli, prior to passing the path to this method, relative paths are converted to absolute paths by resolving against the directory from which the CLI process is launched

Returns:

a Set that contains the ids of the records affected by the import

importString

@Nullable
public abstract Set<Long> importString(String data)

Import data directly from a String, for instance input from stdin.

Parameters:

data - a string with data to import

Returns:

a Set that contains the ids of the records affected by the import

setParams

public void setParams(Map<String,String> params)

Assign a collection of dynamic parameters to this Importer for additional configuration.

Parameters:

params - dynamic paramters, usually passed in from the ImportCli