public abstract class DelimitedLineImporter extends Importer implements Headered
Importer that splits each line of input by a delimiter()
to produce key/value pairs for import. Each line is imported entirely into at
least one record in Concourse.ANNOTATE_DATA_SOURCE_OPTION_NAME, concourse, DATA_SOURCE_ANNOTATION_KEY, params| Modifier | Constructor and Description |
|---|---|
protected |
DelimitedLineImporter(Concourse concourse)
Construct a new instance.
|
| Modifier and Type | Method and Description |
|---|---|
protected abstract char |
delimiter()
The delimiter that is used to split field on each line.
|
protected List<String> |
header()
Provide an ordered list of header columns/keys if they are not provided
in the first line that is processed by this importer.
|
Set<Long> |
importFile(String file)
Import the data from
file into Concourse and return the
record ids where the data was imported. |
Set<Long> |
importFile(String file,
String resolveKey)
Import all of the lines from
file into Concourse and determine
the appropriate destination record(s) for each line using the
resolveKey. |
protected Set<Long> |
importLines(List<String> lines,
String resolveKey,
String source)
Process the
lines and import the parsed objects into Concourse. |
Set<Long> |
importString(String data)
Import
data directly from a String, for instance input
from stdin. |
Set<Long> |
importString(String lines,
String resolveKey)
Import all the
lines into Concourse and determine the appropriate
destination record(s) for each line using the resolveKey. |
void |
parseHeader(String line)
After parsing the
line, assign each of the tokens to be the
header for the Importer if and only if a header has not already
been assigned or parsed. |
protected Multimap<String,Object> |
parseObject(String line)
Transform the delimited line into an object where each field is mapped as
a value from the analogous key in the
header. |
protected Transformer |
transformer()
Return a
Transformer to potentially alter key/value pairs seen by
this importer. |
protected DelimitedLineImporter(Concourse concourse)
concourse - the connection to Concourse to use for importingpublic final Set<Long> importFile(String file)
Importerfile into Concourse and return the
record ids where the data was imported.importFile in class Importerfile - the path of the file to import; relative paths will be
resolved against the user.dir system property. When
using the ImportCli, prior
to passing the path to this method, relative paths are
converted to absolute paths by resolving against the directory
from which the CLI process is launchedSet that contains the ids of the records affected by
the importpublic final Set<Long> importFile(String file, @Nullable String resolveKey)
file into Concourse and determine
the appropriate destination record(s) for each line using the
resolveKey.file - the path to the file to importresolveKey - the key to use when resolving one or more existing
records into which to import a line. When using a
resolveKey, the Importer instructs Concourse to find
all the records that contain the same value for the key that
exists in the line and import the rest of the data in the line
into those recordspublic Set<Long> importString(String data)
Importerdata directly from a String, for instance input
from stdin.importString in class Importerdata - a string with data to importSet that contains the ids of the records affected by
the importpublic final Set<Long> importString(String lines, @Nullable String resolveKey)
lines into Concourse and determine the appropriate
destination record(s) for each line using the resolveKey.lines - the text to importresolveKey - the key to use when resolving one or more existing
records into which to import a line. When using a
resolveKey, the Importer instructs Concourse to find
all the records that contain the same value for the key that
exists in the line and import the rest of the data in the line
into those recordspublic final void parseHeader(String line)
Headeredline, assign each of the tokens to be the
header for the Importer if and only if a header has not already
been assigned or parsed.parseHeader in interface Headeredline - the line that should be parsed to get the headerprotected abstract char delimiter()
protected List<String> header()
protected final Set<Long> importLines(List<String> lines, @Nullable String resolveKey, @Nullable String source)
lines and import the parsed objects into Concourse.lines - resolveKey - source - protected Multimap<String,Object> parseObject(String line)
header. Each key/value
mapping is subject to processing by the transformer.line - protected Transformer transformer()
Transformer to potentially alter key/value pairs seen by
this importer.