net.sf.okapi.steps.tokenization

Class TokenizationStep

java.lang.Object

net.sf.okapi.lib.extra.Component

net.sf.okapi.lib.extra.OkapiComponent

net.sf.okapi.lib.extra.steps.AbstractPipelineStep

net.sf.okapi.steps.tokenization.TokenizationStep

All Implemented Interfaces:

IPipelineStep, IComponent, IConfigurable, INotifiable

public class TokenizationStep
extends AbstractPipelineStep

Field Summary

Fields

Modifier and Type	Field and Description
static int	RAWTEXT

Constructor Summary

Constructors

Constructor and Description
TokenizationStep()

Method Summary

Modifier and Type	Method and Description
protected void	component_init()
boolean	exec(Object sender, String command, Object info) Sends a command for the object to react
String	getConfigInfo()
List<LexerRule>	getIdleRules()
List<ILexer>	getLexers()
protected Event	handleTextUnit(Event event) Handles the EventType.TEXT_UNIT event.
void	setConfiguration(Class<?> classRef, String configLocation)
void	setLexers(List<ILexer> lexers)

Methods inherited from class net.sf.okapi.lib.extra.steps.AbstractPipelineStep

cancel, destroy, getHelpLocation, getSourceLocale, getTargetLocale, handleCustom, handleDocumentPart, handleEndBatch, handleEndBatchItem, handleEndDocument, handleEndGroup, handleEndSubDocument, handleEndSubfilter, handleEvent, handleMultiEvent, handlePipelineParameters, handleRawDocument, handleStartBatch, handleStartBatchItem, handleStartDocument, handleStartGroup, handleStartSubDocument, handleStartSubfilter, isDone, isLastOutputStep, setLastOutputStep, setSourceLocale, setTargetLocale

Methods inherited from class net.sf.okapi.lib.extra.OkapiComponent

component_done, getParameters, getParameters, getParametersClassName, setParameters

Methods inherited from class net.sf.okapi.lib.extra.Component

getDescription, getName, setDescription, setName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.sf.okapi.common.pipeline.IPipelineStep

getDescription, getName, getParameters, setParameters

Field Detail

RAWTEXT

public static final int RAWTEXT

See Also:

Constant Field Values

Constructor Detail

TokenizationStep

public TokenizationStep()

Method Detail

setConfiguration

public void setConfiguration(Class<?> classRef,
                             String configLocation)

component_init

protected void component_init()

Specified by:

component_init in class OkapiComponent

exec

public boolean exec(Object sender,
                    String command,
                    Object info)

Description copied from interface: INotifiable

Sends a command for the object to react

Specified by:

exec in interface INotifiable

Overrides:

exec in class Component

command - a string identifying the command

info - command-specific object

Returns:

true if command has been executed and doesn't need to be sent on

handleTextUnit

protected Event handleTextUnit(Event event)

Description copied from class: AbstractPipelineStep

Handles the EventType.TEXT_UNIT event.

Overrides:

handleTextUnit in class AbstractPipelineStep

Parameters:

event - the event itself.

getIdleRules

public List<LexerRule> getIdleRules()

setLexers

public void setLexers(List<ILexer> lexers)

getLexers

public List<ILexer> getLexers()

getConfigInfo

public String getConfigInfo()