net.sf.okapi.filters.openxml

Class ContentFilter

java.lang.Object

net.sf.okapi.common.filters.AbstractFilter

net.sf.okapi.filters.abstractmarkup.AbstractMarkupFilter

net.sf.okapi.filters.openxml.ContentFilter

All Implemented Interfaces:

AutoCloseable, Iterator<Event>, IFilter

public class ContentFilter
extends AbstractMarkupFilter

Filters Microsoft Office Word, Excel, and Powerpoint Documents. OpenXML is the format of these documents.

Since OpenXML files are Zip files that contain XML documents, OpenXMLFilter handles opening and processing the zip file, and instantiates this filter to process the XML documents.

This filter extends AbstractBaseMarkupFilter, which extends AbstractBaseFilter. It uses the Jericho parser to analyze the XML files.

The filter exhibits slightly differnt behavior depending on whether the XML file is Word, Excel, Powerpoint, or a chart in Word. The tags in these files are configured in yaml configuration files that specify the behavior of the tags. These configuration files are

• wordConfiguration.yml
• excelConfiguration.yml
• powerpointConfiguration.yml
• wordChartConfiguration.yml

In Word and Powerpoint, text is always surrounded by paragraph tags <w:p> or <a:p>, which signal the beginning and end of the text unit for this filter, and are marked as TEXT_UNIT_ELEMENTs in the configuration files. Inside these are one or more text runs surrounded by <w:r> or <a:r> tags and marked as TEXT_RUN_ELEMENTS by the configuration files. The text itself occurs between text marker tags <w:t> or <a:t> tags, which are designated TEXT_MARKER_ELEMENTS by the configuration files. Tags between and including <w:r> and <w:t> (which usually include a <w:rPr> tag sequence for character style) are consolidated into a single MARKER_OPENING code. Tags between and including </w:t> and </w:r>, which sometimes include graphics tags, are consolidated into a single MARKER_CLOSING code. If there is no text between <w:r> and </w:r>, a single MARKER_PLACEHOLDER code is created for the text run. If there is no character style information, <w:r><w:t>text</w:t></w:r> is not surrounded by MARKER_OPENING or MARKER_CLOSING codes, to simplify things for translators; these are supplied by OpenXMLContentSkeletonWriter during output. The same is true for text runs marked by <a:r> and <a:t> in Powerpoint files. Excel files are simpler, and only mark text by <v>, <t>, and <text> tags in worksheet, sharedString, and comment files respectively. These tags work like TEXT_UNIT, TEXT_RUN, and TEXT_MARKER elements combined.

Field Summary

Fields inherited from interface net.sf.okapi.common.filters.IFilter

SUB_FILTER

Constructor Summary

Constructors

Constructor and Description
ContentFilter(ConditionalParameters filterParams, String partName)

Method Summary

Modifier and Type	Method and Description
void	displayOneEvent(Event event) Logs information about the event fir the log level is FINEST.
protected boolean	getBInSettingsFile()
protected TaggedFilterConfiguration	getConfig() Get the current TaggedFilterConfiguration.
ParseType	getConfigurationType()
ConditionalParameters	getFilterParameters()
String	getName() Returns name of the filter.
YamlParameters	getParameters() Gets the current parameters for this filter.
ParseType	getParseType()
protected void	handleComment(net.htmlparser.jericho.Tag tag) Treats XML comments as DocumentParts.
protected void	handleDocTypeDeclaration(net.htmlparser.jericho.Tag tag) Treats XML doc type declaratons as DocumentParts.
protected void	handleEndTag(net.htmlparser.jericho.EndTag endTag) Handles end tags.
protected void	handleMarkupDeclaration(net.htmlparser.jericho.Tag tag) Treats XML markup declaratons as DocumentParts.
protected void	handleProcessingInstruction(net.htmlparser.jericho.Tag tag) Treats XML processing instructions as DocumentParts.
protected void	handleServerCommon(net.htmlparser.jericho.Tag tag) Treats XML server common tags as DocumentParts.
protected void	handleServerCommonEscaped(net.htmlparser.jericho.Tag tag) Treats server common escaped tags as DocumentParts.
protected void	handleStartTag(net.htmlparser.jericho.StartTag startTag) Handles a start tag.
protected void	handleText(CharSequence text) Handles text.
protected void	handleXmlDeclaration(net.htmlparser.jericho.Tag tag) Treats XML declaratons as DocumentParts.
protected String	normalizeAttributeName(String attrName, String attrValue, net.htmlparser.jericho.Tag tag) Normalizes naming of attributes whose values are the encoding or a language name, so that they can be automatically changed to the output encoding and output.
protected void	setBInSettingsFile(boolean bInSettingsFile)
void	setParameters(IParameters params) Sets new parameters for this filter.
void	setUpConfig(ParseType filetype) Sets the name of the Yaml configuration file for the current file type, reads the file, and sets the parameters.
String	toString()

Methods inherited from class net.sf.okapi.filters.abstractmarkup.AbstractMarkupFilter

addCodeToCurrentTextUnit, addCodeToCurrentTextUnit, addFilterEvent, addToDocumentPart, addToTextUnit, addToTextUnit, addToTextUnit, addToTextUnit, addToTextUnit, appendToFirstSkeletonPart, canStartNewTextUnit, close, createEventBuilder, createPropertyTextUnitPlaceholder, createPropertyTextUnitPlaceholders, detectEncoding, determineTagType, endDocumentPart, endFilter, endGroup, endTextUnit, getBufferedWhiteSpace, getCurrentDocName, getDocumentPartId, getEventBuilder, getGroupIdSequence, getParsedHeader, getRuleState, getTextUnitId, handleCdataSection, handleCharacterEntity, handleDocumentPart, handleNumericEntity, hasNext, isBOM, isDocumentEncoding, isInsideTextRun, isPreserveWhitespace, isUtf8Bom, isUtf8Encoding, isWhiteSpace, next, open, open, peekTempEvent, popTempEvent, postProcessTextUnit, preProcess, setCurrentDocName, setDocumentPartId, setGroupIdSequence, setMimeType, setPreserveWhitespace, setTextUnitId, setTextUnitMimeType, setTextUnitName, setTextUnitPreserveWhitespace, setTextUnitTranslatable, setTextUnitType, startDocumentPart, startFilter, startGroup, startGroup, startTextUnit, startTextUnit, startTextUnit, startTextUnit, updateEndTagRuleState, updateStartTagRuleState

Methods inherited from class net.sf.okapi.common.filters.AbstractFilter

addConfiguration, addConfiguration, addConfiguration, addConfigurations, cancel, createEndFilterEvent, createFilterWriter, createSkeletonWriter, createStartFilterEvent, findConfiguration, getConfiguration, getConfigurations, getDisplayName, getDocumentId, getDocumentName, getEncoderManager, getEncoding, getFilterConfigurationMapper, getFilterWriter, getMimeType, getNewlineType, getParameters, getParametersClassName, getParentId, getSrcLoc, getTrgLoc, isCanceled, isGenerateSkeleton, isMultilingual, removeConfiguration, setDisplayName, setDocumentName, setEncoding, setFilterConfigurationMapper, setFilterWriter, setGenerateSkeleton, setMultilingual, setName, setNewlineType, setOptions, setParentId, setSrcLoc, setTrgLoc

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.sf.okapi.common.filters.IFilter

stream

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

ContentFilter

public ContentFilter(ConditionalParameters filterParams,
                     String partName)

Method Detail

displayOneEvent

public void displayOneEvent(Event event)

Logs information about the event fir the log level is FINEST.

Parameters:

event - event to log information about

getParseType

public ParseType getParseType()

setUpConfig

public void setUpConfig(ParseType filetype)

Sets the name of the Yaml configuration file for the current file type, reads the file, and sets the parameters.

Parameters:

filetype - type of XML in the current file

handleText

protected void handleText(CharSequence text)

Handles text. If in a text run, it ends the text run and adds the tags that were in it as a single MARKER_OPENING code. This would correspond to <w:r>...<w:t> in MSWord. It will then start a new text run anticipating </w:t>...</w:r>. If text is found that was not in a text run, i.e. it was not between text markers, it is not text to be processed by a user, so it becomes part of a new text run which will become part of a code. If the text is not in a text unit, then it is added to a document part.

Overrides:

handleText in class AbstractMarkupFilter

Parameters:

text - the text to be handled

handleStartTag

protected void handleStartTag(net.htmlparser.jericho.StartTag startTag)

Handles a start tag. TEXT_UNIT_ELEMENTs start a new TextUnit. TEXT_RUN_ELEMENTs start a new text run. TEXT_MARKER_ELEMENTS set a flag that any following text will be between text markers. ATTRIBUTES_ONLY tags have translatable text in the attributes, so within a text unit, it is added within a text run; otherwise it becomes a DocumentPart.

Overrides:

handleStartTag in class AbstractMarkupFilter

Parameters:

startTag - the start tag to process

handleEndTag

protected void handleEndTag(net.htmlparser.jericho.EndTag endTag)

Handles end tags. These either add to current text runs or end text runs or text units as appropriate.

Overrides:

handleEndTag in class AbstractMarkupFilter

Parameters:

endTag - the end tag to process

handleComment

protected void handleComment(net.htmlparser.jericho.Tag tag)

Treats XML comments as DocumentParts.

Overrides:

handleComment in class AbstractMarkupFilter

Parameters:

tag - comment tag

handleDocTypeDeclaration

protected void handleDocTypeDeclaration(net.htmlparser.jericho.Tag tag)

Treats XML doc type declaratons as DocumentParts.

Overrides:

handleDocTypeDeclaration in class AbstractMarkupFilter

Parameters:

tag - doc type declaration tag

handleMarkupDeclaration

protected void handleMarkupDeclaration(net.htmlparser.jericho.Tag tag)

Treats XML markup declaratons as DocumentParts.

Overrides:

handleMarkupDeclaration in class AbstractMarkupFilter

Parameters:

tag - markup declaration tag

handleProcessingInstruction

protected void handleProcessingInstruction(net.htmlparser.jericho.Tag tag)

Treats XML processing instructions as DocumentParts.

Overrides:

handleProcessingInstruction in class AbstractMarkupFilter

Parameters:

tag - processing instruction tag

handleServerCommon

protected void handleServerCommon(net.htmlparser.jericho.Tag tag)

Treats XML server common tags as DocumentParts.

Overrides:

handleServerCommon in class AbstractMarkupFilter

Parameters:

tag - server common tag

handleServerCommonEscaped

protected void handleServerCommonEscaped(net.htmlparser.jericho.Tag tag)

Treats server common escaped tags as DocumentParts.

Overrides:

handleServerCommonEscaped in class AbstractMarkupFilter

Parameters:

tag - server common escaped tag

handleXmlDeclaration

protected void handleXmlDeclaration(net.htmlparser.jericho.Tag tag)

Treats XML declaratons as DocumentParts.

Overrides:

handleXmlDeclaration in class AbstractMarkupFilter

Parameters:

tag - XML declaration tag

getName

public String getName()

Returns name of the filter.

Specified by:

getName in interface IFilter

Overrides:

getName in class AbstractFilter

Returns:

name of the filter

normalizeAttributeName

protected String normalizeAttributeName(String attrName,
                                        String attrValue,
                                        net.htmlparser.jericho.Tag tag)

Normalizes naming of attributes whose values are the encoding or a language name, so that they can be automatically changed to the output encoding and output. Unfortunately, this hard codes the tags to look for.

Specified by:

normalizeAttributeName in class AbstractMarkupFilter

Parameters:

attrName - name of the attribute

attrValue, - value of the attribute

tag - tag that contains the attribute

Returns:

a normalized name for the attribute

getConfigurationType

public ParseType getConfigurationType()

setBInSettingsFile

protected void setBInSettingsFile(boolean bInSettingsFile)

getBInSettingsFile

protected boolean getBInSettingsFile()

getConfig

protected TaggedFilterConfiguration getConfig()

Description copied from class: AbstractMarkupFilter

Get the current TaggedFilterConfiguration. A TaggedFilterConfiguration is the result of reading in a YAML configuration file and converting it into Java Objects.

Specified by:

getConfig in class AbstractMarkupFilter

Returns:

a TaggedFilterConfiguration

getParameters

public YamlParameters getParameters()

Description copied from interface: IFilter

Gets the current parameters for this filter.

Specified by:

getParameters in interface IFilter

Overrides:

getParameters in class AbstractFilter

Returns:

The current parameters for this filter, or null if this filter has no parameters.

setParameters

public void setParameters(IParameters params)

Description copied from interface: IFilter

Sets new parameters for this filter.

Specified by:

setParameters in interface IFilter

Overrides:

setParameters in class AbstractFilter

Parameters:

params - The new parameters to use.

getFilterParameters

public ConditionalParameters getFilterParameters()

toString

public String toString()

Overrides:

toString in class Object