Module tools.jackson.databind

Package tools.jackson.databind.util

Class TokenBuffer

java.lang.Object

tools.jackson.core.JsonGenerator

tools.jackson.databind.util.TokenBuffer

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable, Versioned

public class TokenBuffer
extends JsonGenerator

Utility class used for efficient storage of JsonToken sequences, needed for temporary buffering. Space efficient for different sequence lengths (especially so for smaller ones; but not significantly less efficient for larger), highly efficient for linear iteration and appending. Implemented as segmented/chunked linked list of tokens; only modifications are via appends.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static final class	TokenBuffer.Parser
protected static final class	TokenBuffer.Segment	Individual segment of TokenBuffer that can store up to 16 tokens (limited by 4 bits per token type marker requirement).

Field Summary

Fields

Modifier and Type	Field	Description
protected int	_appendAt	Offset within last segment,
protected boolean	_closed
protected TokenBuffer.Segment	_first	First segment, for contents this buffer has
protected boolean	_forceBigDecimal	Flag set during construction, if use of BigDecimal is to be forced on all floating-point values.
protected boolean	_hasNativeId	Do we currently have a native type or object id buffered?
protected boolean	_hasNativeObjectIds
protected boolean	_hasNativeTypeIds
protected TokenBuffer.Segment	_last	Last segment of this buffer, one that is used for appending more tokens
protected boolean	_mayHaveNativeIds
protected Object	_objectId	If native object ids supported, this is the id for following value (or first token of one) to be written.
protected ObjectWriteContext	_objectWriteContext
protected TokenStreamContext	_parentContext	Parse context from "parent" parser (one from which content to buffer is read, if specified).
protected final StreamReadConstraints	_streamReadConstraints
protected int	_streamWriteFeatures	Bit flag composed of bits that indicate which StreamWriteFeatures are enabled.
protected SimpleStreamWriteContext	_tokenWriteContext
protected Object	_typeId	If native type ids supported, this is the id for following value (or first token of one) to be written.
protected static final JacksonFeatureSet<StreamWriteCapability>	BOGUS_WRITE_CAPABILITIES
protected static final int	DEFAULT_STREAM_WRITE_FEATURES

Constructor Summary

Constructors

Modifier	Constructor	Description
	TokenBuffer(boolean hasNativeIds)	Deprecated.
protected	TokenBuffer(JsonParser p, ObjectReadContext ctxt)
	TokenBuffer(ObjectWriteContext writeContext, boolean hasNativeIds)

Method Summary

Modifier and Type	Method	Description
protected final void	_appendEndMarker(JsonToken type)	Specialized method used for appending a structural end Object/Array marker
protected final void	_appendName(Object value)
protected final void	_appendStartMarker(JsonToken type)	Specialized method used for appending a structural start Object/Array marker
protected final void	_appendValue(JsonToken type)	Method used for appending token known to represent a "simple" scalar value where token is the only information
protected final void	_appendValue(JsonToken type, Object value)	Method used for appending token known to represent a scalar value where there is additional content (text, number) beyond type token
protected void	_copyBufferContents(JsonParser p)
protected <T> T	_reportUnsupportedOperation()
TokenBuffer	append(TokenBuffer other)	Helper method that will append contents of given buffer into this buffer.
JsonParser	asParser()	Method used to create a JsonParser that can read contents stored in this buffer.
JsonParser	asParser(ObjectReadContext readCtxt)	Method used to create a JsonParser that can read contents stored in this buffer.
JsonParser	asParser(ObjectReadContext readCtxt, JsonParser p0)
JsonParser	asParserOnFirstToken(ObjectReadContext readCtxt)	Same as:
JsonParser	asParserOnFirstToken(ObjectReadContext readCtxt, JsonParser src)
void	assignCurrentValue(Object v)
boolean	canWriteObjectId()
boolean	canWriteTypeId()
void	close()
JsonGenerator	configure(StreamWriteFeature f, boolean state)
void	copyCurrentEvent(JsonParser p)
void	copyCurrentStructure(JsonParser p)
Object	currentValue()
TokenBuffer	deserialize(JsonParser p, DeserializationContext ctxt)	Helper method used by standard deserializer.
JsonToken	firstToken()
void	flush()
static TokenBuffer	forBuffering(JsonParser p, ObjectReadContext ctxt)	Specialized factory method used when we are specifically buffering contents of a token stream for further processing.
TokenBuffer	forceUseOfBigDecimal(boolean b)
static TokenBuffer	forGeneration()	Specialized factory method used when we are generating token stream for further processing without tokens coming from specific input token stream.
CharacterEscapes	getCharacterEscapes()
int	getHighestNonEscapedChar()
PrettyPrinter	getPrettyPrinter()
FormatSchema	getSchema()
boolean	has(StreamWriteCapability capability)
boolean	isClosed()
boolean	isEmpty()	Accessor for checking whether this buffer has one or more tokens or not.
boolean	isEnabled(StreamWriteFeature f)
ObjectWriteContext	objectWriteContext()
TokenBuffer	overrideParentContext(TokenStreamContext ctxt)	Method that allows explicitly specifying parent parse context to associate with contents of this buffer.
void	serialize(JsonGenerator gen)	Helper method that will write all contents of this buffer using given JsonGenerator.
JacksonFeatureSet<StreamWriteCapability>	streamWriteCapabilities()
TokenStreamContext	streamWriteContext()
int	streamWriteFeatures()
int	streamWriteOutputBuffered()
Object	streamWriteOutputTarget()
String	toString()
Version	version()
JsonGenerator	writeBinary(Base64Variant b64variant, byte[] data, int offset, int len)
int	writeBinary(Base64Variant b64variant, InputStream data, int dataLength)	Although we could support this method, it does not necessarily make sense: we cannot make good use of streaming because buffer must hold all the data.
JsonGenerator	writeBoolean(boolean state)
JsonGenerator	writeEmbeddedObject(Object object)
final JsonGenerator	writeEndArray()
final JsonGenerator	writeEndObject()
final JsonGenerator	writeName(String name)
JsonGenerator	writeName(SerializableString name)
JsonGenerator	writeNull()
JsonGenerator	writeNumber(double d)
JsonGenerator	writeNumber(float f)
JsonGenerator	writeNumber(int i)
JsonGenerator	writeNumber(long l)
JsonGenerator	writeNumber(short i)
JsonGenerator	writeNumber(String encodedValue)
void	writeNumber(String encodedValue, boolean isInteger)	Write method that can be used for custom numeric types that can not be (easily?)
JsonGenerator	writeNumber(BigDecimal dec)
JsonGenerator	writeNumber(BigInteger v)
JsonGenerator	writeObjectId(Object id)
JsonGenerator	writePOJO(Object value)
JsonGenerator	writePropertyId(long id)
JsonGenerator	writeRaw(char c)
JsonGenerator	writeRaw(char[] text, int offset, int len)
JsonGenerator	writeRaw(String text)
JsonGenerator	writeRaw(String text, int offset, int len)
JsonGenerator	writeRaw(SerializableString text)
JsonGenerator	writeRawUTF8String(byte[] text, int offset, int length)
JsonGenerator	writeRawValue(char[] text, int offset, int len)
JsonGenerator	writeRawValue(String text)
JsonGenerator	writeRawValue(String text, int offset, int len)
final JsonGenerator	writeStartArray()
final JsonGenerator	writeStartArray(Object forValue)
final JsonGenerator	writeStartArray(Object forValue, int len)
final JsonGenerator	writeStartObject()
JsonGenerator	writeStartObject(Object forValue)
JsonGenerator	writeStartObject(Object forValue, int size)
JsonGenerator	writeString(char[] text, int offset, int len)
JsonGenerator	writeString(Reader reader, int len)
JsonGenerator	writeString(String text)
JsonGenerator	writeString(SerializableString text)
JsonGenerator	writeTree(TreeNode node)
JsonGenerator	writeTypeId(Object id)
JsonGenerator	writeUTF8String(byte[] text, int offset, int length)

Methods inherited from class tools.jackson.core.JsonGenerator

_constructWriteException, _constructWriteException, _constructWriteException, _constructWriteException, _copyCurrentContents, _copyCurrentContentsExact, _copyCurrentFloatValue, _copyCurrentFloatValueExact, _copyCurrentIntValue, _copyCurrentStringValue, _reportArgumentError, _reportError, _reportUnsupportedOperation, _verifyOffsets, _wrapIOFailure, _writeTypePrefixUsingNative, _writeTypePrefixUsingWrapper, canOmitProperties, copyCurrentEventExact, copyCurrentStructureExact, setCharacterEscapes, streamWriteConstraints, writeArray, writeArray, writeArray, writeArray, writeArrayPropertyStart, writeBinary, writeBinary, writeBinary, writeBinaryProperty, writeBooleanProperty, writeNullProperty, writeNumber, writeNumberProperty, writeNumberProperty, writeNumberProperty, writeNumberProperty, writeNumberProperty, writeNumberProperty, writeNumberProperty, writeObjectPropertyStart, writeObjectRef, writeOmittedProperty, writePOJOProperty, writeRawValue, writeStringProperty, writeTypePrefix, writeTypeSuffix

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

DEFAULT_STREAM_WRITE_FEATURES

protected static final int DEFAULT_STREAM_WRITE_FEATURES

BOGUS_WRITE_CAPABILITIES

protected static final JacksonFeatureSet<StreamWriteCapability> BOGUS_WRITE_CAPABILITIES

_parentContext

protected TokenStreamContext _parentContext

Parse context from "parent" parser (one from which content to buffer is read, if specified). Used, if available, when reading content, to present full context as if content was read from the original parser: this is useful in error reporting and sometimes processing as well.

_streamWriteFeatures

protected int _streamWriteFeatures

Bit flag composed of bits that indicate which StreamWriteFeatures are enabled.

NOTE: most features have no effect on this class

_streamReadConstraints

protected final StreamReadConstraints _streamReadConstraints

_closed

protected boolean _closed

_hasNativeTypeIds

protected boolean _hasNativeTypeIds

_hasNativeObjectIds

protected boolean _hasNativeObjectIds

_mayHaveNativeIds

protected boolean _mayHaveNativeIds

_forceBigDecimal

protected boolean _forceBigDecimal

Flag set during construction, if use of BigDecimal is to be forced on all floating-point values.

_first

protected TokenBuffer.Segment _first

First segment, for contents this buffer has

_last

protected TokenBuffer.Segment _last

Last segment of this buffer, one that is used for appending more tokens

_appendAt

protected int _appendAt

Offset within last segment,

_typeId

protected Object _typeId

If native type ids supported, this is the id for following value (or first token of one) to be written.

_objectId

protected Object _objectId

If native object ids supported, this is the id for following value (or first token of one) to be written.

_hasNativeId

protected boolean _hasNativeId

Do we currently have a native type or object id buffered?

_tokenWriteContext

protected SimpleStreamWriteContext _tokenWriteContext

_objectWriteContext

protected ObjectWriteContext _objectWriteContext

Since:

3.0

Constructor Details

TokenBuffer

@Deprecated
public TokenBuffer(boolean hasNativeIds)

Deprecated.

Parameters:

hasNativeIds - Whether resulting JsonParser (if created) is considered to support native type and object ids

TokenBuffer

public TokenBuffer(ObjectWriteContext writeContext,
 boolean hasNativeIds)

Since:

3.0

TokenBuffer

protected TokenBuffer(JsonParser p,
 ObjectReadContext ctxt)

Method Details

forGeneration

public static TokenBuffer forGeneration()

Specialized factory method used when we are generating token stream for further processing without tokens coming from specific input token stream.

Since:

3.0

forBuffering

public static TokenBuffer forBuffering(JsonParser p,
 ObjectReadContext ctxt)

Specialized factory method used when we are specifically buffering contents of a token stream for further processing.

Since:

3.0

overrideParentContext

public TokenBuffer overrideParentContext(TokenStreamContext ctxt)

Method that allows explicitly specifying parent parse context to associate with contents of this buffer. Usually context is assigned at construction, based on given parser; but it is not always available, and may not contain intended context.

forceUseOfBigDecimal

public TokenBuffer forceUseOfBigDecimal(boolean b)

asParser

public JsonParser asParser()

Method used to create a JsonParser that can read contents stored in this buffer. Will create an "empty" read context (see ObjectReadContext.empty() which often is not what you want.

Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

Returns:

Parser that can be used for reading contents stored in this buffer

asParser

public JsonParser asParser(ObjectReadContext readCtxt)

Method used to create a JsonParser that can read contents stored in this buffer.

Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

Parameters:

readCtxt - Active read context to use.

Returns:

Parser that can be used for reading contents stored in this buffer

asParser

public JsonParser asParser(ObjectReadContext readCtxt,
 JsonParser p0)

Parameters:

p0 - Parser to use for accessing source information like location, streamReadConstraints

asParserOnFirstToken

public JsonParser asParserOnFirstToken(ObjectReadContext readCtxt)
                                throws JacksonException

Same as:

  JsonParser p = asParser(readCtxt);
  p.nextToken();
  return p;

Throws:

JacksonException

asParserOnFirstToken

public JsonParser asParserOnFirstToken(ObjectReadContext readCtxt,
 JsonParser src)
                                throws JacksonException

Throws:

JacksonException

version

public Version version()

Specified by:

version in interface Versioned

Specified by:

version in class JsonGenerator

getCharacterEscapes

public CharacterEscapes getCharacterEscapes()

Overrides:

getCharacterEscapes in class JsonGenerator

getHighestNonEscapedChar

public int getHighestNonEscapedChar()

Overrides:

getHighestNonEscapedChar in class JsonGenerator

getPrettyPrinter

public PrettyPrinter getPrettyPrinter()

Overrides:

getPrettyPrinter in class JsonGenerator

getSchema

public FormatSchema getSchema()

Overrides:

getSchema in class JsonGenerator

firstToken

public JsonToken firstToken()

isEmpty

public boolean isEmpty()

Accessor for checking whether this buffer has one or more tokens or not.

Returns:

True if this buffer instance has no tokens

Since:

2.13

append

public TokenBuffer append(TokenBuffer other)

Helper method that will append contents of given buffer into this buffer. Not particularly optimized; can be made faster if there is need.

Returns:

This buffer

serialize

public void serialize(JsonGenerator gen)
               throws JacksonException

Helper method that will write all contents of this buffer using given JsonGenerator.

Note: this method would be enough to implement ValueSerializer for TokenBuffer type; but we cannot have upwards references (from core to mapper package); and as such we also cannot take second argument.

Throws:

JacksonException

deserialize

public TokenBuffer deserialize(JsonParser p,
 DeserializationContext ctxt)
                        throws JacksonException

Helper method used by standard deserializer.

Throws:

JacksonException

toString

public String toString()

Overrides:

toString in class Object

streamWriteContext

public TokenStreamContext streamWriteContext()

Specified by:

streamWriteContext in class JsonGenerator

currentValue

public Object currentValue()

Specified by:

currentValue in class JsonGenerator

assignCurrentValue

public void assignCurrentValue(Object v)

Specified by:

assignCurrentValue in class JsonGenerator

objectWriteContext

public ObjectWriteContext objectWriteContext()

Specified by:

objectWriteContext in class JsonGenerator

configure

public JsonGenerator configure(StreamWriteFeature f,
 boolean state)

Specified by:

configure in class JsonGenerator

isEnabled

public boolean isEnabled(StreamWriteFeature f)

Specified by:

isEnabled in class JsonGenerator

streamWriteFeatures

public int streamWriteFeatures()

Specified by:

streamWriteFeatures in class JsonGenerator

streamWriteCapabilities

public JacksonFeatureSet<StreamWriteCapability> streamWriteCapabilities()

Specified by:

streamWriteCapabilities in class JsonGenerator

has

public boolean has(StreamWriteCapability capability)

Specified by:

has in class JsonGenerator

flush

public void flush()

Specified by:

flush in interface Flushable

Specified by:

flush in class JsonGenerator

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in class JsonGenerator

isClosed

public boolean isClosed()

Specified by:

isClosed in class JsonGenerator

streamWriteOutputTarget

public Object streamWriteOutputTarget()

Specified by:

streamWriteOutputTarget in class JsonGenerator

streamWriteOutputBuffered

public int streamWriteOutputBuffered()

Specified by:

streamWriteOutputBuffered in class JsonGenerator

writeStartArray

public final JsonGenerator writeStartArray()

Specified by:

writeStartArray in class JsonGenerator

writeStartArray

public final JsonGenerator writeStartArray(Object forValue)

Specified by:

writeStartArray in class JsonGenerator

writeStartArray

public final JsonGenerator writeStartArray(Object forValue,
 int len)

Specified by:

writeStartArray in class JsonGenerator

writeEndArray

public final JsonGenerator writeEndArray()

Specified by:

writeEndArray in class JsonGenerator

writeStartObject

public final JsonGenerator writeStartObject()

Specified by:

writeStartObject in class JsonGenerator

writeStartObject

public JsonGenerator writeStartObject(Object forValue)

Specified by:

writeStartObject in class JsonGenerator

writeStartObject

public JsonGenerator writeStartObject(Object forValue,
 int size)

Specified by:

writeStartObject in class JsonGenerator

writeEndObject

public final JsonGenerator writeEndObject()

Specified by:

writeEndObject in class JsonGenerator

writeName

public final JsonGenerator writeName(String name)

Specified by:

writeName in class JsonGenerator

writeName

public JsonGenerator writeName(SerializableString name)

Specified by:

writeName in class JsonGenerator

writePropertyId

public JsonGenerator writePropertyId(long id)

Specified by:

writePropertyId in class JsonGenerator

writeString

public JsonGenerator writeString(String text)

Specified by:

writeString in class JsonGenerator

writeString

public JsonGenerator writeString(char[] text,
 int offset,
 int len)

Specified by:

writeString in class JsonGenerator

writeString

public JsonGenerator writeString(SerializableString text)

Specified by:

writeString in class JsonGenerator

writeString

public JsonGenerator writeString(Reader reader,
 int len)

Specified by:

writeString in class JsonGenerator

writeRawUTF8String

public JsonGenerator writeRawUTF8String(byte[] text,
 int offset,
 int length)

Specified by:

writeRawUTF8String in class JsonGenerator

writeUTF8String

public JsonGenerator writeUTF8String(byte[] text,
 int offset,
 int length)

Specified by:

writeUTF8String in class JsonGenerator

writeRaw

public JsonGenerator writeRaw(String text)

Specified by:

writeRaw in class JsonGenerator

writeRaw

public JsonGenerator writeRaw(String text,
 int offset,
 int len)

Specified by:

writeRaw in class JsonGenerator

writeRaw

public JsonGenerator writeRaw(SerializableString text)

Overrides:

writeRaw in class JsonGenerator

writeRaw

public JsonGenerator writeRaw(char[] text,
 int offset,
 int len)

Specified by:

writeRaw in class JsonGenerator

writeRaw

public JsonGenerator writeRaw(char c)

Specified by:

writeRaw in class JsonGenerator

writeRawValue

public JsonGenerator writeRawValue(String text)

Specified by:

writeRawValue in class JsonGenerator

writeRawValue

public JsonGenerator writeRawValue(String text,
 int offset,
 int len)

Specified by:

writeRawValue in class JsonGenerator

writeRawValue

public JsonGenerator writeRawValue(char[] text,
 int offset,
 int len)

Specified by:

writeRawValue in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(short i)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(int i)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(long l)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(double d)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(float f)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(BigDecimal dec)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(BigInteger v)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public JsonGenerator writeNumber(String encodedValue)

Specified by:

writeNumber in class JsonGenerator

writeNumber

public void writeNumber(String encodedValue,
 boolean isInteger)
                 throws IOException

Write method that can be used for custom numeric types that can not be (easily?) converted to "standard" Java number types. Because numbers are not surrounded by double quotes, regular writeString(java.lang.String) method cannot be used; nor writeRaw(java.lang.String) because that does not properly handle value separators needed in Array or Object contexts.

Parameters:

encodedValue - Textual (possibly formatted) number representation to write

isInteger - Whether value should be considered an integer

Throws:

IOException - if there is either an underlying I/O problem or encoding issue at format layer

Since:

2.18

writeBoolean

public JsonGenerator writeBoolean(boolean state)

Specified by:

writeBoolean in class JsonGenerator

writeNull

public JsonGenerator writeNull()

Specified by:

writeNull in class JsonGenerator

writePOJO

public JsonGenerator writePOJO(Object value)

Specified by:

writePOJO in class JsonGenerator

writeTree

public JsonGenerator writeTree(TreeNode node)

Specified by:

writeTree in class JsonGenerator

writeBinary

public JsonGenerator writeBinary(Base64Variant b64variant,
 byte[] data,
 int offset,
 int len)

Specified by:

writeBinary in class JsonGenerator

writeBinary

public int writeBinary(Base64Variant b64variant,
 InputStream data,
 int dataLength)

Although we could support this method, it does not necessarily make sense: we cannot make good use of streaming because buffer must hold all the data. Because of this, currently this will simply throw UnsupportedOperationException

Specified by:

writeBinary in class JsonGenerator

canWriteTypeId

public boolean canWriteTypeId()

Overrides:

canWriteTypeId in class JsonGenerator

canWriteObjectId

public boolean canWriteObjectId()

Overrides:

canWriteObjectId in class JsonGenerator

writeTypeId

public JsonGenerator writeTypeId(Object id)

Overrides:

writeTypeId in class JsonGenerator

writeObjectId

public JsonGenerator writeObjectId(Object id)

Overrides:

writeObjectId in class JsonGenerator

writeEmbeddedObject

public JsonGenerator writeEmbeddedObject(Object object)

Overrides:

writeEmbeddedObject in class JsonGenerator

copyCurrentEvent

public void copyCurrentEvent(JsonParser p)

Overrides:

copyCurrentEvent in class JsonGenerator

copyCurrentStructure

public void copyCurrentStructure(JsonParser p)

Overrides:

copyCurrentStructure in class JsonGenerator

_copyBufferContents

protected void _copyBufferContents(JsonParser p)

_appendValue

protected final void _appendValue(JsonToken type)

Method used for appending token known to represent a "simple" scalar value where token is the only information

_appendValue

protected final void _appendValue(JsonToken type,
 Object value)

Method used for appending token known to represent a scalar value where there is additional content (text, number) beyond type token

_appendName

protected final void _appendName(Object value)

_appendStartMarker

protected final void _appendStartMarker(JsonToken type)

Specialized method used for appending a structural start Object/Array marker

_appendEndMarker

protected final void _appendEndMarker(JsonToken type)

Specialized method used for appending a structural end Object/Array marker

_reportUnsupportedOperation

protected <T> T _reportUnsupportedOperation()

Overrides:

_reportUnsupportedOperation in class JsonGenerator