org.tensorflow.op.core

Class DecodeProtoV2

java.lang.Object

org.tensorflow.op.PrimitiveOp

org.tensorflow.op.core.DecodeProtoV2

All Implemented Interfaces:

Op

@Operator
public final class DecodeProtoV2
extends PrimitiveOp

The op extracts fields from a serialized protocol buffers message into tensors.

The `decode_proto` op extracts fields from a serialized protocol buffers message into tensors. The fields in `field_names` are decoded and converted to the corresponding `output_types` if possible.

A `message_type` name must be provided to give context for the field names. The actual message descriptor can be looked up either in the linked-in descriptor pool or a filename provided by the caller using the `descriptor_source` attribute.

Each output tensor is a dense tensor. This means that it is padded to hold the largest number of repeated elements seen in the input minibatch. (The shape is also padded by one to prevent zero-sized dimensions). The actual repeat counts for each example in the minibatch can be found in the `sizes` output. In many cases the output of `decode_proto` is fed immediately into tf.squeeze if missing values are not a concern. When using tf.squeeze, always pass the squeeze dimension explicitly to avoid surprises.

For the most part, the mapping between Proto field types and TensorFlow dtypes is straightforward. However, there are a few special cases:

- A proto field that contains a submessage or group can only be converted to `DT_STRING` (the serialized submessage). This is to reduce the complexity of the API. The resulting string can be used as input to another instance of the decode_proto op.

- TensorFlow lacks support for unsigned integers. The ops represent uint64 types as a `DT_INT64` with the same twos-complement bit pattern (the obvious way). Unsigned int32 values can be represented exactly by specifying type `DT_INT64`, or using twos-complement if the caller specifies `DT_INT32` in the `output_types` attribute.

The `descriptor_source` attribute selects a source of protocol descriptors to consult when looking up `message_type`. This may be a filename containing a serialized `FileDescriptorSet` message, or the special value `local://`, in which case only descriptors linked into the code will be searched; the filename can be on any filesystem accessible to TensorFlow.

You can build a `descriptor_source` file using the `--descriptor_set_out` and `--include_imports` options to the protocol compiler `protoc`.

The `local://` database only covers descriptors linked into the code via C++ libraries, not Python imports. You can link in a proto descriptor by creating a cc_library target with alwayslink=1.

Both binary and text proto serializations are supported, and can be chosen using the `format` attribute.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DecodeProtoV2.Options Optional attributes for DecodeProtoV2

Field Summary

Fields inherited from class org.tensorflow.op.PrimitiveOp

operation

Method Summary

Modifier and Type	Method and Description
static DecodeProtoV2	create(Scope scope, Operand<String> bytes, String messageType, List<String> fieldNames, List<Class<?>> outputTypes, DecodeProtoV2.Options... options) Factory method to create a class to wrap a new DecodeProtoV2 operation to the graph.
static DecodeProtoV2.Options	descriptorSource(String descriptorSource)
static DecodeProtoV2.Options	messageFormat(String messageFormat)
static DecodeProtoV2.Options	sanitize(Boolean sanitize)
Output<Integer>	sizes() Tensor of int32 with shape `[batch_shape, len(field_names)]`.
List<Output<?>>	values() List of tensors containing values for the corresponding field.

Methods inherited from class org.tensorflow.op.PrimitiveOp

equals, hashCode, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Method Detail

create

public static DecodeProtoV2 create(Scope scope,
                                   Operand<String> bytes,
                                   String messageType,
                                   List<String> fieldNames,
                                   List<Class<?>> outputTypes,
                                   DecodeProtoV2.Options... options)

Factory method to create a class to wrap a new DecodeProtoV2 operation to the graph.

Parameters:

scope - current graph scope

bytes - Tensor of serialized protos with shape `batch_shape`.

messageType - Name of the proto message type to decode.

fieldNames - List of strings containing proto field names.

outputTypes - List of TF types to use for the respective field in field_names.

options - carries optional attributes values

Returns:

a new instance of DecodeProtoV2

descriptorSource

public static DecodeProtoV2.Options descriptorSource(String descriptorSource)

Parameters:

descriptorSource - Either the special value `local://` or a path to a file containing a serialized `FileDescriptorSet`.

messageFormat

public static DecodeProtoV2.Options messageFormat(String messageFormat)

Parameters:

messageFormat - Either `binary` or `text`.

sanitize

public static DecodeProtoV2.Options sanitize(Boolean sanitize)

Parameters:

sanitize - Whether to sanitize the result or not.

sizes

public Output<Integer> sizes()

Tensor of int32 with shape `[batch_shape, len(field_names)]`. Each entry is the number of values found for the corresponding field. Optional fields may have 0 or 1 values.

values

public List<Output<?>> values()

List of tensors containing values for the corresponding field. `values[i]` has datatype `output_types[i]` and shape `[batch_shape, max(sizes[...,i])]`.