Package org.apache.parquet.format

Class SizeStatistics

    • Field Detail

      • unencoded_byte_array_data_bytes

        public long unencoded_byte_array_data_bytes
        The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

      • repetition_level_histogram

        public List<Long> repetition_level_histogram
        When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

      • definition_level_histogram

        public List<Long> definition_level_histogram
        Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

    • Constructor Detail

      • SizeStatistics

        public SizeStatistics()

      • SizeStatistics

        public SizeStatistics​(SizeStatistics other)
        Performs a deep copy on other.

    • Method Detail

      • getUnencoded_byte_array_data_bytes

        public long getUnencoded_byte_array_data_bytes()
        The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

      • setUnencoded_byte_array_data_bytes

        public SizeStatistics setUnencoded_byte_array_data_bytes​(long unencoded_byte_array_data_bytes)
        The number of physical bytes stored for BYTE_ARRAY data values assuming no encoding. This is exclusive of the bytes needed to store the length of each byte array. In other words, this field is equivalent to the `(size of PLAIN-ENCODING the byte array values) - (4 bytes * number of values written)`. To determine unencoded sizes of other types readers can use schema information multiplied by the number of non-null and null values. The number of null/non-null values can be inferred from the histograms below. For example, if a column chunk is dictionary-encoded with dictionary ["a", "bc", "cde"], and a data page contains the indices [0, 0, 1, 2], then this value for that data page should be 7 (1 + 1 + 2 + 3). This field should only be set for types that use BYTE_ARRAY as their physical type.

      • unsetUnencoded_byte_array_data_bytes

        public void unsetUnencoded_byte_array_data_bytes()

      • isSetUnencoded_byte_array_data_bytes

        public boolean isSetUnencoded_byte_array_data_bytes()
        Returns true if field unencoded_byte_array_data_bytes is set (has been assigned a value) and false otherwise

      • setUnencoded_byte_array_data_bytesIsSet

        public void setUnencoded_byte_array_data_bytesIsSet​(boolean value)

      • getRepetition_level_histogramSize

        public int getRepetition_level_histogramSize()

      • getRepetition_level_histogramIterator

        public Iterator<Long> getRepetition_level_histogramIterator()

      • addToRepetition_level_histogram

        public void addToRepetition_level_histogram​(long elem)

      • getRepetition_level_histogram

        public List<Long> getRepetition_level_histogram()
        When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

      • setRepetition_level_histogram

        public SizeStatistics setRepetition_level_histogram​(List<Long> repetition_level_histogram)
        When present, there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) where each element represents the number of times the repetition level was observed in the data. This field may be omitted if max_repetition_level is 0 without loss of information.

      • unsetRepetition_level_histogram

        public void unsetRepetition_level_histogram()

      • isSetRepetition_level_histogram

        public boolean isSetRepetition_level_histogram()
        Returns true if field repetition_level_histogram is set (has been assigned a value) and false otherwise

      • setRepetition_level_histogramIsSet

        public void setRepetition_level_histogramIsSet​(boolean value)

      • getDefinition_level_histogramSize

        public int getDefinition_level_histogramSize()

      • getDefinition_level_histogramIterator

        public Iterator<Long> getDefinition_level_histogramIterator()

      • addToDefinition_level_histogram

        public void addToDefinition_level_histogram​(long elem)

      • getDefinition_level_histogram

        public List<Long> getDefinition_level_histogram()
        Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

      • setDefinition_level_histogram

        public SizeStatistics setDefinition_level_histogram​(List<Long> definition_level_histogram)
        Same as repetition_level_histogram except for definition levels. This field may be omitted if max_definition_level is 0 or 1 without loss of information.

      • unsetDefinition_level_histogram

        public void unsetDefinition_level_histogram()

      • isSetDefinition_level_histogram

        public boolean isSetDefinition_level_histogram()
        Returns true if field definition_level_histogram is set (has been assigned a value) and false otherwise

      • setDefinition_level_histogramIsSet

        public void setDefinition_level_histogramIsSet​(boolean value)

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object

      • read

        public void read​(org.apache.thrift.protocol.TProtocol iprot)
          throws org.apache.thrift.TException
        Specified by:
        read in interface org.apache.thrift.TSerializable
        Throws:
        org.apache.thrift.TException

      • write

        public void write​(org.apache.thrift.protocol.TProtocol oprot)
           throws org.apache.thrift.TException
        Specified by:
        write in interface org.apache.thrift.TSerializable
        Throws:
        org.apache.thrift.TException

      • validate

        public void validate()
              throws org.apache.thrift.TException
        Throws:
        org.apache.thrift.TException