org.apache.carbondata.spark.vectorreader

Class VectorizedCarbonRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<Void,T>

org.apache.carbondata.hadoop.AbstractRecordReader<Object>

org.apache.carbondata.spark.vectorreader.VectorizedCarbonRecordReader

All Implemented Interfaces:

Closeable, AutoCloseable

public class VectorizedCarbonRecordReader
extends AbstractRecordReader<Object>

A specialized RecordReader that reads into InternalRows or ColumnarBatches directly using the carbondata column APIs and fills the data directly into columns.

Field Summary

Fields inherited from class org.apache.carbondata.hadoop.AbstractRecordReader

rowCount

Constructor Summary

Constructors

Constructor and Description
VectorizedCarbonRecordReader(org.apache.carbondata.core.scan.model.QueryModel queryModel, InputMetricsStats inputMetricsStats, String enableBatch)

Method Summary

Modifier and Type	Method and Description
void	close()
void	enableReturningBatches()
Void	getCurrentKey()
Object	getCurrentValue()
float	getProgress()
void	initBatch(org.apache.spark.memory.MemoryMode memMode, org.apache.spark.sql.types.StructType partitionColumns, org.apache.spark.sql.catalyst.InternalRow partitionValues) Returns the ColumnarBatch object that will be used for all rows returned by this reader.
void	initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext) Implementation of RecordReader API.
boolean	nextKeyValue()

Methods inherited from class org.apache.carbondata.hadoop.AbstractRecordReader

logStatistics

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorizedCarbonRecordReader

public VectorizedCarbonRecordReader(org.apache.carbondata.core.scan.model.QueryModel queryModel,
                                    InputMetricsStats inputMetricsStats,
                                    String enableBatch)

Method Detail

enableReturningBatches

public void enableReturningBatches()

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                       org.apache.hadoop.mapreduce.TaskAttemptContext taskAttemptContext)
                throws IOException,
                       InterruptedException,
                       UnsupportedOperationException

Implementation of RecordReader API.

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

InterruptedException

UnsupportedOperationException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

InterruptedException

getCurrentValue

public Object getCurrentValue()
                       throws IOException,
                              InterruptedException

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

InterruptedException

getCurrentKey

public Void getCurrentKey()
                   throws IOException,
                          InterruptedException

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

InterruptedException

getProgress

public float getProgress()
                  throws IOException,
                         InterruptedException

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<Void,Object>

Throws:

IOException

InterruptedException

initBatch

public void initBatch(org.apache.spark.memory.MemoryMode memMode,
                      org.apache.spark.sql.types.StructType partitionColumns,
                      org.apache.spark.sql.catalyst.InternalRow partitionValues)

Returns the ColumnarBatch object that will be used for all rows returned by this reader. This object is reused. Calling this enables the vectorized reader. This should be called before any calls to nextKeyValue/nextBatch.