Package org.apache.beam.sdk.extensions.arrow

Class ArrowConversion

java.lang.Object

org.apache.beam.sdk.extensions.arrow.ArrowConversion

public class ArrowConversion
extends Object

Utilities to create Iterables of Beam Row instances backed by Arrow record batches.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ArrowConversion.ArrowSchemaTranslator	Converts Arrow schema to Beam row schema.
static class	ArrowConversion.RecordBatchRowIterator

Method Summary

Modifier and Type	Method	Description
static Schema	arrowSchemaFromInput(InputStream input)
static ArrowConversion.RecordBatchRowIterator	rowsFromRecordBatch(Schema schema, VectorSchemaRoot vectorSchemaRoot)	Returns a ArrowConversion.RecordBatchRowIterator backed by the Arrow record batch stored in vectorSchemaRoot.
static ArrowConversion.RecordBatchRowIterator	rowsFromSerializedRecordBatch(Schema arrowSchema, InputStream inputStream, RootAllocator allocator)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

rowsFromRecordBatch

public static ArrowConversion.RecordBatchRowIterator rowsFromRecordBatch(Schema schema,
 VectorSchemaRoot vectorSchemaRoot)

Returns a ArrowConversion.RecordBatchRowIterator backed by the Arrow record batch stored in vectorSchemaRoot.

Note this is a lazy interface. The data in the underlying Arrow buffer is not read until a field of one of the returned Rows is accessed.

rowsFromSerializedRecordBatch

public static ArrowConversion.RecordBatchRowIterator rowsFromSerializedRecordBatch(Schema arrowSchema,
 InputStream inputStream,
 RootAllocator allocator)
                                                                            throws IOException

Throws:

IOException

arrowSchemaFromInput

public static Schema arrowSchemaFromInput(InputStream input)
                                   throws IOException

Throws:

IOException