org.apache.beam.sdk.io.parquet

Class ParquetIO.ReadFiles

java.lang.Object

<any>

org.apache.beam.sdk.io.parquet.ParquetIO.ReadFiles

Enclosing class:

ParquetIO

public abstract static class ParquetIO.ReadFiles
extends <any>

Implementation of ParquetIO.readFiles(Schema).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParquetIO.ReadFiles.BlockTracker

Constructor Summary

Constructors

Constructor and Description
ReadFiles()

Method Summary

Modifier and Type	Method and Description
PCollection<GenericRecord>	expand(PCollection<FileIO.ReadableFile> input)
void	populateDisplayData(DisplayData.Builder builder)
ParquetIO.ReadFiles	withAvroDataModel(GenericData model) Define the Avro data model; see AvroParquetReader.Builder#withDataModel(GenericData).
ParquetIO.ReadFiles	withBeamSchemas(boolean inferBeamSchema)
ParquetIO.ReadFiles	withConfiguration(Configuration configuration) Specify Hadoop configuration for ParquetReader.
ParquetIO.ReadFiles	withConfiguration(java.util.Map<java.lang.String,java.lang.String> configuration) Specify Hadoop configuration for ParquetReader.
ParquetIO.ReadFiles	withoutSplit() Deprecated. This method may currently be used to opt-out of the default, splittable, behavior. However, this will be removed in a future release assuming no issues are discovered.
ParquetIO.ReadFiles	withProjection(Schema projectionSchema, Schema encoderSchema)
ParquetIO.ReadFiles	withSplit() Deprecated. as of version 2.35.0. Splittable reading is enabled by default.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ReadFiles

public ReadFiles()

Method Detail

withAvroDataModel

public ParquetIO.ReadFiles withAvroDataModel(GenericData model)

Define the Avro data model; see AvroParquetReader.Builder#withDataModel(GenericData).

withProjection

public ParquetIO.ReadFiles withProjection(Schema projectionSchema,
                                          Schema encoderSchema)

withConfiguration

public ParquetIO.ReadFiles withConfiguration(java.util.Map<java.lang.String,java.lang.String> configuration)

Specify Hadoop configuration for ParquetReader.

withConfiguration

public ParquetIO.ReadFiles withConfiguration(Configuration configuration)

Specify Hadoop configuration for ParquetReader.

withBeamSchemas

@Experimental(value=SCHEMAS)
public ParquetIO.ReadFiles withBeamSchemas(boolean inferBeamSchema)

withSplit

@Deprecated
public ParquetIO.ReadFiles withSplit()

Deprecated. as of version 2.35.0. Splittable reading is enabled by default.

Enable the Splittable reading.

withoutSplit

@Deprecated
public ParquetIO.ReadFiles withoutSplit()

Deprecated. This method may currently be used to opt-out of the default, splittable, behavior. However, this will be removed in a future release assuming no issues are discovered.

Disable the Splittable reading.

expand

public PCollection<GenericRecord> expand(PCollection<FileIO.ReadableFile> input)

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)