org.apache.beam.sdk.extensions.sql.meta.provider.text

Class TextTable

java.lang.Object

org.apache.beam.sdk.extensions.sql.meta.BaseBeamTable

org.apache.beam.sdk.extensions.sql.meta.SchemaBaseBeamTable

org.apache.beam.sdk.extensions.sql.meta.provider.text.TextTable

All Implemented Interfaces:

java.io.Serializable, BeamSqlTable

Direct Known Subclasses:

TextJsonTable

@Internal
public class TextTable
extends SchemaBaseBeamTable

TextTable is a BeamSqlTable that reads text files and converts them according to the specified format.

Support formats are "csv" and "lines".

CSVFormat itself has many dialects, check its javadoc for more info.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.extensions.sql.meta.SchemaBaseBeamTable

schema

Constructor Summary

Constructors

Constructor and Description
TextTable(Schema schema, java.lang.String filePattern, PTransform<PCollection<java.lang.String>,PCollection<Row>> readConverter, PTransform<PCollection<Row>,PCollection<java.lang.String>> writeConverter) Text table with the specified read and write transforms.

Method Summary

Modifier and Type	Method and Description
PCollection<Row>	buildIOReader(PBegin begin) create a PCollection<Row> from source.
PDone	buildIOWriter(PCollection<Row> input) create a IO.write() instance to write to target.
java.lang.String	getFilePattern()
BeamTableStatistics	getTableStatistics(PipelineOptions options) Estimates the number of rows or the rate for unbounded Tables.
PCollection.IsBounded	isBounded() Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).

Methods inherited from class org.apache.beam.sdk.extensions.sql.meta.SchemaBaseBeamTable

getSchema

Methods inherited from class org.apache.beam.sdk.extensions.sql.meta.BaseBeamTable

buildIOReader, constructFilter, supportsProjects

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextTable

public TextTable(Schema schema,
                 java.lang.String filePattern,
                 PTransform<PCollection<java.lang.String>,PCollection<Row>> readConverter,
                 PTransform<PCollection<Row>,PCollection<java.lang.String>> writeConverter)

Text table with the specified read and write transforms.

Method Detail

getFilePattern

public java.lang.String getFilePattern()

getTableStatistics

public BeamTableStatistics getTableStatistics(PipelineOptions options)

Description copied from interface: BeamSqlTable

Estimates the number of rows or the rate for unbounded Tables. If it is not possible to estimate the row count or rate it will return BeamTableStatistics.BOUNDED_UNKNOWN.

Specified by:

getTableStatistics in interface BeamSqlTable

Overrides:

getTableStatistics in class BaseBeamTable

isBounded

public PCollection.IsBounded isBounded()

Description copied from interface: BeamSqlTable

Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).

buildIOReader

public PCollection<Row> buildIOReader(PBegin begin)

Description copied from interface: BeamSqlTable

create a PCollection<Row> from source.

buildIOWriter

public PDone buildIOWriter(PCollection<Row> input)

Description copied from interface: BeamSqlTable

create a IO.write() instance to write to target.