org.apache.beam.sdk.extensions.sql.meta.provider.text

Class TextTable

java.lang.Object

org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

org.apache.beam.sdk.extensions.sql.meta.provider.text.TextTable

All Implemented Interfaces:

java.io.Serializable, BeamSqlTable

@Internal
public class TextTable
extends BaseBeamTable

TextTable is a BeamSqlTable that reads text files and converts them according to the specified format.

Support formats are "csv" and "lines".

CSVFormat itself has many dialects, check its javadoc for more info.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

schema

Constructor Summary

Constructors

Constructor and Description
TextTable(Schema schema, java.lang.String filePattern, PTransform<PCollection<java.lang.String>,PCollection<Row>> readConverter, PTransform<PCollection<Row>,PCollection<java.lang.String>> writeConverter) Text table with the specified read and write transforms.

Method Summary

Modifier and Type	Method and Description
PCollection<Row>	buildIOReader(PBegin begin) create a PCollection<Row> from source.
PDone	buildIOWriter(PCollection<Row> input) create a IO.write() instance to write to target.
java.lang.String	getFilePattern()
BeamTableStatistics	getRowCount(PipelineOptions options) Estimates the number of rows or the rate for unbounded Tables.
PCollection.IsBounded	isBounded() Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).

Methods inherited from class org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

getSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextTable

public TextTable(Schema schema,
                 java.lang.String filePattern,
                 PTransform<PCollection<java.lang.String>,PCollection<Row>> readConverter,
                 PTransform<PCollection<Row>,PCollection<java.lang.String>> writeConverter)

Text table with the specified read and write transforms.

Method Detail

getFilePattern

public java.lang.String getFilePattern()

getRowCount

public BeamTableStatistics getRowCount(PipelineOptions options)

Description copied from interface: BeamSqlTable

Estimates the number of rows or the rate for unbounded Tables.

isBounded

public PCollection.IsBounded isBounded()

Description copied from interface: BeamSqlTable

Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).

buildIOReader

public PCollection<Row> buildIOReader(PBegin begin)

Description copied from interface: BeamSqlTable

create a PCollection<Row> from source.

buildIOWriter

public PDone buildIOWriter(PCollection<Row> input)

Description copied from interface: BeamSqlTable

create a IO.write() instance to write to target.