org.apache.beam.sdk.extensions.sql.meta

Interface BeamSqlTable

All Known Implementing Classes:

AvroTable, BaseBeamTable, BeamKafkaCSVTable, BeamKafkaTable, BeamPCollectionTable, HCatalogTable, MongoDbTable, ParquetTable, SchemaBaseBeamTable, TestBoundedTable, TestTable, TestUnboundedTable, TextJsonTable, TextTable

public interface BeamSqlTable

This interface defines a Beam Sql Table.

Method Summary

Modifier and Type	Method and Description
PCollection<Row>	buildIOReader(PBegin begin) create a PCollection<Row> from source.
PCollection<Row>	buildIOReader(PBegin begin, BeamSqlTableFilter filters, java.util.List<java.lang.String> fieldNames) create a PCollection<Row> from source with predicate and/or project pushed-down.
POutput	buildIOWriter(PCollection<Row> input) create a IO.write() instance to write to target.
BeamSqlTableFilter	constructFilter(java.util.List<org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.rex.RexNode> filter) Generate an IO implementation of BeamSqlTableFilter for predicate push-down.
Schema	getSchema() Get the schema info of the table.
BeamTableStatistics	getTableStatistics(PipelineOptions options) Estimates the number of rows or the rate for unbounded Tables.
PCollection.IsBounded	isBounded() Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).
ProjectSupport	supportsProjects() Whether project push-down is supported by the IO API.

Method Detail

buildIOReader

PCollection<Row> buildIOReader(PBegin begin)

create a PCollection<Row> from source.

buildIOReader

PCollection<Row> buildIOReader(PBegin begin,
                               BeamSqlTableFilter filters,
                               java.util.List<java.lang.String> fieldNames)

create a PCollection<Row> from source with predicate and/or project pushed-down.

buildIOWriter

POutput buildIOWriter(PCollection<Row> input)

create a IO.write() instance to write to target.

constructFilter

BeamSqlTableFilter constructFilter(java.util.List<org.apache.beam.vendor.calcite.v1_20_0.org.apache.calcite.rex.RexNode> filter)

Generate an IO implementation of BeamSqlTableFilter for predicate push-down.

supportsProjects

ProjectSupport supportsProjects()

Whether project push-down is supported by the IO API.

isBounded

PCollection.IsBounded isBounded()

Whether this table is bounded (known to be finite) or unbounded (may or may not be finite).

getSchema

Schema getSchema()

Get the schema info of the table.

getTableStatistics

BeamTableStatistics getTableStatistics(PipelineOptions options)

Estimates the number of rows or the rate for unbounded Tables. If it is not possible to estimate the row count or rate it will return BeamTableStatistics.BOUNDED_UNKNOWN.