org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<TableRow>>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.Read

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

BigQueryIO

public abstract static class BigQueryIO.Read
extends PTransform<PBegin,PCollection<TableRow>>

Implementation of BigQueryIO.read().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Read()

Method Summary

Modifier and Type	Method and Description
PCollection<TableRow>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
BigQueryIO.Read	from(java.lang.String tableSpec) Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
BigQueryIO.Read	from(TableReference table) Read from table specified by a TableReference.
BigQueryIO.Read	from(ValueProvider<java.lang.String> tableSpec) Same as from(String), but with a ValueProvider.
BigQueryIO.Read	fromQuery(java.lang.String query) Reads results received after executing the given query.
BigQueryIO.Read	fromQuery(ValueProvider<java.lang.String> query) Same as fromQuery(String), but with a ValueProvider.
protected Coder<TableRow>	getDefaultOutputCoder() Returns the default Coder to use for the output of this single-output PTransform.
TableReference	getTable() Returns the table to read, or null if reading from a query instead.
ValueProvider<TableReference>	getTableProvider() Returns the table to read, or null if reading from a query instead.
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
BigQueryIO.Read	usingStandardSql() Enables BigQuery's Standard SQL dialect when reading from a query.
void	validate(PipelineOptions options) Called before running the Pipeline to verify this transform is fully and correctly specified.
BigQueryIO.Read	withoutResultFlattening() Disable flattening of query results.
BigQueryIO.Read	withoutValidation() Disable validation that the table exists or the query succeeds prior to pipeline submission.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Read

public Read()

Method Detail

from

public BigQueryIO.Read from(java.lang.String tableSpec)

Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.

from

public BigQueryIO.Read from(ValueProvider<java.lang.String> tableSpec)

Same as from(String), but with a ValueProvider.

fromQuery

public BigQueryIO.Read fromQuery(java.lang.String query)

Reads results received after executing the given query.

By default, the query results will be flattened -- see "flattenResults" in the Jobs documentation for more information. To disable flattening, use withoutResultFlattening().

By default, the query will use BigQuery's legacy SQL dialect. To use the BigQuery Standard SQL dialect, use usingStandardSql().

fromQuery

public BigQueryIO.Read fromQuery(ValueProvider<java.lang.String> query)

Same as fromQuery(String), but with a ValueProvider.

from

public BigQueryIO.Read from(TableReference table)

Read from table specified by a TableReference.

withoutValidation

public BigQueryIO.Read withoutValidation()

Disable validation that the table exists or the query succeeds prior to pipeline submission. Basic validation (such as ensuring that a query or table is specified) still occurs.

withoutResultFlattening

public BigQueryIO.Read withoutResultFlattening()

Disable flattening of query results.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

usingStandardSql

public BigQueryIO.Read usingStandardSql()

Enables BigQuery's Standard SQL dialect when reading from a query.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

validate

public void validate(PipelineOptions options)

Description copied from class: PTransform

Called before running the Pipeline to verify this transform is fully and correctly specified.

By default, does nothing.

Overrides:

validate in class PTransform<PBegin,PCollection<TableRow>>

expand

public PCollection<TableRow> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<TableRow>>

getDefaultOutputCoder

protected Coder<TableRow> getDefaultOutputCoder()

Description copied from class: PTransform

Returns the default Coder to use for the output of this single-output PTransform.

By default, always throws

Overrides:

getDefaultOutputCoder in class PTransform<PBegin,PCollection<TableRow>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PBegin,PCollection<TableRow>>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

getTableProvider

@Nullable
public ValueProvider<TableReference> getTableProvider()

Returns the table to read, or null if reading from a query instead.

getTable

@Nullable
public TableReference getTable()

Returns the table to read, or null if reading from a query instead.