org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<TableRow>>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.Read

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

BigQueryIO

public static class BigQueryIO.Read
extends PTransform<PBegin,PCollection<TableRow>>

Implementation of BigQueryIO.read().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Method Summary

Modifier and Type	Method and Description
PCollection<TableRow>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
BigQueryIO.Read	from(java.lang.String tableSpec) Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
BigQueryIO.Read	from(TableReference table) Read from table specified by a TableReference.
BigQueryIO.Read	from(ValueProvider<java.lang.String> tableSpec) Same as from(String), but with a ValueProvider.
BigQueryIO.Read	fromQuery(java.lang.String query) Reads results received after executing the given query.
BigQueryIO.Read	fromQuery(ValueProvider<java.lang.String> query) Same as fromQuery(String), but with a ValueProvider.
@Nullable TableReference	getTable() Returns the table to read, or null if reading from a query instead.
@Nullable ValueProvider<TableReference>	getTableProvider() Returns the table to read, or null if reading from a query instead.
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
BigQueryIO.Read	usingStandardSql() Enables BigQuery's Standard SQL dialect when reading from a query.
BigQueryIO.Read	withoutResultFlattening() Disable flattening of query results.
BigQueryIO.Read	withoutValidation() Disable validation that the table exists or the query succeeds prior to pipeline submission.
BigQueryIO.Read	withTemplateCompatibility() Use new template-compatible source implementation.
BigQueryIO.Read	withTestServices(BigQueryServices testServices)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

expand

public PCollection<TableRow> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<TableRow>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PBegin,PCollection<TableRow>>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

withTestServices

public BigQueryIO.Read withTestServices(BigQueryServices testServices)

getTableProvider

public @Nullable ValueProvider<TableReference> getTableProvider()

Returns the table to read, or null if reading from a query instead.

getTable

public @Nullable TableReference getTable()

Returns the table to read, or null if reading from a query instead.

from

public BigQueryIO.Read from(java.lang.String tableSpec)

Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.

from

public BigQueryIO.Read from(ValueProvider<java.lang.String> tableSpec)

Same as from(String), but with a ValueProvider.

from

public BigQueryIO.Read from(TableReference table)

Read from table specified by a TableReference.

fromQuery

public BigQueryIO.Read fromQuery(java.lang.String query)

Reads results received after executing the given query.

By default, the query results will be flattened -- see "flattenResults" in the Jobs documentation for more information. To disable flattening, use Read#withoutResultFlattening.

By default, the query will use BigQuery's legacy SQL dialect. To use the BigQuery Standard SQL dialect, use Read#usingStandardSql.

fromQuery

public BigQueryIO.Read fromQuery(ValueProvider<java.lang.String> query)

Same as fromQuery(String), but with a ValueProvider.

withoutValidation

public BigQueryIO.Read withoutValidation()

Disable validation that the table exists or the query succeeds prior to pipeline submission. Basic validation (such as ensuring that a query or table is specified) still occurs.

withoutResultFlattening

public BigQueryIO.Read withoutResultFlattening()

Disable flattening of query results.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

usingStandardSql

public BigQueryIO.Read usingStandardSql()

Enables BigQuery's Standard SQL dialect when reading from a query.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

withTemplateCompatibility

public BigQueryIO.Read withTemplateCompatibility()

Use new template-compatible source implementation.

Use new template-compatible source implementation. This implementation is compatible with repeated template invocations. It does not support dynamic work rebalancing.