org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryIO.TypedRead<T>

java.lang.Object

<any>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead<T>

Enclosing class:

BigQueryIO

public abstract static class BigQueryIO.TypedRead<T>
extends <any>

Implementation of BigQueryIO.read(SerializableFunction).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BigQueryIO.TypedRead.Method Determines the method used to read data from BigQuery.
static class	BigQueryIO.TypedRead.QueryPriority An enumeration type for the priority of a query.

Constructor Summary

Constructors

Constructor and Description
TypedRead()

Method Summary

Modifier and Type	Method and Description
PCollection<T>	expand(PBegin input)
BigQueryIO.TypedRead<T>	from(java.lang.String tableSpec) See BigQueryIO.Read.from(String).
BigQueryIO.TypedRead<T>	from(TableReference table) See BigQueryIO.Read.from(TableReference).
BigQueryIO.TypedRead<T>	from(ValueProvider<java.lang.String> tableSpec) See BigQueryIO.Read.from(ValueProvider).
BigQueryIO.TypedRead<T>	fromQuery(java.lang.String query) See BigQueryIO.Read.fromQuery(String).
BigQueryIO.TypedRead<T>	fromQuery(ValueProvider<java.lang.String> query) See BigQueryIO.Read.fromQuery(ValueProvider).
TableReference	getTable() See BigQueryIO.Read.getTable().
ValueProvider<TableReference>	getTableProvider() See BigQueryIO.Read.getTableProvider().
void	populateDisplayData(DisplayData.Builder builder)
BigQueryIO.TypedRead<T>	usingStandardSql() See BigQueryIO.Read.usingStandardSql().
void	validate(PipelineOptions options)
BigQueryIO.TypedRead<T>	withBeamRowConverters(TypeDescriptor<T> typeDescriptor, org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.ToBeamRowFunction<T> toRowFn, org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.FromBeamRowFunction<T> fromRowFn) Sets the functions to convert elements to/from Row objects.
BigQueryIO.TypedRead<T>	withCoder(Coder<T> coder) Sets a Coder for the result of the parse function.
BigQueryIO.TypedRead<T>	withKmsKey(java.lang.String kmsKey) For query sources, use this Cloud KMS key to encrypt any temporary tables created.
BigQueryIO.TypedRead<T>	withMethod(BigQueryIO.TypedRead.Method method) See BigQueryIO.TypedRead.Method.
BigQueryIO.TypedRead<T>	withoutResultFlattening() See BigQueryIO.Read.withoutResultFlattening().
BigQueryIO.TypedRead<T>	withoutValidation() See BigQueryIO.Read.withoutValidation().
BigQueryIO.TypedRead<T>	withQueryLocation(java.lang.String location) BigQuery geographic location where the query job will be executed.
BigQueryIO.TypedRead<T>	withQueryPriority(BigQueryIO.TypedRead.QueryPriority priority) See BigQueryIO.TypedRead.QueryPriority.
BigQueryIO.TypedRead<T>	withQueryTempDataset(java.lang.String queryTempDatasetRef) Temporary dataset reference when using fromQuery(String).
BigQueryIO.TypedRead<T>	withReadOptions(com.google.cloud.bigquery.storage.v1beta1.ReadOptions.TableReadOptions readOptions) Deprecated. Use withSelectedFields(List) and withRowRestriction(String) instead.
BigQueryIO.TypedRead<T>	withRowRestriction(java.lang.String rowRestriction) See withRowRestriction(ValueProvider).
BigQueryIO.TypedRead<T>	withRowRestriction(ValueProvider<java.lang.String> rowRestriction) Read only rows which match the specified filter, which must be a SQL expression compatible with Google standard SQL.
BigQueryIO.TypedRead<T>	withSelectedFields(java.util.List<java.lang.String> selectedFields) See withSelectedFields(ValueProvider).
BigQueryIO.TypedRead<T>	withSelectedFields(ValueProvider<java.util.List<java.lang.String>> selectedFields) Read only the specified fields (columns) from a BigQuery table.
BigQueryIO.TypedRead<T>	withTemplateCompatibility()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TypedRead

public TypedRead()

Method Detail

validate

public void validate(PipelineOptions options)

expand

public PCollection<T> expand(PBegin input)

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

getTableProvider

@Nullable
public ValueProvider<TableReference> getTableProvider()

See BigQueryIO.Read.getTableProvider().

getTable

@Nullable
public TableReference getTable()

See BigQueryIO.Read.getTable().

withCoder

public BigQueryIO.TypedRead<T> withCoder(Coder<T> coder)

Sets a Coder for the result of the parse function. This may be required if a coder can not be inferred automatically.

withKmsKey

public BigQueryIO.TypedRead<T> withKmsKey(java.lang.String kmsKey)

For query sources, use this Cloud KMS key to encrypt any temporary tables created.

withBeamRowConverters

@Experimental(value=SCHEMAS)
public BigQueryIO.TypedRead<T> withBeamRowConverters(TypeDescriptor<T> typeDescriptor,
                                                                                  org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.ToBeamRowFunction<T> toRowFn,
                                                                                  org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.TypedRead.FromBeamRowFunction<T> fromRowFn)

Sets the functions to convert elements to/from Row objects.

Setting these conversion functions is necessary to enable Schema support.

from

public BigQueryIO.TypedRead<T> from(java.lang.String tableSpec)

See BigQueryIO.Read.from(String).

from

public BigQueryIO.TypedRead<T> from(ValueProvider<java.lang.String> tableSpec)

See BigQueryIO.Read.from(ValueProvider).

fromQuery

public BigQueryIO.TypedRead<T> fromQuery(java.lang.String query)

See BigQueryIO.Read.fromQuery(String).

fromQuery

public BigQueryIO.TypedRead<T> fromQuery(ValueProvider<java.lang.String> query)

See BigQueryIO.Read.fromQuery(ValueProvider).

from

public BigQueryIO.TypedRead<T> from(TableReference table)

See BigQueryIO.Read.from(TableReference).

withoutValidation

public BigQueryIO.TypedRead<T> withoutValidation()

See BigQueryIO.Read.withoutValidation().

withoutResultFlattening

public BigQueryIO.TypedRead<T> withoutResultFlattening()

See BigQueryIO.Read.withoutResultFlattening().

usingStandardSql

public BigQueryIO.TypedRead<T> usingStandardSql()

See BigQueryIO.Read.usingStandardSql().

withQueryPriority

public BigQueryIO.TypedRead<T> withQueryPriority(BigQueryIO.TypedRead.QueryPriority priority)

See BigQueryIO.TypedRead.QueryPriority.

withQueryLocation

public BigQueryIO.TypedRead<T> withQueryLocation(java.lang.String location)

BigQuery geographic location where the query job will be executed. If not specified, Beam tries to determine the location by examining the tables referenced by the query. Location must be specified for queries not executed in US or EU, or when you are reading from an authorized view. See BigQuery Jobs: query.

withQueryTempDataset

public BigQueryIO.TypedRead<T> withQueryTempDataset(java.lang.String queryTempDatasetRef)

Temporary dataset reference when using fromQuery(String). When reading from a query, BigQuery will create a temporary dataset and a temporary table to store the results of the query. With this option, you can set an existing dataset to create the temporary table. BigQueryIO will create a temporary table in that dataset, and will remove it once it is not needed. No other tables in the dataset will be modified. If your job does not have permissions to create a new dataset, and you want to use fromQuery(String) (for instance, to read from a view), you should use this option. Remember that the dataset must exist and your job needs permissions to create and remove tables inside that dataset.

withMethod

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withMethod(BigQueryIO.TypedRead.Method method)

See BigQueryIO.TypedRead.Method.

withReadOptions

@Deprecated
 @Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withReadOptions(com.google.cloud.bigquery.storage.v1beta1.ReadOptions.TableReadOptions readOptions)

Deprecated. Use withSelectedFields(List) and withRowRestriction(String) instead.

withSelectedFields

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withSelectedFields(java.util.List<java.lang.String> selectedFields)

See withSelectedFields(ValueProvider).

withSelectedFields

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withSelectedFields(ValueProvider<java.util.List<java.lang.String>> selectedFields)

Read only the specified fields (columns) from a BigQuery table. Fields may not be returned in the order specified. If no value is specified, then all fields are returned.

Requires BigQueryIO.TypedRead.Method.DIRECT_READ. Not compatible with fromQuery(String).

withRowRestriction

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withRowRestriction(java.lang.String rowRestriction)

See withRowRestriction(ValueProvider).

withRowRestriction

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withRowRestriction(ValueProvider<java.lang.String> rowRestriction)

Read only rows which match the specified filter, which must be a SQL expression compatible with Google standard SQL. If no value is specified, then all rows are returned.

Requires BigQueryIO.TypedRead.Method.DIRECT_READ. Not compatible with fromQuery(String).

withTemplateCompatibility

@Experimental(value=SOURCE_SINK)
public BigQueryIO.TypedRead<T> withTemplateCompatibility()