Package org.apache.beam.sdk.io.gcp.bigquery

Class StorageApiDynamicDestinationsTableRow<T,DestinationT extends @NonNull Object>

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations<T,DestinationT>

org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinationsTableRow<T,DestinationT>

All Implemented Interfaces:

Serializable

public class StorageApiDynamicDestinationsTableRow<T,DestinationT extends @NonNull Object>
extends DynamicDestinations<T,DestinationT>

See Also:

• Serialized Form

Method Summary

Modifier and Type	Method	Description
DestinationT	getDestination(@Nullable ValueInSingleWindow<T> element)	Returns an object that represents at a high level which table is being written to.
@Nullable Coder<DestinationT>	getDestinationCoder()	Returns the coder for DynamicDestinations.
org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations.MessageConverter<T>	getMessageConverter(DestinationT destination, BigQueryServices.DatasetService datasetService)
@Nullable TableSchema	getSchema(DestinationT destination)	Returns the table schema for the destination.
List<PCollectionView<?>>	getSideInputs()	Specifies that this object needs access to one or more side inputs.
TableDestination	getTable(DestinationT destination)	Returns a TableDestination object for the destination.
@Nullable TableConstraints	getTableConstraints(DestinationT destination)	Returns TableConstraints (including primary and foreign key) to be used when creating the table.
String	toString()

Methods inherited from class org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations

sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

getMessageConverter

public org.apache.beam.sdk.io.gcp.bigquery.StorageApiDynamicDestinations.MessageConverter<T> getMessageConverter(DestinationT destination,
 BigQueryServices.DatasetService datasetService)
                                                                                                          throws Exception

Throws:

Exception

getDestination

public DestinationT getDestination(@Nullable ValueInSingleWindow<T> element)

Description copied from class: DynamicDestinations

Returns an object that represents at a high level which table is being written to. May not return null.

The method must return a unique object for different destination tables involved over all BigQueryIO write transforms in the same pipeline. See https://github.com/apache/beam/issues/32335 for details.

Specified by:

getDestination in class DynamicDestinations<T,DestinationT>

getSchema

public @Nullable TableSchema getSchema(DestinationT destination)

Description copied from class: DynamicDestinations

Returns the table schema for the destination.

Specified by:

getSchema in class DynamicDestinations<T,DestinationT>

getTableConstraints

public @Nullable TableConstraints getTableConstraints(DestinationT destination)

Description copied from class: DynamicDestinations

Returns TableConstraints (including primary and foreign key) to be used when creating the table. Note: this is not currently supported when using FILE_LOADS!.

Overrides:

getTableConstraints in class DynamicDestinations<T,DestinationT>

getTable

public TableDestination getTable(DestinationT destination)

Description copied from class: DynamicDestinations

Returns a TableDestination object for the destination. May not return null. Return value needs to be unique to each destination: may not return the same TableDestination for different destinations.

Specified by:

getTable in class DynamicDestinations<T,DestinationT>

getDestinationCoder

public @Nullable Coder<DestinationT> getDestinationCoder()

Description copied from class: DynamicDestinations

Returns the coder for DynamicDestinations. If this is not overridden, then BigQueryIO will look in the coder registry for a suitable coder. This must be a deterministic coder, as DynamicDestinations will be used as a key type in a GroupByKey.

Overrides:

getDestinationCoder in class DynamicDestinations<T,DestinationT>

getSideInputs

public List<PCollectionView<?>> getSideInputs()

Description copied from class: DynamicDestinations

Specifies that this object needs access to one or more side inputs. This side inputs must be globally windowed, as they will be accessed from the global window.

Overrides:

getSideInputs in class DynamicDestinations<T,DestinationT>

toString

public String toString()

Overrides:

toString in class Object