Package org.apache.beam.sdk.io.gcp.bigquery.providers

Class PortableBigQueryDestinations

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations<Row,String>

org.apache.beam.sdk.io.gcp.bigquery.providers.PortableBigQueryDestinations

All Implemented Interfaces:

Serializable

@Internal
public class PortableBigQueryDestinations
extends DynamicDestinations<Row,String>

See Also:

• Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
static final String	DESTINATION
static final String	RECORD

Constructor Summary

Constructors

Constructor	Description
PortableBigQueryDestinations(Schema rowSchema, BigQueryWriteConfiguration configuration)

Method Summary

Modifier and Type	Method	Description
SerializableFunction<AvroWriteRequest<Row>,GenericRecord>	getAvroFilterFormatFunction(boolean fetchNestedRecord)
String	getDestination(@Nullable ValueInSingleWindow<Row> element)	Returns an object that represents at a high level which table is being written to.
SerializableFunction<Row,TableRow>	getFilterFormatFunction(boolean fetchNestedRecord)
@Nullable TableSchema	getSchema(String destination)	Returns the table schema for the destination.
TableDestination	getTable(String destination)	Returns a TableDestination object for the destination.
@Nullable TableConstraints	getTableConstraints(String destination)	Returns TableConstraints (including primary and foreign key) to be used when creating the table.

Methods inherited from class org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations

getDestinationCoder, getSideInputs, sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DESTINATION

public static final String DESTINATION

See Also:

• Constant Field Values

RECORD

public static final String RECORD

See Also:

• Constant Field Values

Constructor Details

PortableBigQueryDestinations

public PortableBigQueryDestinations(Schema rowSchema,
 BigQueryWriteConfiguration configuration)

Method Details

getDestination

public String getDestination(@Nullable ValueInSingleWindow<Row> element)

Description copied from class: DynamicDestinations

Returns an object that represents at a high level which table is being written to. May not return null.

The method must return a unique object for different destination tables involved over all BigQueryIO write transforms in the same pipeline. See https://github.com/apache/beam/issues/32335 for details.

Specified by:

getDestination in class DynamicDestinations<Row,String>

getTable

public TableDestination getTable(String destination)

Description copied from class: DynamicDestinations

Returns a TableDestination object for the destination. May not return null. Return value needs to be unique to each destination: may not return the same TableDestination for different destinations.

Specified by:

getTable in class DynamicDestinations<Row,String>

getSchema

public @Nullable TableSchema getSchema(String destination)

Description copied from class: DynamicDestinations

Returns the table schema for the destination.

Specified by:

getSchema in class DynamicDestinations<Row,String>

getTableConstraints

public @Nullable TableConstraints getTableConstraints(String destination)

Description copied from class: DynamicDestinations

Returns TableConstraints (including primary and foreign key) to be used when creating the table. Note: this is not currently supported when using FILE_LOADS!.

Overrides:

getTableConstraints in class DynamicDestinations<Row,String>

getFilterFormatFunction

public SerializableFunction<Row,TableRow> getFilterFormatFunction(boolean fetchNestedRecord)

getAvroFilterFormatFunction

public SerializableFunction<AvroWriteRequest<Row>,GenericRecord> getAvroFilterFormatFunction(boolean fetchNestedRecord)