org.apache.beam.sdk.io.gcp.bigquery.providers

Class PortableBigQueryDestinations

java.lang.Object

org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations<Row,java.lang.String>

org.apache.beam.sdk.io.gcp.bigquery.providers.PortableBigQueryDestinations

All Implemented Interfaces:

java.io.Serializable

@Internal
public class PortableBigQueryDestinations
extends DynamicDestinations<Row,java.lang.String>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DESTINATION
static java.lang.String	RECORD

Constructor Summary

Constructors

Constructor and Description
PortableBigQueryDestinations(Schema rowSchema, BigQueryWriteConfiguration configuration)

Method Summary

Modifier and Type	Method and Description
SerializableFunction<AvroWriteRequest<Row>,GenericRecord>	getAvroFilterFormatFunction(boolean fetchNestedRecord)
java.lang.String	getDestination(@Nullable ValueInSingleWindow<Row> element) Returns an object that represents at a high level which table is being written to.
SerializableFunction<Row,TableRow>	getFilterFormatFunction(boolean fetchNestedRecord)
@Nullable TableSchema	getSchema(java.lang.String destination) Returns the table schema for the destination.
TableDestination	getTable(java.lang.String destination) Returns a TableDestination object for the destination.
@Nullable TableConstraints	getTableConstraints(java.lang.String destination) Returns TableConstraints (including primary and foreign key) to be used when creating the table.

Methods inherited from class org.apache.beam.sdk.io.gcp.bigquery.DynamicDestinations

getDestinationCoder, getSideInputs, sideInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DESTINATION

public static final java.lang.String DESTINATION

See Also:

Constant Field Values

RECORD

public static final java.lang.String RECORD

See Also:

Constant Field Values

Constructor Detail

PortableBigQueryDestinations

public PortableBigQueryDestinations(Schema rowSchema,
                                    BigQueryWriteConfiguration configuration)

Method Detail

getDestination

public java.lang.String getDestination(@Nullable ValueInSingleWindow<Row> element)

Description copied from class: DynamicDestinations

Returns an object that represents at a high level which table is being written to. May not return null.

The method must return a unique object for different destination tables involved over all BigQueryIO write transforms in the same pipeline. See https://github.com/apache/beam/issues/32335 for details.

Specified by:

getDestination in class DynamicDestinations<Row,java.lang.String>

getTable

public TableDestination getTable(java.lang.String destination)

Description copied from class: DynamicDestinations

Returns a TableDestination object for the destination. May not return null. Return value needs to be unique to each destination: may not return the same TableDestination for different destinations.

Specified by:

getTable in class DynamicDestinations<Row,java.lang.String>

getSchema

public @Nullable TableSchema getSchema(java.lang.String destination)

Description copied from class: DynamicDestinations

Returns the table schema for the destination.

Specified by:

getSchema in class DynamicDestinations<Row,java.lang.String>

getTableConstraints

public @Nullable TableConstraints getTableConstraints(java.lang.String destination)

Description copied from class: DynamicDestinations

Returns TableConstraints (including primary and foreign key) to be used when creating the table. Note: this is not currently supported when using FILE_LOADS!.

Overrides:

getTableConstraints in class DynamicDestinations<Row,java.lang.String>

getFilterFormatFunction

public SerializableFunction<Row,TableRow> getFilterFormatFunction(boolean fetchNestedRecord)

getAvroFilterFormatFunction

public SerializableFunction<AvroWriteRequest<Row>,GenericRecord> getAvroFilterFormatFunction(boolean fetchNestedRecord)