org.apache.beam.sdk.io.gcp.bigquery

Class StreamingWriteTables<ElementT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<TableDestination,ElementT>>,WriteResult>

org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteTables<ElementT>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

public class StreamingWriteTables<ElementT>
extends PTransform<PCollection<KV<TableDestination,ElementT>>,WriteResult>

This transform takes in key-value pairs of TableRow entries and the TableDestination it should be written to. The BigQuery streaming-write service is used to stream these writes to the appropriate table.

This transform assumes that all destination tables already exist by the time it sees a write for that table.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor and Description
StreamingWriteTables()

Method Summary

Modifier and Type	Method and Description
WriteResult	expand(PCollection<KV<TableDestination,ElementT>> input) Override this method to specify how this PTransform should be expanded on the given InputT.
<TupleTagT> TupleTag<TupleTagT>	getFailedRowsTupleTag()

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

StreamingWriteTables

public StreamingWriteTables()

Method Detail

getFailedRowsTupleTag

public <TupleTagT> TupleTag<TupleTagT> getFailedRowsTupleTag()

expand

public WriteResult expand(PCollection<KV<TableDestination,ElementT>> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<KV<TableDestination,ElementT>>,WriteResult>