Package org.apache.beam.sdk.io.gcp.bigquery

Class PatchTableSchemaDoFn<DestinationT extends @NonNull Object,ElementT>

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema>,KV<DestinationT,ElementT>>

org.apache.beam.sdk.io.gcp.bigquery.PatchTableSchemaDoFn<DestinationT,ElementT>

All Implemented Interfaces:

Serializable, HasDisplayData

public class PatchTableSchemaDoFn<DestinationT extends @NonNull Object,ElementT>
extends DoFn<KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema>,KV<DestinationT,ElementT>>

This DoFn is responsible for updating a BigQuery's table schema. The input is a TableSchema containing only the schema delta (new fields, relaxed fields). It outputs elements for all updated tables, which act as notifcations to the buffering stage that the elements can be retried.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

DoFn.AlwaysFetched, DoFn.BoundedPerElement, DoFn.BundleFinalizer, DoFn.CurrentRecordId, DoFn.CurrentRecordOffset, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.FireTimestamp, DoFn.GetInitialRestriction, DoFn.GetInitialWatermarkEstimatorState, DoFn.GetRestrictionCoder, DoFn.GetSize, DoFn.GetWatermarkEstimatorStateCoder, DoFn.Key, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.NewWatermarkEstimator, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnTimerFamily, DoFn.OnWindowExpiration, DoFn.OnWindowExpirationContext, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.RequiresTimeSortedInput, DoFn.Restriction, DoFn.Setup, DoFn.SideInput, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerFamily, DoFn.TimerId, DoFn.Timestamp, DoFn.TruncateRestriction, DoFn.UnboundedPerElement, DoFn.WatermarkEstimatorState, DoFn.WindowedContext

Method Summary

Modifier and Type	Method	Description
void	onTeardown()
void	processElement(KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema> element, DoFn.OutputReceiver<KV<DestinationT,@Nullable ElementT>> o, DoFn<KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema>,KV<DestinationT,ElementT>>.ProcessContext context, PipelineOptions pipelineOptions)

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

onTeardown

@Teardown
public void onTeardown()

processElement

@ProcessElement
public void processElement(@Element
 KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema> element,
 DoFn.OutputReceiver<KV<DestinationT,@Nullable ElementT>> o,
 DoFn<KV<DestinationT,com.google.cloud.bigquery.storage.v1.TableSchema>,KV<DestinationT,ElementT>>.ProcessContext context,
 PipelineOptions pipelineOptions)
                    throws Exception

Throws:

Exception