org.apache.beam.sdk.extensions.sql.meta.provider.pubsub

Class PubsubMessageToRow

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<PubsubMessage,Row>

org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

@Internal
 @Experimental
public abstract class PubsubMessageToRow
extends DoFn<PubsubMessage,Row>

A DoFn to convert PubsubMessage with JSON payload to Row.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

DoFn.BoundedPerElement, DoFn.BundleFinalizer, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.GetInitialRestriction, DoFn.GetPartition, DoFn.GetRestrictionCoder, DoFn.GetSize, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnWindowExpiration, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.Setup, DoFn.SideInput, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerId, DoFn.Timestamp, DoFn.UnboundedPerElement, DoFn.WindowedContext

Constructor Summary

Constructors

Constructor and Description
PubsubMessageToRow()

Method Summary

Modifier and Type	Method and Description
static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder	builder()
abstract Schema	messageSchema() Schema of the Pubsub message.
void	processElement(DoFn.ProcessContext context)
abstract boolean	useDlq()

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PubsubMessageToRow

public PubsubMessageToRow()

Method Detail

messageSchema

public abstract Schema messageSchema()

Schema of the Pubsub message.

Required to have exactly 3 top level fields at the moment:

• 'event_timestamp' of type Schema.FieldType#DATETIME
• 'attributes' of type MAP<VARCHAR,VARCHAR>
• 'payload' of type ROW<...>

Only UTF-8 JSON objects are supported.

useDlq

public abstract boolean useDlq()

builder

public static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder builder()

processElement

@DoFn.ProcessElement
public void processElement(DoFn.ProcessContext context)