org.apache.beam.sdk.extensions.sql.meta.provider.pubsub

Class PubsubMessageToRow

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<PubsubMessage>,PCollectionTuple>

org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

@Internal
 @Experimental
public abstract class PubsubMessageToRow
extends PTransform<PCollection<PubsubMessage>,PCollectionTuple>
implements java.io.Serializable

Read side converter for PubsubMessage with JSON payload.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
PubsubMessageToRow()

Method Summary

Modifier and Type	Method and Description
static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder	builder()
PCollectionTuple	expand(PCollection<PubsubMessage> input) Override this method to specify how this PTransform should be expanded on the given InputT.
abstract Schema	messageSchema() Schema of the Pubsub message.
abstract boolean	useDlq()
abstract boolean	useFlatSchema()

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PubsubMessageToRow

public PubsubMessageToRow()

Method Detail

messageSchema

public abstract Schema messageSchema()

Schema of the Pubsub message.

Required to have at least 'event_timestamp' field of type Schema.FieldType#DATETIME.

If useFlatSchema() is set every other field is assumed to be part of the payload. Otherwise, the schema must contain exactly:

• 'attributes' of type MAP<VARCHAR,VARCHAR>
• 'payload' of type ROW<...>

Only UTF-8 JSON objects are supported.

useDlq

public abstract boolean useDlq()

useFlatSchema

public abstract boolean useFlatSchema()

builder

public static org.apache.beam.sdk.extensions.sql.meta.provider.pubsub.PubsubMessageToRow.Builder builder()

expand

public PCollectionTuple expand(PCollection<PubsubMessage> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<PubsubMessage>,PCollectionTuple>