org.apache.beam.sdk.extensions.sql.meta.provider.kafka

Class BeamKafkaTable

java.lang.Object

org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

org.apache.beam.sdk.extensions.sql.meta.provider.kafka.BeamKafkaTable

All Implemented Interfaces:

java.io.Serializable, BeamSqlTable

Direct Known Subclasses:

BeamKafkaCSVTable

public abstract class BeamKafkaTable
extends BaseBeamTable

BeamKafkaTable represent a Kafka topic, as source or target. Need to extend to convert between BeamSqlRow and KV<byte[], byte[]>.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

schema

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BeamKafkaTable(Schema beamSchema)
	BeamKafkaTable(Schema beamSchema, java.util.List<org.apache.kafka.common.TopicPartition> topicPartitions, java.lang.String bootstrapServers)
	BeamKafkaTable(Schema beamSchema, java.lang.String bootstrapServers, java.util.List<java.lang.String> topics)

Method Summary

Modifier and Type	Method and Description
PCollection<Row>	buildIOReader(PBegin begin) create a PCollection<Row> from source.
POutput	buildIOWriter(PCollection<Row> input) create a IO.write() instance to write to target.
java.lang.String	getBootstrapServers()
abstract PTransform<PCollection<KV<byte[],byte[]>>,PCollection<Row>>	getPTransformForInput()
abstract PTransform<PCollection<Row>,PCollection<KV<byte[],byte[]>>>	getPTransformForOutput()
java.util.List<java.lang.String>	getTopics()
BeamKafkaTable	updateConsumerProperties(java.util.Map<java.lang.String,java.lang.Object> configUpdates)

Methods inherited from class org.apache.beam.sdk.extensions.sql.impl.schema.BaseBeamTable

getSchema

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BeamKafkaTable

protected BeamKafkaTable(Schema beamSchema)

BeamKafkaTable

public BeamKafkaTable(Schema beamSchema,
                      java.lang.String bootstrapServers,
                      java.util.List<java.lang.String> topics)

BeamKafkaTable

public BeamKafkaTable(Schema beamSchema,
                      java.util.List<org.apache.kafka.common.TopicPartition> topicPartitions,
                      java.lang.String bootstrapServers)

Method Detail

updateConsumerProperties

public BeamKafkaTable updateConsumerProperties(java.util.Map<java.lang.String,java.lang.Object> configUpdates)

getPTransformForInput

public abstract PTransform<PCollection<KV<byte[],byte[]>>,PCollection<Row>> getPTransformForInput()

getPTransformForOutput

public abstract PTransform<PCollection<Row>,PCollection<KV<byte[],byte[]>>> getPTransformForOutput()

buildIOReader

public PCollection<Row> buildIOReader(PBegin begin)

Description copied from interface: BeamSqlTable

create a PCollection<Row> from source.

buildIOWriter

public POutput buildIOWriter(PCollection<Row> input)

Description copied from interface: BeamSqlTable

create a IO.write() instance to write to target.

getBootstrapServers

public java.lang.String getBootstrapServers()

getTopics

public java.util.List<java.lang.String> getTopics()