org.apache.beam.sdk.io.kinesis

Class KinesisIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PBegin,PCollection<KinesisRecord>>

org.apache.beam.sdk.io.kinesis.KinesisIO.Read

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

KinesisIO

public abstract static class KinesisIO.Read
extends PTransform<PBegin,PCollection<KinesisRecord>>

Implementation of KinesisIO.read().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Read()

Method Summary

Modifier and Type	Method and Description
PCollection<KinesisRecord>	expand(PBegin input) Override this method to specify how this PTransform should be expanded on the given InputT.
KinesisIO.Read	from(java.lang.String streamName, com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream initialPosition) Specify reading from streamName at some initial position.
KinesisIO.Read	from(java.lang.String streamName, Instant initialTimestamp) Specify reading from streamName beginning at given Instant.
KinesisIO.Read	withClientProvider(org.apache.beam.sdk.io.kinesis.KinesisClientProvider kinesisClientProvider) Allows to specify custom KinesisClientProvider.
KinesisIO.Read	withClientProvider(java.lang.String awsAccessKey, java.lang.String awsSecretKey, com.amazonaws.regions.Regions region) Specify credential details and region to be used to read from Kinesis.
KinesisIO.Read	withMaxNumRecords(int maxNumRecords) Specifies to read at most a given number of records.
KinesisIO.Read	withMaxReadTime(Duration maxReadTime) Specifies to read at most a given number of records.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Read

public Read()

Method Detail

from

public KinesisIO.Read from(java.lang.String streamName,
                           com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream initialPosition)

Specify reading from streamName at some initial position.

from

public KinesisIO.Read from(java.lang.String streamName,
                           Instant initialTimestamp)

Specify reading from streamName beginning at given Instant. This Instant must be in the past, i.e. before Instant.now().

withClientProvider

public KinesisIO.Read withClientProvider(org.apache.beam.sdk.io.kinesis.KinesisClientProvider kinesisClientProvider)

Allows to specify custom KinesisClientProvider. KinesisClientProvider provides AmazonKinesis instances which are later used for communication with Kinesis. You should use this method if withClientProvider(String, String, Regions) does not suit your needs.

withClientProvider

public KinesisIO.Read withClientProvider(java.lang.String awsAccessKey,
                                         java.lang.String awsSecretKey,
                                         com.amazonaws.regions.Regions region)

Specify credential details and region to be used to read from Kinesis. If you need more sophisticated credential protocol, then you should look at withClientProvider(KinesisClientProvider).

withMaxNumRecords

public KinesisIO.Read withMaxNumRecords(int maxNumRecords)

Specifies to read at most a given number of records.

withMaxReadTime

public KinesisIO.Read withMaxReadTime(Duration maxReadTime)

Specifies to read at most a given number of records.

expand

public PCollection<KinesisRecord> expand(PBegin input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PBegin,PCollection<KinesisRecord>>