org.apache.beam.runners.spark.structuredstreaming.translation.batch.functions

Interface SideInputValues<T>

All Superinterfaces:

com.esotericsoftware.kryo.KryoSerializable, java.io.Serializable

All Known Implementing Classes:

SideInputValues.BaseSideInputValues, SideInputValues.ByWindow, SideInputValues.Global

@Internal
public interface SideInputValues<T>
extends java.io.Serializable, com.esotericsoftware.kryo.KryoSerializable

SideInputValues serves as a Kryo serializable container that contains a materialized view of side inputs. Once the materialized view is build, the container is broadcasted for use in the SparkSideInputReader. This happens during translation time of the pipeline.

If Kryo serialization is disabled in Spark, Java serialization will be used instead and some optimizations will not be available.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	SideInputValues.BaseSideInputValues<BinaryT,ValuesT,T>
static class	SideInputValues.ByWindow<T> General SideInputValues for BoundedWindows in two possible states.
static class	SideInputValues.Global<T> Specialized SideInputValues for use with the GlobalWindow in two possible states.
static interface	SideInputValues.Loader<T> Factory function for load SideInputValues from a Dataset.

Method Summary

Modifier and Type	Method and Description
java.util.List<T>	get(BoundedWindow window)
static <T> SideInputValues.Loader<T>	loader(PCollection<T> pCol) Factory to load SideInputValues from a Dataset based on the window strategy.

Methods inherited from interface com.esotericsoftware.kryo.KryoSerializable

read, write

Method Detail

get

@Nullable
java.util.List<T> get(BoundedWindow window)

loader

static <T> SideInputValues.Loader<T> loader(PCollection<T> pCol)

Factory to load SideInputValues from a Dataset based on the window strategy.