Package org.apache.beam.runners.spark.translation

Class SparkStreamingTranslationContext

java.lang.Object

org.apache.beam.runners.spark.translation.SparkTranslationContext

org.apache.beam.runners.spark.translation.SparkStreamingTranslationContext

public class SparkStreamingTranslationContext
extends SparkTranslationContext

Translation context used to lazily store Spark datasets during streaming portable pipeline translation and compute them after translation.

Constructor Summary

Constructors

Constructor	Description
SparkStreamingTranslationContext(org.apache.spark.api.java.JavaSparkContext jsc, SparkPipelineOptions options, JobInfo jobInfo)

Method Summary

Modifier and Type	Method	Description
Instant	getFirstTimestamp()
org.apache.spark.streaming.api.java.JavaStreamingContext	getStreamingContext()

Methods inherited from class org.apache.beam.runners.spark.translation.SparkTranslationContext

computeOutputs, getSerializableOptions, getSparkContext, nextSinkId, popDataset, pushDataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SparkStreamingTranslationContext

public SparkStreamingTranslationContext(org.apache.spark.api.java.JavaSparkContext jsc,
 SparkPipelineOptions options,
 JobInfo jobInfo)

Method Details

getStreamingContext

public org.apache.spark.streaming.api.java.JavaStreamingContext getStreamingContext()

getFirstTimestamp

public Instant getFirstTimestamp()