org.apache.beam.runners.spark.structuredstreaming.translation

Class SparkSessionFactory

java.lang.Object

org.apache.beam.runners.spark.structuredstreaming.translation.SparkSessionFactory

public class SparkSessionFactory
extends java.lang.Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SparkSessionFactory.SparkKryoRegistrator KryoRegistrator for Spark to serialize broadcast variables used for side-inputs.

Constructor Summary

Constructors

Constructor and Description
SparkSessionFactory()

Method Summary

Modifier and Type	Method and Description
static org.apache.spark.sql.SparkSession	getOrCreateSession(SparkStructuredStreamingPipelineOptions options) Gets active SparkSession or creates one using SparkStructuredStreamingPipelineOptions.
static org.apache.spark.sql.SparkSession.Builder	sessionBuilder(java.lang.String master) Creates Spark session builder with some optimizations for local mode, e.g.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkSessionFactory

public SparkSessionFactory()

Method Detail

getOrCreateSession

public static org.apache.spark.sql.SparkSession getOrCreateSession(SparkStructuredStreamingPipelineOptions options)

Gets active SparkSession or creates one using SparkStructuredStreamingPipelineOptions.

sessionBuilder

public static org.apache.spark.sql.SparkSession.Builder sessionBuilder(java.lang.String master)

Creates Spark session builder with some optimizations for local mode, e.g. in tests.