org.apache.beam.runners.spark.structuredstreaming.translation.batch

Class DatasetSourceBatch

java.lang.Object

org.apache.beam.runners.spark.structuredstreaming.translation.batch.DatasetSourceBatch

All Implemented Interfaces:

org.apache.spark.sql.sources.v2.DataSourceV2, org.apache.spark.sql.sources.v2.ReadSupport

public class DatasetSourceBatch
extends java.lang.Object
implements org.apache.spark.sql.sources.v2.DataSourceV2, org.apache.spark.sql.sources.v2.ReadSupport

This is a spark structured streaming DataSourceV2 implementation that wraps an BoundedSource.As Continuous streaming is tagged experimental in spark (no aggregation support + no exactly once guaranty), this class does no implement ContinuousReadSupport.

Constructor Summary

Constructors

Constructor and Description
DatasetSourceBatch()

Method Summary

Modifier and Type	Method and Description
org.apache.spark.sql.sources.v2.reader.DataSourceReader	createReader(org.apache.spark.sql.sources.v2.DataSourceOptions options)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.sources.v2.ReadSupport

createReader

Constructor Detail

DatasetSourceBatch

public DatasetSourceBatch()

Method Detail

createReader

public org.apache.spark.sql.sources.v2.reader.DataSourceReader createReader(org.apache.spark.sql.sources.v2.DataSourceOptions options)

Specified by:

createReader in interface org.apache.spark.sql.sources.v2.ReadSupport