org.apache.beam.sdk.io.snowflake

Class SnowflakeIO.Write<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<T>,PDone>

org.apache.beam.sdk.io.snowflake.SnowflakeIO.Write<T>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

SnowflakeIO

public abstract static class SnowflakeIO.Write<T>
extends PTransform<PCollection<T>,PDone>

Implementation of SnowflakeIO.write().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Write()

Method Summary

Modifier and Type	Method and Description
PDone	expand(PCollection<T> input) Override this method to specify how this PTransform should be expanded on the given InputT.
SnowflakeIO.Write<T>	withDataSourceConfiguration(SnowflakeIO.DataSourceConfiguration config) Setting information about Snowflake server.
SnowflakeIO.Write<T>	withDataSourceProviderFn(SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn) Setting function that will provide SnowflakeIO.DataSourceConfiguration in runtime.
SnowflakeIO.Write<T>	withFileNameTemplate(java.lang.String fileNameTemplate) A template name for files saved to GCP.
SnowflakeIO.Write<T>	withQueryTransformation(java.lang.String query) A query to be executed in Snowflake.
SnowflakeIO.Write<T>	withSnowflakeService(SnowflakeService snowflakeService) A snowflake service which is supposed to be used.
SnowflakeIO.Write<T>	withStagingBucketName(java.lang.String stagingBucketName) Name of the cloud bucket (GCS by now) to use as tmp location of CSVs during COPY statement.
SnowflakeIO.Write<T>	withStorageIntegrationName(java.lang.String integrationName) Name of the Storage Integration in Snowflake to be used.
SnowflakeIO.Write<T>	withTable(java.lang.String table) A table name to be written in Snowflake.
SnowflakeIO.Write<T>	withUserDataMapper(SnowflakeIO.UserDataMapper userDataMapper) User-defined function mapping user data into CSV lines.
SnowflakeIO.Write<T>	withWriteDisposition(WriteDisposition writeDisposition) A disposition to be used during writing to table phase.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Write

public Write()

Method Detail

withDataSourceConfiguration

public SnowflakeIO.Write<T> withDataSourceConfiguration(SnowflakeIO.DataSourceConfiguration config)

Setting information about Snowflake server.

Parameters:

config - - An instance of SnowflakeIO.DataSourceConfiguration.

withDataSourceProviderFn

public SnowflakeIO.Write<T> withDataSourceProviderFn(SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn)

Setting function that will provide SnowflakeIO.DataSourceConfiguration in runtime.

Parameters:

dataSourceProviderFn - a SerializableFunction.

withTable

public SnowflakeIO.Write<T> withTable(java.lang.String table)

A table name to be written in Snowflake.

Parameters:

table - - String with the name of the table.

withStagingBucketName

public SnowflakeIO.Write<T> withStagingBucketName(java.lang.String stagingBucketName)

Name of the cloud bucket (GCS by now) to use as tmp location of CSVs during COPY statement.

Parameters:

stagingBucketName - - String with the name of the bucket.

withStorageIntegrationName

public SnowflakeIO.Write<T> withStorageIntegrationName(java.lang.String integrationName)

Name of the Storage Integration in Snowflake to be used. See https://docs.snowflake.com/en/sql-reference/sql/create-storage-integration.html for reference.

Parameters:

integrationName - - String with the name of the Storage Integration.

withQueryTransformation

public SnowflakeIO.Write<T> withQueryTransformation(java.lang.String query)

A query to be executed in Snowflake.

Parameters:

query - - String with query.

withFileNameTemplate

public SnowflakeIO.Write<T> withFileNameTemplate(java.lang.String fileNameTemplate)

A template name for files saved to GCP.

Parameters:

fileNameTemplate - - String with template name for files.

withUserDataMapper

public SnowflakeIO.Write<T> withUserDataMapper(SnowflakeIO.UserDataMapper userDataMapper)

User-defined function mapping user data into CSV lines.

Parameters:

userDataMapper - - an instance of SnowflakeIO.UserDataMapper.

withWriteDisposition

public SnowflakeIO.Write<T> withWriteDisposition(WriteDisposition writeDisposition)

A disposition to be used during writing to table phase.

Parameters:

writeDisposition - - an instance of WriteDisposition.

withSnowflakeService

public SnowflakeIO.Write<T> withSnowflakeService(SnowflakeService snowflakeService)

A snowflake service which is supposed to be used. Note: Currently we have SnowflakeServiceImpl with corresponding FakeSnowflakeServiceImpl used for testing.

Parameters:

snowflakeService - - an instance of SnowflakeService.

expand

public PDone expand(PCollection<T> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<T>,PDone>