org.apache.beam.sdk.io.snowflake

Class SnowflakeIO.Read.CleanTmpFilesFromGcsFn

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<java.lang.Object,java.lang.Object>

org.apache.beam.sdk.io.snowflake.SnowflakeIO.Read.CleanTmpFilesFromGcsFn

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

SnowflakeIO.Read<T>

public static class SnowflakeIO.Read.CleanTmpFilesFromGcsFn
extends DoFn<java.lang.Object,java.lang.Object>

Removes temporary staged files after reading.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

DoFn.AlwaysFetched, DoFn.BoundedPerElement, DoFn.BundleFinalizer, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.GetInitialRestriction, DoFn.GetInitialWatermarkEstimatorState, DoFn.GetRestrictionCoder, DoFn.GetSize, DoFn.GetWatermarkEstimatorStateCoder, DoFn.Key, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.NewWatermarkEstimator, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnTimerFamily, DoFn.OnWindowExpiration, DoFn.OnWindowExpirationContext, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.RequiresTimeSortedInput, DoFn.Restriction, DoFn.Setup, DoFn.SideInput, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerFamily, DoFn.TimerId, DoFn.Timestamp, DoFn.TruncateRestriction, DoFn.UnboundedPerElement, DoFn.WatermarkEstimatorState, DoFn.WindowedContext

Constructor Summary

Constructors

Constructor and Description
CleanTmpFilesFromGcsFn(ValueProvider<java.lang.String> stagingBucketDir, java.lang.String tmpDirName) Created object that will remove temp files from stage.

Method Summary

Modifier and Type	Method and Description
void	processElement(DoFn.ProcessContext c)

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CleanTmpFilesFromGcsFn

public CleanTmpFilesFromGcsFn(ValueProvider<java.lang.String> stagingBucketDir,
                              java.lang.String tmpDirName)

Created object that will remove temp files from stage.

Parameters:

stagingBucketDir - bucket and directory where temporary files are saved

tmpDirName - temporary directory created on bucket where files were saved

Method Detail

processElement

@DoFn.ProcessElement
public void processElement(DoFn.ProcessContext c)
                                         throws java.io.IOException

Throws:

java.io.IOException