Package org.apache.beam.sdk.io.snowflake

Class SnowflakeIO.Read.CleanTmpFilesFromGcsFn

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<Object,Object>

org.apache.beam.sdk.io.snowflake.SnowflakeIO.Read.CleanTmpFilesFromGcsFn

All Implemented Interfaces:

Serializable, HasDisplayData

Enclosing class:

SnowflakeIO.Read<T>

public static class SnowflakeIO.Read.CleanTmpFilesFromGcsFn
extends DoFn<Object,Object>

Removes temporary staged files after reading.

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

DoFn.AlwaysFetched, DoFn.BoundedPerElement, DoFn.BundleFinalizer, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.GetInitialRestriction, DoFn.GetInitialWatermarkEstimatorState, DoFn.GetRestrictionCoder, DoFn.GetSize, DoFn.GetWatermarkEstimatorStateCoder, DoFn.Key, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.NewWatermarkEstimator, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnTimerFamily, DoFn.OnWindowExpiration, DoFn.OnWindowExpirationContext, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.RequiresTimeSortedInput, DoFn.Restriction, DoFn.Setup, DoFn.SideInput, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerFamily, DoFn.TimerId, DoFn.Timestamp, DoFn.TruncateRestriction, DoFn.UnboundedPerElement, DoFn.WatermarkEstimatorState, DoFn.WindowedContext

Constructor Summary

Constructors

Constructor	Description
CleanTmpFilesFromGcsFn(ValueProvider<String> stagingBucketDir, String tmpDirName)	Created object that will remove temp files from stage.

Method Summary

Modifier and Type	Method	Description
void	processElement(DoFn<Object,Object>.ProcessContext c)

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CleanTmpFilesFromGcsFn

public CleanTmpFilesFromGcsFn(ValueProvider<String> stagingBucketDir,
 String tmpDirName)

Created object that will remove temp files from stage.

Parameters:

stagingBucketDir - bucket and directory where temporary files are saved

tmpDirName - temporary directory created on bucket where files were saved

Method Details

processElement

@ProcessElement
public void processElement(DoFn<Object,Object>.ProcessContext c)
                    throws IOException

Throws:

IOException