org.apache.beam.sdk.io

Class FileBasedSink.FilenamePolicy

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.FilenamePolicy

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

DefaultFilenamePolicy

Enclosing class:

FileBasedSink<T>

public abstract static class FileBasedSink.FilenamePolicy
extends java.lang.Object
implements java.io.Serializable

A naming policy for output files.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileBasedSink.FilenamePolicy.Context Context used for generating a name based on shard number, and num shards.
static class	FileBasedSink.FilenamePolicy.WindowedContext Context used for generating a name based on window, pane, shard number, and num shards.

Constructor Summary

Constructors

Constructor and Description
FilenamePolicy()

Method Summary

Modifier and Type	Method and Description
void	populateDisplayData(DisplayData.Builder builder) Populates the display data.
abstract ResourceId	unwindowedFilename(ResourceId outputDirectory, FileBasedSink.FilenamePolicy.Context c, java.lang.String extension) When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a (possibly empty) extension applied by additional FileBasedSink configuration (e.g., FileBasedSink.CompressionType).
abstract ResourceId	windowedFilename(ResourceId outputDirectory, FileBasedSink.FilenamePolicy.WindowedContext c, java.lang.String extension) When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a (possibly empty) extension from FileBasedSink configuration (e.g., FileBasedSink.CompressionType).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FilenamePolicy

public FilenamePolicy()

Method Detail

windowedFilename

@Experimental(value=FILESYSTEM)
public abstract ResourceId windowedFilename(ResourceId outputDirectory,
                                                                            FileBasedSink.FilenamePolicy.WindowedContext c,
                                                                            java.lang.String extension)

When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a (possibly empty) extension from FileBasedSink configuration (e.g., FileBasedSink.CompressionType).

The FileBasedSink.FilenamePolicy.WindowedContext object gives access to the window and pane, as well as sharding information. The policy must return unique and consistent filenames for different windows and panes.

unwindowedFilename

@Experimental(value=FILESYSTEM)
 @Nullable
public abstract ResourceId unwindowedFilename(ResourceId outputDirectory,
                                                                                         FileBasedSink.FilenamePolicy.Context c,
                                                                                         java.lang.String extension)

When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a (possibly empty) extension applied by additional FileBasedSink configuration (e.g., FileBasedSink.CompressionType).

The FileBasedSink.FilenamePolicy.Context object only provides sharding information, which is used by the policy to generate unique and consistent filenames.

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Populates the display data.