Package org.apache.beam.sdk.io

Class FileBasedSink.FilenamePolicy

java.lang.Object

org.apache.beam.sdk.io.FileBasedSink.FilenamePolicy

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

DefaultFilenamePolicy

Enclosing class:

FileBasedSink<UserT,DestinationT,OutputT>

public abstract static class FileBasedSink.FilenamePolicy
extends Object
implements Serializable

A naming policy for output files.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
FilenamePolicy()

Method Summary

Modifier and Type	Method	Description
void	populateDisplayData(DisplayData.Builder builder)	Populates the display data.
abstract @Nullable ResourceId	unwindowedFilename(int shardNumber, int numShards, FileBasedSink.OutputFileHints outputFileHints)	When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested (e.g.
abstract ResourceId	windowedFilename(int shardNumber, int numShards, BoundedWindow window, PaneInfo paneInfo, FileBasedSink.OutputFileHints outputFileHints)	When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested extension (e.g.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FilenamePolicy

public FilenamePolicy()

Method Details

windowedFilename

public abstract ResourceId windowedFilename(int shardNumber,
 int numShards,
 BoundedWindow window,
 PaneInfo paneInfo,
 FileBasedSink.OutputFileHints outputFileHints)

When a sink has requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested extension (e.g. coming from Compression).

The policy must return unique and consistent filenames for different windows and panes.

unwindowedFilename

public abstract @Nullable ResourceId unwindowedFilename(int shardNumber,
 int numShards,
 FileBasedSink.OutputFileHints outputFileHints)

When a sink has not requested windowed or triggered output, this method will be invoked to return the file resource to be created given the base output directory and a FileBasedSink.OutputFileHints containing information about the file, including a suggested (e.g. coming from Compression).

The shardNumber and numShards parameters, should be used by the policy to generate unique and consistent filenames.

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Populates the display data.