T
- the type of values to write.public abstract static class FileBasedSink.Writer<T>
extends java.lang.Object
FileBasedSink
. Subclass
implementations provide a method that can write a single value to a
WritableByteChannel
.
Subclass implementations may also override methods that write headers and footers before and after the values in a bundle, respectively, as well as provide a MIME type for the output channel.
Multiple FileBasedSink.Writer
instances may be created on the same worker, and therefore
any access to static members or methods should be thread safe.
Constructor and Description |
---|
Writer(FileBasedSink.WriteOperation<T> writeOperation,
java.lang.String mimeType)
Construct a new
FileBasedSink.Writer that will produce files of the given MIME type. |
Modifier and Type | Method and Description |
---|---|
void |
cleanup() |
FileBasedSink.FileResult |
close()
Closes the channel and returns the bundle result.
|
protected void |
finishWrite()
|
FileBasedSink.WriteOperation<T> |
getWriteOperation()
Return the WriteOperation that this Writer belongs to.
|
void |
openUnwindowed(java.lang.String uId,
int shard)
Similar to
openWindowed(java.lang.String, org.apache.beam.sdk.transforms.windowing.BoundedWindow, org.apache.beam.sdk.transforms.windowing.PaneInfo, int) however for the case where unwindowed writes were
requested. |
void |
openWindowed(java.lang.String uId,
BoundedWindow window,
PaneInfo paneInfo,
int shard)
Performs bundle initialization.
|
protected abstract void |
prepareWrite(java.nio.channels.WritableByteChannel channel)
Called with the channel that a subclass will write its header, footer, and values to.
|
abstract void |
write(T value)
Called for each value in the bundle.
|
protected void |
writeFooter()
Writes footer at the end of output files.
|
protected void |
writeHeader()
Writes header at the beginning of output files.
|
public Writer(FileBasedSink.WriteOperation<T> writeOperation, java.lang.String mimeType)
FileBasedSink.Writer
that will produce files of the given MIME type.protected abstract void prepareWrite(java.nio.channels.WritableByteChannel channel) throws java.lang.Exception
Called before any subsequent calls to writeHeader, writeFooter, and write.
java.lang.Exception
protected void writeHeader() throws java.lang.Exception
java.lang.Exception
protected void writeFooter() throws java.lang.Exception
java.lang.Exception
protected void finishWrite() throws java.lang.Exception
writeHeader()
, write(T)
and writeFooter()
.
If any resources opened in the write processes need to be flushed, flush them here.java.lang.Exception
public final void openWindowed(java.lang.String uId, BoundedWindow window, PaneInfo paneInfo, int shard) throws java.lang.Exception
write(T)
.
The unique id that is given to open should be used to ensure that the writer's output does not interfere with the output of other Writers, as a bundle may be executed many times for fault tolerance.
The window and paneInfo arguments are populated when windowed writes are requested. shard id populated for the case of static sharding. In cases where the runner is dynamically picking sharding, shard might be set to -1.
java.lang.Exception
public abstract void write(T value) throws java.lang.Exception
java.lang.Exception
public final void openUnwindowed(java.lang.String uId, int shard) throws java.lang.Exception
openWindowed(java.lang.String, org.apache.beam.sdk.transforms.windowing.BoundedWindow, org.apache.beam.sdk.transforms.windowing.PaneInfo, int)
however for the case where unwindowed writes were
requested.java.lang.Exception
public final void cleanup() throws java.lang.Exception
java.lang.Exception
public final FileBasedSink.FileResult close() throws java.lang.Exception
java.lang.Exception
public FileBasedSink.WriteOperation<T> getWriteOperation()