org.apache.beam.sdk.io

Class TextIO.Write

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<java.lang.String>,PDone>

org.apache.beam.sdk.io.TextIO.Write

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

TextIO

public static class TextIO.Write
extends PTransform<PCollection<java.lang.String>,PDone>

This class is used as the default return value of TextIO.write().

All methods in this class delegate to the appropriate method of TextIO.TypedWrite. This class exists for backwards compatibility, and will be removed in Beam 3.0.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method and Description
PDone	expand(PCollection<java.lang.String> input) Override this method to specify how this PTransform should be expanded on the given InputT.
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
TextIO.Write	to(FileBasedSink.DynamicDestinations<java.lang.String,?,java.lang.String> dynamicDestinations) Deprecated. Use FileIO.write() or FileIO.writeDynamic() ()} with TextIO.sink() instead.
TextIO.Write	to(FileBasedSink.FilenamePolicy filenamePolicy) See TypedWrite#to(FilenamePolicy).
TextIO.Write	to(ResourceId filenamePrefix) See TextIO.TypedWrite.to(ResourceId).
TextIO.Write	to(SerializableFunction<java.lang.String,DefaultFilenamePolicy.Params> destinationFunction, DefaultFilenamePolicy.Params emptyDestination) Deprecated. Use FileIO.write() or FileIO.writeDynamic() ()} with TextIO.sink() instead.
TextIO.Write	to(java.lang.String filenamePrefix) See TextIO.TypedWrite.to(String).
TextIO.Write	to(ValueProvider<java.lang.String> outputPrefix) See TextIO.TypedWrite.to(ValueProvider).
TextIO.Write	toResource(ValueProvider<ResourceId> filenamePrefix) See TextIO.TypedWrite.toResource(ValueProvider).
TextIO.Write	withAutoSharding() See TextIO.TypedWrite.withAutoSharding().
TextIO.Write	withBatchMaxBufferingDuration(@Nullable Duration batchMaxBufferingDuration) See TextIO.TypedWrite.withBatchMaxBufferingDuration(Duration).
TextIO.Write	withBatchSize(@Nullable java.lang.Integer batchSize) See TextIO.TypedWrite.withBatchSize(Integer).
TextIO.Write	withBatchSizeBytes(@Nullable java.lang.Integer batchSizeBytes) See TextIO.TypedWrite.withBatchSizeBytes(Integer).
TextIO.Write	withCompression(Compression compression) See TextIO.TypedWrite.withCompression(Compression).
TextIO.Write	withDelimiter(char[] delimiter) See TextIO.TypedWrite.withDelimiter(char[]).
TextIO.Write	withFooter(@Nullable java.lang.String footer) See TextIO.TypedWrite.withFooter(String).
TextIO.Write	withHeader(@Nullable java.lang.String header) See TextIO.TypedWrite.withHeader(String).
TextIO.Write	withNoSpilling() See TextIO.TypedWrite.withNoSpilling().
TextIO.Write	withNumShards(int numShards) See TextIO.TypedWrite.withNumShards(int).
TextIO.Write	withNumShards(@Nullable ValueProvider<java.lang.Integer> numShards) See TextIO.TypedWrite.withNumShards(ValueProvider).
<DestinationT> TextIO.TypedWrite<java.lang.String,DestinationT>	withOutputFilenames() Specify that output filenames are wanted.
TextIO.Write	withoutSharding() See TextIO.TypedWrite.withoutSharding().
TextIO.Write	withShardNameTemplate(java.lang.String shardTemplate) See TextIO.TypedWrite.withShardNameTemplate(String).
TextIO.Write	withSuffix(java.lang.String filenameSuffix) See TextIO.TypedWrite.withSuffix(String).
TextIO.Write	withTempDirectory(ResourceId tempDirectory) See TextIO.TypedWrite.withTempDirectory(ResourceId).
TextIO.Write	withTempDirectory(ValueProvider<ResourceId> tempDirectory) See TextIO.TypedWrite.withTempDirectory(ValueProvider).
TextIO.Write	withWindowedWrites() See TextIO.TypedWrite.withWindowedWrites().
TextIO.Write	withWritableByteChannelFactory(FileBasedSink.WritableByteChannelFactory writableByteChannelFactory) See TypedWrite#withWritableByteChannelFactory(WritableByteChannelFactory).

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

to

public TextIO.Write to(java.lang.String filenamePrefix)

See TextIO.TypedWrite.to(String).

to

public TextIO.Write to(ResourceId filenamePrefix)

See TextIO.TypedWrite.to(ResourceId).

to

public TextIO.Write to(ValueProvider<java.lang.String> outputPrefix)

See TextIO.TypedWrite.to(ValueProvider).

toResource

public TextIO.Write toResource(ValueProvider<ResourceId> filenamePrefix)

See TextIO.TypedWrite.toResource(ValueProvider).

to

public TextIO.Write to(FileBasedSink.FilenamePolicy filenamePolicy)

See TypedWrite#to(FilenamePolicy).

to

@Deprecated
public TextIO.Write to(FileBasedSink.DynamicDestinations<java.lang.String,?,java.lang.String> dynamicDestinations)

Deprecated. Use FileIO.write() or FileIO.writeDynamic() ()} with TextIO.sink() instead.

See TypedWrite#to(DynamicDestinations).

to

@Deprecated
public TextIO.Write to(SerializableFunction<java.lang.String,DefaultFilenamePolicy.Params> destinationFunction,
                                   DefaultFilenamePolicy.Params emptyDestination)

Deprecated. Use FileIO.write() or FileIO.writeDynamic() ()} with TextIO.sink() instead.

See TypedWrite#to(SerializableFunction, Params).

withTempDirectory

public TextIO.Write withTempDirectory(ValueProvider<ResourceId> tempDirectory)

See TextIO.TypedWrite.withTempDirectory(ValueProvider).

withTempDirectory

public TextIO.Write withTempDirectory(ResourceId tempDirectory)

See TextIO.TypedWrite.withTempDirectory(ResourceId).

withShardNameTemplate

public TextIO.Write withShardNameTemplate(java.lang.String shardTemplate)

See TextIO.TypedWrite.withShardNameTemplate(String).

withSuffix

public TextIO.Write withSuffix(java.lang.String filenameSuffix)

See TextIO.TypedWrite.withSuffix(String).

withNumShards

public TextIO.Write withNumShards(int numShards)

See TextIO.TypedWrite.withNumShards(int).

withNumShards

public TextIO.Write withNumShards(@Nullable ValueProvider<java.lang.Integer> numShards)

See TextIO.TypedWrite.withNumShards(ValueProvider).

withoutSharding

public TextIO.Write withoutSharding()

See TextIO.TypedWrite.withoutSharding().

withDelimiter

public TextIO.Write withDelimiter(char[] delimiter)

See TextIO.TypedWrite.withDelimiter(char[]).

withHeader

public TextIO.Write withHeader(@Nullable java.lang.String header)

See TextIO.TypedWrite.withHeader(String).

withFooter

public TextIO.Write withFooter(@Nullable java.lang.String footer)

See TextIO.TypedWrite.withFooter(String).

withWritableByteChannelFactory

public TextIO.Write withWritableByteChannelFactory(FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)

See TypedWrite#withWritableByteChannelFactory(WritableByteChannelFactory).

withCompression

public TextIO.Write withCompression(Compression compression)

See TextIO.TypedWrite.withCompression(Compression).

withWindowedWrites

public TextIO.Write withWindowedWrites()

See TextIO.TypedWrite.withWindowedWrites().

withAutoSharding

public TextIO.Write withAutoSharding()

See TextIO.TypedWrite.withAutoSharding().

withNoSpilling

public TextIO.Write withNoSpilling()

See TextIO.TypedWrite.withNoSpilling().

withBatchSize

public TextIO.Write withBatchSize(@Nullable java.lang.Integer batchSize)

See TextIO.TypedWrite.withBatchSize(Integer).

withBatchSizeBytes

public TextIO.Write withBatchSizeBytes(@Nullable java.lang.Integer batchSizeBytes)

See TextIO.TypedWrite.withBatchSizeBytes(Integer).

withBatchMaxBufferingDuration

public TextIO.Write withBatchMaxBufferingDuration(@Nullable Duration batchMaxBufferingDuration)

See TextIO.TypedWrite.withBatchMaxBufferingDuration(Duration).

withOutputFilenames

public <DestinationT> TextIO.TypedWrite<java.lang.String,DestinationT> withOutputFilenames()

Specify that output filenames are wanted.

The nested TextIO.TypedWritetransform always has access to output filenames, however due to backwards-compatibility concerns, TextIO.Write cannot return them. This method simply returns the inner TextIO.TypedWrite transform which has WriteFilesResult as its output type, allowing access to output files.

The supplied DestinationT type must be: the same as that supplied in #to(DynamicDestinations) if that method was used; DefaultFilenamePolicy.Params if #to(SerializableFunction, Params) was used, or Void otherwise.

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PCollection<java.lang.String>,PDone>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

expand

public PDone expand(PCollection<java.lang.String> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<java.lang.String>,PDone>