Package org.apache.beam.sdk.io.json

Class JsonIO.Write<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<T>,WriteFilesResult<String>>

org.apache.beam.sdk.io.json.JsonIO.Write<T>

All Implemented Interfaces:

Serializable, HasDisplayData

Enclosing class:

JsonIO

public abstract static class JsonIO.Write<T>
extends PTransform<PCollection<T>,WriteFilesResult<String>>

PTransform for writing JSON files.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor	Description
Write()

Method Summary

Modifier and Type	Method	Description
WriteFilesResult<String>	expand(PCollection<T> input)	Override this method to specify how this PTransform should be expanded on the given InputT.
JsonIO.Write<T>	withCompression(Compression compression)	Specifies the Compression of all generated shard files.
JsonIO.Write<T>	withMaxNumWritersPerBundle(@Nullable Integer maxNumWritersPerBundle)	Set the maximum number of writers created in a bundle before spilling to shuffle.
JsonIO.Write<T>	withNoSpilling()	Whether to skip the spilling of data.
JsonIO.Write<T>	withNumShards(Integer numShards)	Specifies to use a given fixed number of shards per window.
JsonIO.Write<T>	withoutSharding()	Forces a single file as output and empty shard name template.
JsonIO.Write<T>	withShardTemplate(String shardTemplate)	Uses the given ShardNameTemplate for naming output files.
JsonIO.Write<T>	withSuffix(String suffix)	Configures the filename suffix for written files.
JsonIO.Write<T>	withTempDirectory(ResourceId tempDirectory)	Set the base directory used to generate temporary files.
JsonIO.Write<T>	withWindowedWrites()	Preserves windowing of input elements and writes them to files based on the element's window.
JsonIO.Write<T>	withWritableByteChannelFactory(FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)	Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

Write

public Write()

Method Details

withCompression

public JsonIO.Write<T> withCompression(Compression compression)

Specifies the Compression of all generated shard files.

withNoSpilling

public JsonIO.Write<T> withNoSpilling()

Whether to skip the spilling of data. See WriteFiles.withNoSpilling().

withMaxNumWritersPerBundle

public JsonIO.Write<T> withMaxNumWritersPerBundle(@Nullable Integer maxNumWritersPerBundle)

Set the maximum number of writers created in a bundle before spilling to shuffle. See

invalid reference

WriteFiles#withMaxNumWritersPerBundle()

.

withNumShards

public JsonIO.Write<T> withNumShards(Integer numShards)

Specifies to use a given fixed number of shards per window. See TextIO.Write.withNumShards(int).

withoutSharding

public JsonIO.Write<T> withoutSharding()

Forces a single file as output and empty shard name template. See TextIO.Write.withoutSharding().

withShardTemplate

public JsonIO.Write<T> withShardTemplate(String shardTemplate)

Uses the given ShardNameTemplate for naming output files. See TextIO.Write.withShardNameTemplate(java.lang.String).

withSuffix

public JsonIO.Write<T> withSuffix(String suffix)

Configures the filename suffix for written files. See TextIO.Write.withSuffix(java.lang.String).

withTempDirectory

public JsonIO.Write<T> withTempDirectory(ResourceId tempDirectory)

Set the base directory used to generate temporary files. See TextIO.Write.withTempDirectory(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId>).

withWindowedWrites

public JsonIO.Write<T> withWindowedWrites()

Preserves windowing of input elements and writes them to files based on the element's window. See TextIO.Write.withWindowedWrites().

withWritableByteChannelFactory

public JsonIO.Write<T> withWritableByteChannelFactory(FileBasedSink.WritableByteChannelFactory writableByteChannelFactory)

Returns a transform for writing to text files like this one but that has the given FileBasedSink.WritableByteChannelFactory to be used by the FileBasedSink during output. See TextIO.Write.withWritableByteChannelFactory(org.apache.beam.sdk.io.FileBasedSink.WritableByteChannelFactory).

expand

public WriteFilesResult<String> expand(PCollection<T> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<T>,WriteFilesResult<String>>