org.apache.beam.sdk.io.elasticsearch

Class ElasticsearchIO.Write

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<java.lang.String>,PDone>

org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.Write

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

ElasticsearchIO

public abstract static class ElasticsearchIO.Write
extends PTransform<PCollection<java.lang.String>,PDone>

A PTransform writing data to Elasticsearch.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Write()

Method Summary

Modifier and Type	Method and Description
PDone	expand(PCollection<java.lang.String> input) Override this method to specify how this PTransform should be expanded on the given InputT.
void	validate(PipelineOptions options) Called before running the Pipeline to verify this transform is fully and correctly specified.
ElasticsearchIO.Write	withConnectionConfiguration(ElasticsearchIO.ConnectionConfiguration connectionConfiguration) Provide the Elasticsearch connection configuration object.
ElasticsearchIO.Write	withMaxBatchSize(long batchSize) Provide a maximum size in number of documents for the batch see bulk API (https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-bulk.html).
ElasticsearchIO.Write	withMaxBatchSizeBytes(long batchSizeBytes) Provide a maximum size in bytes for the batch see bulk API (https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-bulk.html).

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Write

public Write()

Method Detail

withConnectionConfiguration

public ElasticsearchIO.Write withConnectionConfiguration(ElasticsearchIO.ConnectionConfiguration connectionConfiguration)

Provide the Elasticsearch connection configuration object.

Parameters:

connectionConfiguration - the Elasticsearch ElasticsearchIO.ConnectionConfiguration object

Returns:

the ElasticsearchIO.Write with connection configuration set

withMaxBatchSize

public ElasticsearchIO.Write withMaxBatchSize(long batchSize)

Provide a maximum size in number of documents for the batch see bulk API (https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-bulk.html). Default is 1000 docs (like Elasticsearch bulk size advice). See https://www.elastic.co/guide/en/elasticsearch/guide/current/bulk.html Depending on the execution engine, size of bundles may vary, this sets the maximum size. Change this if you need to have smaller ElasticSearch bulks.

Parameters:

batchSize - maximum batch size in number of documents

Returns:

the ElasticsearchIO.Write with connection batch size set

withMaxBatchSizeBytes

public ElasticsearchIO.Write withMaxBatchSizeBytes(long batchSizeBytes)

Provide a maximum size in bytes for the batch see bulk API (https://www.elastic.co/guide/en/elasticsearch/reference/2.4/docs-bulk.html). Default is 5MB (like Elasticsearch bulk size advice). See https://www.elastic.co/guide/en/elasticsearch/guide/current/bulk.html Depending on the execution engine, size of bundles may vary, this sets the maximum size. Change this if you need to have smaller ElasticSearch bulks.

Parameters:

batchSizeBytes - maximum batch size in bytes

Returns:

the ElasticsearchIO.Write with connection batch size in bytes set

validate

public void validate(PipelineOptions options)

Description copied from class: PTransform

Called before running the Pipeline to verify this transform is fully and correctly specified.

By default, does nothing.

Overrides:

validate in class PTransform<PCollection<java.lang.String>,PDone>

expand

public PDone expand(PCollection<java.lang.String> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<java.lang.String>,PDone>