org.apache.beam.sdk.io.hcatalog

Class HCatalogIO.Write

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<org.apache.hive.hcatalog.data.HCatRecord>,PDone>

org.apache.beam.sdk.io.hcatalog.HCatalogIO.Write

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

HCatalogIO

public abstract static class HCatalogIO.Write
extends PTransform<PCollection<org.apache.hive.hcatalog.data.HCatRecord>,PDone>

A PTransform to write to a HCatalog managed source.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
Write()

Method Summary

Modifier and Type	Method and Description
PDone	expand(PCollection<org.apache.hive.hcatalog.data.HCatRecord> input) Override this method to specify how this PTransform should be expanded on the given InputT.
void	validate(PipelineOptions options) Called before running the Pipeline to verify this transform is fully and correctly specified.
HCatalogIO.Write	withBatchSize(long batchSize) Sets batch size for the write operation.
HCatalogIO.Write	withConfigProperties(java.util.Map<java.lang.String,java.lang.String> configProperties) Sets the configuration properties like metastore URI.
HCatalogIO.Write	withDatabase(java.lang.String database) Sets the database name.
HCatalogIO.Write	withPartition(java.util.Map<java.lang.String,java.lang.String> partition) Sets the partition details.
HCatalogIO.Write	withTable(java.lang.String table) Sets the table name to write to, the table should exist beforehand.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Write

public Write()

Method Detail

withConfigProperties

public HCatalogIO.Write withConfigProperties(java.util.Map<java.lang.String,java.lang.String> configProperties)

Sets the configuration properties like metastore URI.

withDatabase

public HCatalogIO.Write withDatabase(java.lang.String database)

Sets the database name. This is optional, assumes 'default' database if none specified

withTable

public HCatalogIO.Write withTable(java.lang.String table)

Sets the table name to write to, the table should exist beforehand.

withPartition

public HCatalogIO.Write withPartition(java.util.Map<java.lang.String,java.lang.String> partition)

Sets the partition details.

withBatchSize

public HCatalogIO.Write withBatchSize(long batchSize)

Sets batch size for the write operation. This is optional, assumes a default batch size of 1024 if not set

expand

public PDone expand(PCollection<org.apache.hive.hcatalog.data.HCatRecord> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<org.apache.hive.hcatalog.data.HCatRecord>,PDone>

validate

public void validate(PipelineOptions options)

Description copied from class: PTransform

Called before running the Pipeline to verify this transform is fully and correctly specified.

By default, does nothing.

Overrides:

validate in class PTransform<PCollection<org.apache.hive.hcatalog.data.HCatRecord>,PDone>