org.apache.beam.sdk.io.gcp.datastore

Class DatastoreV1.Write

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<com.google.datastore.v1.Entity>,PDone>

org.apache.beam.sdk.io.gcp.datastore.DatastoreV1.Write

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

Enclosing class:

DatastoreV1

public static class DatastoreV1.Write
extends PTransform<PCollection<com.google.datastore.v1.Entity>,PDone>

A PTransform that writes Entity objects to Cloud Datastore.

See Also:

DatastoreIO, Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method and Description
PDone	expand(PCollection<com.google.datastore.v1.Entity> input) Override this method to specify how this PTransform should be expanded on the given InputT.
java.lang.String	getDatabaseId()
java.lang.String	getProjectId()
void	populateDisplayData(DisplayData.Builder builder) Register display data for the given transform or component.
java.lang.String	toString()
DatastoreV1.Write	withDatabaseId(java.lang.String databaseId) Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the database id.
DatastoreV1.Write	withDatabaseId(ValueProvider<java.lang.String> databaseId) Same as withDatabaseId(String) but with a ValueProvider.
DatastoreV1.Write	withHintNumWorkers(int hintNumWorkers) Returns a new DatastoreV1.Write with a different worker count hint for ramp-up throttling.
DatastoreV1.Write	withHintNumWorkers(ValueProvider<java.lang.Integer> hintNumWorkers) Same as withHintNumWorkers(int) but with a ValueProvider.
DatastoreV1.Write	withLocalhost(java.lang.String localhost) Returns a new DatastoreV1.Write that writes to the Cloud Datastore Emulator running locally on the specified host port.
DatastoreV1.Write	withProjectId(java.lang.String projectId) Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the default database.
DatastoreV1.Write	withProjectId(ValueProvider<java.lang.String> projectId) Same as withProjectId(String) but with a ValueProvider.
DatastoreV1.Write	withRampupThrottlingDisabled() Returns a new DatastoreV1.Write that does not throttle during ramp-up.
DatastoreV1.WriteWithSummary	withResults() Returns DatastoreV1.WriteWithSummary transform which can be used in Wait.on(PCollection[]) to wait until all data is written.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

withProjectId

public DatastoreV1.Write withProjectId(java.lang.String projectId)

Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the default database.

withDatabaseId

public DatastoreV1.Write withDatabaseId(java.lang.String databaseId)

Returns a new DatastoreV1.Write that writes to the Cloud Datastore for the database id.

withProjectId

public DatastoreV1.Write withProjectId(ValueProvider<java.lang.String> projectId)

Same as withProjectId(String) but with a ValueProvider.

withDatabaseId

public DatastoreV1.Write withDatabaseId(ValueProvider<java.lang.String> databaseId)

Same as withDatabaseId(String) but with a ValueProvider.

withLocalhost

public DatastoreV1.Write withLocalhost(java.lang.String localhost)

Returns a new DatastoreV1.Write that writes to the Cloud Datastore Emulator running locally on the specified host port.

withRampupThrottlingDisabled

public DatastoreV1.Write withRampupThrottlingDisabled()

Returns a new DatastoreV1.Write that does not throttle during ramp-up.

withHintNumWorkers

public DatastoreV1.Write withHintNumWorkers(int hintNumWorkers)

Returns a new DatastoreV1.Write with a different worker count hint for ramp-up throttling. Value is ignored if ramp-up throttling is disabled.

withHintNumWorkers

public DatastoreV1.Write withHintNumWorkers(ValueProvider<java.lang.Integer> hintNumWorkers)

Same as withHintNumWorkers(int) but with a ValueProvider.

withResults

public DatastoreV1.WriteWithSummary withResults()

Returns DatastoreV1.WriteWithSummary transform which can be used in Wait.on(PCollection[]) to wait until all data is written.

Example: write a PCollection to one database and then to another database, making sure that writing a window of data to the second database starts only after the respective window has been fully written to the first database.

 PCollection<Entity> entities = ... ;
 PCollection<DatastoreV1.WriteSuccessSummary> writeSummary =
         entities.apply(DatastoreIO.v1().write().withProjectId(project).withResults());
 

toString

public java.lang.String toString()

Overrides:

toString in class PTransform<PCollection<com.google.datastore.v1.Entity>,PDone>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Description copied from class: PTransform

Register display data for the given transform or component.

populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect display data via DisplayData.from(HasDisplayData). Implementations may call super.populateDisplayData(builder) in order to register display data in the current namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use the namespace of the subcomponent.

By default, does not register any display data. Implementors may override this method to provide their own display data.

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class PTransform<PCollection<com.google.datastore.v1.Entity>,PDone>

Parameters:

builder - The builder to populate with display data.

See Also:

HasDisplayData

getProjectId

public java.lang.String getProjectId()

getDatabaseId

public java.lang.String getDatabaseId()

expand

public PDone expand(PCollection<com.google.datastore.v1.Entity> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<com.google.datastore.v1.Entity>,PDone>