org.apache.beam.sdk.extensions.gcp.options

Interface GcpOptions

All Superinterfaces:

GoogleApiDebugOptions, HasDisplayData, PipelineOptions

All Known Subinterfaces:

BigQueryOptions, CloudResourceManagerOptions, DataflowPipelineOptions, DataflowPipelineWorkerPoolOptions, DataflowWorkerHarnessOptions, GcsOptions, PubsubOptions, TestBigQueryOptions, TestDataflowPipelineOptions, TestPubsubOptions

public interface GcpOptions
extends GoogleApiDebugOptions, PipelineOptions

Options used to configure Google Cloud Platform specific options such as the project and credentials.

These options defer to the application default credentials for authentication. See the Google Auth Library for alternative mechanisms for creating credentials.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	GcpOptions.DefaultProjectFactory Attempts to infer the default project based upon the environment this application is executing within.
static class	GcpOptions.EnableStreamingEngineFactory EnableStreamingEngine defaults to false unless one of the two experiments is set.
static class	GcpOptions.GcpOAuthScopesFactory Returns the default set of OAuth scopes.
static class	GcpOptions.GcpTempLocationFactory Returns PipelineOptions.getTempLocation() as the default GCP temp location.
static class	GcpOptions.GcpUserCredentialsFactory Attempts to load the GCP credentials.

Nested classes/interfaces inherited from interface org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner, PipelineOptions.JobNameFactory, PipelineOptions.UserAgentFactory

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	STREAMING_ENGINE_EXPERIMENT Experiment to turn on the Streaming Engine experiment.
static java.lang.String	WINDMILL_SERVICE_EXPERIMENT Deprecated. Use STREAMING_ENGINE_EXPERIMENT instead.

Method Summary

Modifier and Type	Method and Description
java.lang.Class<? extends CredentialFactory>	getCredentialFactoryClass() The class of the credential factory that should be created and used to create credentials.
@Nullable java.lang.String	getDataflowKmsKey() GCP Cloud KMS key for Dataflow pipelines and buckets created by GcpTempLocationFactory.
com.google.auth.Credentials	getGcpCredential() The credential instance that should be used to authenticate against GCP services.
java.util.List<java.lang.String>	getGcpOauthScopes() Controls the OAuth scopes that will be requested when creating Credentials with the GcpCredentialFactory (which is the default CredentialFactory).
@Nullable java.lang.String	getGcpTempLocation() A GCS path for storing temporary files in GCP.
@Nullable java.lang.String	getImpersonateServiceAccount() All API requests will be made as the given service account or target service account in an impersonation delegation chain instead of the currently selected account.
java.lang.String	getProject() Project id to use when launching jobs.
java.lang.String	getWorkerRegion() The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g.
java.lang.String	getWorkerZone() The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g.
java.lang.String	getZone() Deprecated. Use getWorkerZone() instead.
boolean	isEnableStreamingEngine()
void	setCredentialFactoryClass(java.lang.Class<? extends CredentialFactory> credentialFactoryClass)
void	setDataflowKmsKey(java.lang.String dataflowKmsKey)
void	setEnableStreamingEngine(boolean value)
void	setGcpCredential(com.google.auth.Credentials value)
void	setGcpOauthScopes(java.util.List<java.lang.String> oauthScopes)
void	setGcpTempLocation(java.lang.String value)
void	setImpersonateServiceAccount(java.lang.String impersonateServiceAccount)
void	setProject(java.lang.String value)
void	setWorkerRegion(java.lang.String workerRegion)
void	setWorkerZone(java.lang.String workerZone)
void	setZone(java.lang.String value) Deprecated. Use setWorkerZone(java.lang.String) instead.

Methods inherited from interface org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, getJobName, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, getUserAgent, outputRuntimeOptions, setJobName, setOptionsId, setRunner, setStableUniqueNames, setTempLocation, setUserAgent

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Field Detail

STREAMING_ENGINE_EXPERIMENT

static final java.lang.String STREAMING_ENGINE_EXPERIMENT

Experiment to turn on the Streaming Engine experiment.

See Also:

Constant Field Values

WINDMILL_SERVICE_EXPERIMENT

@Deprecated
static final java.lang.String WINDMILL_SERVICE_EXPERIMENT

Deprecated. Use STREAMING_ENGINE_EXPERIMENT instead.

See Also:

Constant Field Values

Method Detail

getProject

@Default.InstanceFactory(value=GcpOptions.DefaultProjectFactory.class)
java.lang.String getProject()

Project id to use when launching jobs.

setProject

void setProject(java.lang.String value)

getZone

@Deprecated
java.lang.String getZone()

Deprecated. Use getWorkerZone() instead.

GCP availability zone for operations.

Default is set on a per-service basis.

setZone

@Deprecated
void setZone(java.lang.String value)

Deprecated. Use setWorkerZone(java.lang.String) instead.

getWorkerRegion

java.lang.String getWorkerRegion()

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with getWorkerZone(). If neither workerRegion nor workerZone is specified, default to same value as region.

setWorkerRegion

void setWorkerRegion(java.lang.String workerRegion)

getWorkerZone

java.lang.String getWorkerZone()

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with getWorkerRegion(). If neither workerRegion nor workerZone is specified, the Dataflow service will choose a zone in region based on available capacity.

setWorkerZone

void setWorkerZone(java.lang.String workerZone)

getGcpOauthScopes

@Default.InstanceFactory(value=GcpOptions.GcpOAuthScopesFactory.class)
java.util.List<java.lang.String> getGcpOauthScopes()

Controls the OAuth scopes that will be requested when creating Credentials with the GcpCredentialFactory (which is the default CredentialFactory). If the credential or credential factory have been set then this field may do nothing.

setGcpOauthScopes

void setGcpOauthScopes(java.util.List<java.lang.String> oauthScopes)

getCredentialFactoryClass

@Default.Class(value=GcpCredentialFactory.class)
java.lang.Class<? extends CredentialFactory> getCredentialFactoryClass()

The class of the credential factory that should be created and used to create credentials. If gcpCredential has not been set explicitly, an instance of this class will be constructed and used as a credential factory.

setCredentialFactoryClass

void setCredentialFactoryClass(java.lang.Class<? extends CredentialFactory> credentialFactoryClass)

getGcpCredential

@Default.InstanceFactory(value=GcpOptions.GcpUserCredentialsFactory.class)
com.google.auth.Credentials getGcpCredential()

The credential instance that should be used to authenticate against GCP services. If no credential has been set explicitly, the default is to use the instance factory that constructs a credential based upon the currently set credentialFactoryClass.

setGcpCredential

void setGcpCredential(com.google.auth.Credentials value)

getImpersonateServiceAccount

@Nullable java.lang.String getImpersonateServiceAccount()

All API requests will be made as the given service account or target service account in an impersonation delegation chain instead of the currently selected account. You can specify either a single service account as the impersonator, or a comma-separated list of service accounts to create an impersonation delegation chain.

setImpersonateServiceAccount

void setImpersonateServiceAccount(java.lang.String impersonateServiceAccount)

isEnableStreamingEngine

@Default.InstanceFactory(value=GcpOptions.EnableStreamingEngineFactory.class)
boolean isEnableStreamingEngine()

setEnableStreamingEngine

void setEnableStreamingEngine(boolean value)

getGcpTempLocation

@Default.InstanceFactory(value=GcpOptions.GcpTempLocationFactory.class)
@Nullable java.lang.String getGcpTempLocation()

A GCS path for storing temporary files in GCP.

Its default to PipelineOptions.getTempLocation().

setGcpTempLocation

void setGcpTempLocation(java.lang.String value)

getDataflowKmsKey

@Experimental
@Nullable java.lang.String getDataflowKmsKey()

GCP Cloud KMS key for Dataflow pipelines and buckets created by GcpTempLocationFactory.

setDataflowKmsKey

void setDataflowKmsKey(java.lang.String dataflowKmsKey)