org.apache.beam.sdk.extensions.gcp.options

Interface GcsOptions

All Superinterfaces:

ApplicationNameOptions, GcpOptions, GoogleApiDebugOptions, HasDisplayData, PipelineOptions

All Known Subinterfaces:

DataflowPipelineOptions, DataflowWorkerHarnessOptions, TestDataflowPipelineOptions

public interface GcsOptions
extends ApplicationNameOptions, GcpOptions, PipelineOptions

Options used to configure Google Cloud Storage.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	GcsOptions.ExecutorServiceFactory Returns the default ExecutorService to use within the Apache Beam SDK.
static class	GcsOptions.PathValidatorFactory Creates a PathValidator object using the class specified in getPathValidatorClass().

Nested classes/interfaces inherited from interface org.apache.beam.sdk.extensions.gcp.options.GcpOptions

GcpOptions.DefaultProjectFactory, GcpOptions.GcpTempLocationFactory, GcpOptions.GcpUserCredentialsFactory

Nested classes/interfaces inherited from interface org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions

GoogleApiDebugOptions.GoogleApiTracer

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner, PipelineOptions.JobNameFactory, PipelineOptions.NoOpMetricsSink, PipelineOptions.UserAgentFactory

Method Summary

Modifier and Type	Method and Description
java.util.concurrent.ExecutorService	getExecutorService() The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment.
java.lang.String	getGcsEndpoint() GCS endpoint to use.
java.lang.Integer	getGcsUploadBufferSizeBytes() The buffer size (in bytes) to use when uploading files to GCS.
org.apache.beam.sdk.util.GcsUtil	getGcsUtil() The GcsUtil instance that should be used to communicate with Google Cloud Storage.
PathValidator	getPathValidator() The path validator instance that should be used to validate paths.
java.lang.Class<? extends PathValidator>	getPathValidatorClass() The class of the validator that should be created and used to validate paths.
void	setExecutorService(java.util.concurrent.ExecutorService value)
void	setGcsEndpoint(java.lang.String value)
void	setGcsUploadBufferSizeBytes(java.lang.Integer bytes)
void	setGcsUtil(org.apache.beam.sdk.util.GcsUtil value)
void	setPathValidator(PathValidator validator)
void	setPathValidatorClass(java.lang.Class<? extends PathValidator> validatorClass)

Methods inherited from interface org.apache.beam.sdk.options.ApplicationNameOptions

getAppName, setAppName

Methods inherited from interface org.apache.beam.sdk.extensions.gcp.options.GcpOptions

getCredentialFactoryClass, getGcpCredential, getGcpTempLocation, getProject, getZone, setCredentialFactoryClass, setGcpCredential, setGcpTempLocation, setProject, setZone

Methods inherited from interface org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions

getGoogleApiTrace, setGoogleApiTrace

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, getJobName, getMetricsHttpSinkUrl, getMetricsPushPeriod, getMetricsSink, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, getUserAgent, outputRuntimeOptions, setJobName, setMetricsHttpSinkUrl, setMetricsPushPeriod, setMetricsSink, setOptionsId, setRunner, setStableUniqueNames, setTempLocation, setUserAgent

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Method Detail

getGcsUtil

@Default.InstanceFactory(value=org.apache.beam.sdk.util.GcsUtil.GcsUtilFactory.class)
 @Hidden
org.apache.beam.sdk.util.GcsUtil getGcsUtil()

The GcsUtil instance that should be used to communicate with Google Cloud Storage.

setGcsUtil

void setGcsUtil(org.apache.beam.sdk.util.GcsUtil value)

getExecutorService

@Default.InstanceFactory(value=GcsOptions.ExecutorServiceFactory.class)
 @Hidden
java.util.concurrent.ExecutorService getExecutorService()

The ExecutorService instance to use to create threads, can be overridden to specify an ExecutorService that is compatible with the users environment. If unset, the default is to create an ExecutorService with an unbounded number of threads; this is compatible with Google AppEngine.

setExecutorService

void setExecutorService(java.util.concurrent.ExecutorService value)

getGcsEndpoint

@Hidden
java.lang.String getGcsEndpoint()

GCS endpoint to use. If unspecified, uses the default endpoint.

setGcsEndpoint

void setGcsEndpoint(java.lang.String value)

getGcsUploadBufferSizeBytes

@Nullable
java.lang.Integer getGcsUploadBufferSizeBytes()

The buffer size (in bytes) to use when uploading files to GCS. Please see the documentation for AbstractGoogleAsyncWriteChannel.setUploadBufferSize(int) for more information on the restrictions and performance implications of this value.

setGcsUploadBufferSizeBytes

void setGcsUploadBufferSizeBytes(@Nullable
                                 java.lang.Integer bytes)

getPathValidatorClass

@Default.Class(value=GcsPathValidator.class)
java.lang.Class<? extends PathValidator> getPathValidatorClass()

The class of the validator that should be created and used to validate paths. If pathValidator has not been set explicitly, an instance of this class will be constructed and used as the path validator.

setPathValidatorClass

void setPathValidatorClass(java.lang.Class<? extends PathValidator> validatorClass)

getPathValidator

@Default.InstanceFactory(value=GcsOptions.PathValidatorFactory.class)
PathValidator getPathValidator()

The path validator instance that should be used to validate paths. If no path validator has been set explicitly, the default is to use the instance factory that constructs a path validator based upon the currently set pathValidatorClass.

setPathValidator

void setPathValidator(PathValidator validator)