org.apache.beam.sdk.io.aws2.options

Interface AwsOptions

All Superinterfaces:

HasDisplayData, PipelineOptions

All Known Subinterfaces:

S3Options

@Experimental(value=SOURCE_SINK)
public interface AwsOptions
extends PipelineOptions

Options used to configure Amazon Web Services specific options such as credentials and region.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	AwsOptions.AwsRegionFactory Attempt to load default region.
static class	AwsOptions.AwsUserCredentialsFactory Return DefaultCredentialsProvider as default provider.

Nested classes/interfaces inherited from interface org.apache.beam.sdk.options.PipelineOptions

PipelineOptions.AtomicLongFactory, PipelineOptions.CheckEnabled, PipelineOptions.DirectRunner, PipelineOptions.JobNameFactory, PipelineOptions.UserAgentFactory

Method Summary

Modifier and Type	Method and Description
AwsCredentialsProvider	getAwsCredentialsProvider() AwsCredentialsProvider used to configure AWS service clients.
Region	getAwsRegion() Region used to configure AWS service clients.
java.lang.Class<? extends ClientBuilderFactory>	getClientBuilderFactory()
java.net.URI	getEndpoint() Endpoint used to configure AWS service clients.
HttpClientConfiguration	getHttpClientConfiguration() HttpClientConfiguration used to configure AWS service clients.
software.amazon.awssdk.http.apache.ProxyConfiguration	getProxyConfiguration() ProxyConfiguration used to configure AWS service clients.
void	setAwsCredentialsProvider(AwsCredentialsProvider value)
void	setAwsRegion(Region region)
void	setClientBuilderFactory(java.lang.Class<? extends ClientBuilderFactory> clazz)
void	setEndpoint(java.net.URI uri)
void	setHttpClientConfiguration(HttpClientConfiguration value)
void	setProxyConfiguration(software.amazon.awssdk.http.apache.ProxyConfiguration value)

Methods inherited from interface org.apache.beam.sdk.options.PipelineOptions

as, getJobName, getOptionsId, getRunner, getStableUniqueNames, getTempLocation, getUserAgent, outputRuntimeOptions, setJobName, setOptionsId, setRunner, setStableUniqueNames, setTempLocation, setUserAgent

Methods inherited from interface org.apache.beam.sdk.transforms.display.HasDisplayData

populateDisplayData

Method Detail

getAwsRegion

@Default.InstanceFactory(value=AwsOptions.AwsRegionFactory.class)
Region getAwsRegion()

Region used to configure AWS service clients.

setAwsRegion

void setAwsRegion(Region region)

getEndpoint

java.net.URI getEndpoint()

Endpoint used to configure AWS service clients.

setEndpoint

void setEndpoint(java.net.URI uri)

getAwsCredentialsProvider

@Default.InstanceFactory(value=AwsOptions.AwsUserCredentialsFactory.class)
AwsCredentialsProvider getAwsCredentialsProvider()

AwsCredentialsProvider used to configure AWS service clients.

The class name of the provider must be set in the @type field. Note: Not all available providers are supported and some configuration options might be ignored.

Most providers rely on system's environment to follow AWS conventions, there's no further configuration:

DefaultCredentialsProvider EnvironmentVariableCredentialsProvider SystemPropertyCredentialsProvider ProfileCredentialsProvider ContainerCredentialsProvider

Example:

 --awsCredentialsProvider={"@type": "ProfileCredentialsProvider"}

Some other providers require additional configuration:

StaticCredentialsProvider StsAssumeRoleCredentialsProvider

Examples:

 --awsCredentialsProvider={
   "@type": "StaticCredentialsProvider",
   "awsAccessKeyId": "key_id_value",
   "awsSecretKey": "secret_value"
 }

 --awsCredentialsProvider={
   "@type": "StaticCredentialsProvider",
   "awsAccessKeyId": "key_id_value",
   "awsSecretKey": "secret_value",
   "sessionToken": "token_value"
 }

 --awsCredentialsProvider={
   "@type": "StsAssumeRoleCredentialsProvider",
   "roleArn": "role_arn_Value",
   "roleSessionName": "session_name_value",
   "policy": "policy_value",
   "durationSeconds": 3600
 }

See Also:

DefaultCredentialsProvider

setAwsCredentialsProvider

void setAwsCredentialsProvider(AwsCredentialsProvider value)

getProxyConfiguration

software.amazon.awssdk.http.apache.ProxyConfiguration getProxyConfiguration()

ProxyConfiguration used to configure AWS service clients.

Note, only the options shown in the example below are supported. username and password are optional.

Example:

 --proxyConfiguration={
   "endpoint": "http://hostname:port",
   "username": "username",
   "password": "password"
 }

setProxyConfiguration

void setProxyConfiguration(software.amazon.awssdk.http.apache.ProxyConfiguration value)

getHttpClientConfiguration

HttpClientConfiguration getHttpClientConfiguration()

HttpClientConfiguration used to configure AWS service clients.

Example:

 --httpClientConfiguration={"socketTimeout":1000, "maxConnections":10}

setHttpClientConfiguration

void setHttpClientConfiguration(HttpClientConfiguration value)

getClientBuilderFactory

@Default.Class(value=ClientBuilderFactory.DefaultClientBuilder.class)
java.lang.Class<? extends ClientBuilderFactory> getClientBuilderFactory()

setClientBuilderFactory

void setClientBuilderFactory(java.lang.Class<? extends ClientBuilderFactory> clazz)