Package org.apache.beam.sdk.io.elasticsearch

Class ElasticsearchIO.ConnectionConfiguration

java.lang.Object
org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.ConnectionConfiguration
All Implemented Interfaces:
Serializable
Enclosing class:
ElasticsearchIO
public abstract static class ElasticsearchIO.ConnectionConfiguration extends Object implements Serializable
A POJO describing a connection configuration to Elasticsearch.
See Also:

  • Constructor Details

    • ConnectionConfiguration

      public ConnectionConfiguration()

  • Method Details

    • getAddresses

      public abstract List<String> getAddresses()

    • getUsername

      public abstract @Nullable String getUsername()

    • getPassword

      public abstract @Nullable String getPassword()

    • getApiKey

      public abstract @Nullable String getApiKey()

    • getBearerToken

      public abstract @Nullable String getBearerToken()

    • getDefaultHeaders

      public abstract @Nullable List<Header> getDefaultHeaders()

    • getKeystorePath

      public abstract @Nullable String getKeystorePath()

    • getKeystorePassword

      public abstract @Nullable String getKeystorePassword()

    • getPathPrefix

      public abstract @Nullable String getPathPrefix()

    • getIndex

      public abstract String getIndex()

    • getType

      public abstract @Nullable String getType()

    • getSocketTimeout

      public abstract @Nullable Integer getSocketTimeout()

    • getConnectTimeout

      public abstract @Nullable Integer getConnectTimeout()

    • isTrustSelfSignedCerts

      public abstract boolean isTrustSelfSignedCerts()

    • isCompressionEnabled

      public abstract boolean isCompressionEnabled()

    • create

      public static ElasticsearchIO.ConnectionConfiguration create(String[] addresses, String index, String type)
      Creates a new Elasticsearch connection configuration.
      Parameters:
      addresses - list of addresses of Elasticsearch nodes
      index - the index toward which the requests will be issued
      type - the document type toward which the requests will be issued
      Returns:
      the connection configuration object

    • create

      public static ElasticsearchIO.ConnectionConfiguration create(String[] addresses, String index)
      Creates a new Elasticsearch connection configuration with no default type.
      Parameters:
      addresses - list of addresses of Elasticsearch nodes
      index - the index toward which the requests will be issued
      Returns:
      the connection configuration object

    • create

      public static ElasticsearchIO.ConnectionConfiguration create(String[] addresses)
      Creates a new Elasticsearch connection configuration with no default index nor type.
      Parameters:
      addresses - list of addresses of Elasticsearch nodes
      Returns:
      the connection configuration object

    • getApiPrefix

      public String getApiPrefix()
      Generates the API endpoint prefix based on the set values.

      Based on ConnectionConfiguration constructors, we know that one of the following is true:

      • index and type are non-empty strings
      • index is non-empty string, type is empty string
      • index and type are empty string

      Example valid endpoints therefore include:

      • /_bulk
      • /index_name/_bulk
      • /index_name/type_name/_bulk

    • getPrefixedEndpoint

      public String getPrefixedEndpoint(String endpoint)

    • getBulkEndPoint

      public String getBulkEndPoint()

    • getSearchEndPoint

      public String getSearchEndPoint()

    • getCountEndPoint

      public String getCountEndPoint()

    • withUsername

      public ElasticsearchIO.ConnectionConfiguration withUsername(String username)
      If Elasticsearch authentication is enabled, provide the username.
      Parameters:
      username - the username used to authenticate to Elasticsearch
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withPassword

      public ElasticsearchIO.ConnectionConfiguration withPassword(String password)
      If Elasticsearch authentication is enabled, provide the password.
      Parameters:
      password - the password used to authenticate to Elasticsearch
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withPathPrefix

      public ElasticsearchIO.ConnectionConfiguration withPathPrefix(String pathPrefix)
      If Elasticsearch is not running at the root path, e.g. 'host:9200/path/to/index', specify path prefix 'path/to'.
      Parameters:
      pathPrefix - the path prefix used in Elasticsearch instance.
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withApiKey

      public ElasticsearchIO.ConnectionConfiguration withApiKey(String apiKey)
      If Elasticsearch authentication is enabled, provide an API key. Be aware that you can only use one of , withBearerToken() and withDefaultHeaders at the same time, as they (potentially) use the same header.
      Parameters:
      apiKey - the API key used to authenticate to Elasticsearch
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withBearerToken

      public ElasticsearchIO.ConnectionConfiguration withBearerToken(String bearerToken)
      If Elasticsearch authentication is enabled, provide a bearer token. Be aware that you can only use one of , withBearerToken() and withDefaultHeaders at the same time, as they (potentially) use the same header.
      Parameters:
      bearerToken - the bearer token used to authenticate to Elasticsearch
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withDefaultHeaders

      public ElasticsearchIO.ConnectionConfiguration withDefaultHeaders(Header[] defaultHeaders)
      For authentication or custom requirements, provide a set if default headers for the client. Be aware that you can only use one of withApiToken(), withBearerToken() and withDefaultHeaders at the same time, as they (potentially) use the same header.

      An example of where this could be useful is if the client needs to use short-lived credentials that need to be renewed on a certain interval. To implement that, a user could implement a custom header that tracks the renewal period, for example: 

       class OAuthTokenHeader extends BasicHeader {
     OAuthToken accessToken;

     ...

     @Override
     public String getValue() {
         if (accessToken.isExpired()) {
             accessToken.renew();
         }
         return String.format("Bearer %s", accessToken.getToken());
     }
 }
      Parameters:
      defaultHeaders - the headers to add to outgoing requests
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withKeystorePath

      public ElasticsearchIO.ConnectionConfiguration withKeystorePath(String keystorePath)
      If Elasticsearch uses SSL/TLS with mutual authentication (via shield), provide the keystore containing the client key.
      Parameters:
      keystorePath - the location of the keystore containing the client key.
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withKeystorePassword

      public ElasticsearchIO.ConnectionConfiguration withKeystorePassword(String keystorePassword)
      If Elasticsearch uses SSL/TLS with mutual authentication (via shield), provide the password to open the client keystore.
      Parameters:
      keystorePassword - the password of the client keystore.
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withTrustSelfSignedCerts

      public ElasticsearchIO.ConnectionConfiguration withTrustSelfSignedCerts(boolean trustSelfSignedCerts)
      If Elasticsearch uses SSL/TLS then configure whether to trust self signed certs or not. The default is false.
      Parameters:
      trustSelfSignedCerts - Whether to trust self signed certs
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withCompressionEnabled

      public ElasticsearchIO.ConnectionConfiguration withCompressionEnabled(boolean compressionEnabled)
      Configure whether the REST client should compress requests using gzip content encoding and add the "Accept-Encoding: gzip". The default is true.
      Parameters:
      compressionEnabled - Whether to compress requests using gzip content encoding and add the "Accept-Encoding: gzip"
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withSocketTimeout

      public ElasticsearchIO.ConnectionConfiguration withSocketTimeout(Integer socketTimeout)
      If set, overwrites the default max retry timeout (30000ms) in the Elastic RestClient and the default socket timeout (30000ms) in the RequestConfig of the Elastic RestClient.
      Parameters:
      socketTimeout - the socket and retry timeout in millis.
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.

    • withConnectTimeout

      public ElasticsearchIO.ConnectionConfiguration withConnectTimeout(Integer connectTimeout)
      If set, overwrites the default connect timeout (1000ms) in the RequestConfig of the Elastic RestClient.
      Parameters:
      connectTimeout - the socket and retry timeout in millis.
      Returns:
      a ElasticsearchIO.ConnectionConfiguration describes a connection configuration to Elasticsearch.