org.apache.beam.it.gcp.storage

Class GcsResourceManager

java.lang.Object

org.apache.beam.it.gcp.storage.GcsResourceManager

All Implemented Interfaces:

org.apache.beam.it.common.ResourceManager, ArtifactClient

public final class GcsResourceManager
extends java.lang.Object
implements ArtifactClient, org.apache.beam.it.common.ResourceManager

Client for working with test artifacts stored in Google Cloud Storage (GCS).

Tests should store this as a static value of the class and call ArtifactClient.cleanupAll() in the @AfterClass method.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GcsResourceManager.Builder Builder for GcsResourceManager.

Constructor Summary

Constructors

Constructor and Description
GcsResourceManager(GcsResourceManager.Builder builder)

Method Summary

Modifier and Type	Method and Description
static GcsResourceManager.Builder	builder(java.lang.String bucket, java.lang.String testClassName, com.google.auth.Credentials credentials) Returns a new GcsResourceManager.Builder for configuring a client.
void	cleanupAll() Deletes all the files located under test-class-name/run-id.
Artifact	copyFileToGcs(java.nio.file.Path localPath, java.lang.String objectName) Copies a file from a local path to a specified object name in Google Cloud Storage.
Artifact	createArtifact(java.lang.String artifactName, byte[] contents) Creates a new artifact in whatever service is being used to store them.
Artifact	createArtifact(java.lang.String artifactName, java.lang.String contents) Creates a new artifact in whatever service is being used to store them.
com.google.cloud.storage.Notification	createNotification(java.lang.String topicName, java.lang.String gcsPrefix) Creates a new notification for the given topic and GCS prefix.
java.lang.String	getPathForArtifact(java.lang.String artifactName) Returns a path the artifact will be created at.
java.util.List<Artifact>	listArtifacts(java.lang.String prefix, java.util.regex.Pattern regex) Lists all artifacts under test-class-name/run-id/prefix.
java.util.List<Artifact>	listArtifacts(TestName testName, java.util.regex.Pattern regex) Lists all artifacts under test-class-name/run-id/prefix.
void	registerTempDir(java.lang.String dirName) Register a temporary directory that will be cleaned up after test.
java.lang.String	runId() Returns the id associated with the particular run of the test class.
Artifact	uploadArtifact(java.lang.String artifactName, java.nio.file.Path localPath) Uploads a local file to the service being used for storing artifacts.
Artifact	uploadArtifact(java.lang.String artifactName, java.lang.String localPath) Uploads a local file to the service being used for storing artifacts.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GcsResourceManager

public GcsResourceManager(GcsResourceManager.Builder builder)

Method Detail

builder

public static GcsResourceManager.Builder builder(java.lang.String bucket,
                                                 java.lang.String testClassName,
                                                 com.google.auth.Credentials credentials)

Returns a new GcsResourceManager.Builder for configuring a client.

runId

public java.lang.String runId()

Description copied from interface: ArtifactClient

Returns the id associated with the particular run of the test class.

Specified by:

runId in interface ArtifactClient

getPathForArtifact

public java.lang.String getPathForArtifact(java.lang.String artifactName)

Description copied from interface: ArtifactClient

Returns a path the artifact will be created at.

Specified by:

getPathForArtifact in interface ArtifactClient

Parameters:

artifactName - Artifact name

Returns:

GCS path where the artifact will be created

createArtifact

public Artifact createArtifact(java.lang.String artifactName,
                               java.lang.String contents)

Description copied from interface: ArtifactClient

Creates a new artifact in whatever service is being used to store them.

Specified by:

createArtifact in interface ArtifactClient

Parameters:

artifactName - the name of the artifact. If this is supposed to go under an input/output directory, then it should include that (example: input/artifact.txt)

contents - the contents of the artifact in String format

Returns:

a representation of the created artifact

createArtifact

public Artifact createArtifact(java.lang.String artifactName,
                               byte[] contents)

Description copied from interface: ArtifactClient

Creates a new artifact in whatever service is being used to store them.

Specified by:

createArtifact in interface ArtifactClient

Parameters:

artifactName - the name of the artifact. If this is supposed to go under an input/output directory, then it should include that (example: input/artifact.txt)

contents - the contents of the artifact in byte array format

Returns:

a representation of the created artifact

uploadArtifact

public Artifact uploadArtifact(java.lang.String artifactName,
                               java.lang.String localPath)
                        throws java.io.IOException

Description copied from interface: ArtifactClient

Uploads a local file to the service being used for storing artifacts.

Specified by:

uploadArtifact in interface ArtifactClient

Parameters:

artifactName - the name of the artifact. If this is supposed to go under an input/output directory, then it should include that (example: input/artifact.txt)

localPath - the absolute local path to the file to upload

Returns:

a representation of the uploaded artifact

Throws:

java.io.IOException - if there is an issue reading the local file

uploadArtifact

public Artifact uploadArtifact(java.lang.String artifactName,
                               java.nio.file.Path localPath)
                        throws java.io.IOException

Description copied from interface: ArtifactClient

Uploads a local file to the service being used for storing artifacts.

Specified by:

uploadArtifact in interface ArtifactClient

Parameters:

artifactName - the name of the artifact. If this is supposed to go under an input/output directory, then it should include that (example: input/artifact.txt)

localPath - the local path to the file to upload

Returns:

a representation of the uploaded artifact

Throws:

java.io.IOException - if there is an issue reading the local file

copyFileToGcs

public Artifact copyFileToGcs(java.nio.file.Path localPath,
                              java.lang.String objectName)
                       throws java.io.IOException

Copies a file from a local path to a specified object name in Google Cloud Storage.

Parameters:

localPath - the path of the file to be copied.

objectName - the name of the object to be created in Google Cloud Storage.

Returns:

the URI of the copied object in Google Cloud Storage.

Throws:

java.io.IOException - if there is an error reading the file at the specified local path.

listArtifacts

public java.util.List<Artifact> listArtifacts(TestName testName,
                                              java.util.regex.Pattern regex)

Description copied from interface: ArtifactClient

Lists all artifacts under test-class-name/run-id/prefix.

Specified by:

listArtifacts in interface ArtifactClient

Parameters:

testName - the test name to use as the prefix on the testing artifacts.

regex - a regex to use for filtering out unwanted artifacts

Returns:

all the artifacts whose name matches regex

listArtifacts

public java.util.List<Artifact> listArtifacts(java.lang.String prefix,
                                              java.util.regex.Pattern regex)

Description copied from interface: ArtifactClient

Lists all artifacts under test-class-name/run-id/prefix.

Specified by:

listArtifacts in interface ArtifactClient

Parameters:

prefix - the prefix to use along with the fixed values method above. This must include the test-method-name value, but it can include other directories or files under it.

regex - a regex to use for filtering out unwanted artifacts

Returns:

all the artifacts whose name matches regex

createNotification

public com.google.cloud.storage.Notification createNotification(java.lang.String topicName,
                                                                java.lang.String gcsPrefix)

Creates a new notification for the given topic and GCS prefix.

Parameters:

topicName - the name of the Pub/Sub topic to which the notification should be sent.

gcsPrefix - the prefix of the object names to which the notification applies.

Returns:

the created notification.

registerTempDir

public void registerTempDir(java.lang.String dirName)

Register a temporary directory that will be cleaned up after test.

Parameters:

dirName - name of the temporary directory

cleanupAll

public void cleanupAll()

Description copied from interface: ArtifactClient

Deletes all the files located under test-class-name/run-id.

Specified by:

cleanupAll in interface org.apache.beam.it.common.ResourceManager

Specified by:

cleanupAll in interface ArtifactClient