org.apache.beam.runners.fnexecution.artifact

Class BeamFileSystemArtifactStagingService

java.lang.Object

org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

org.apache.beam.runners.fnexecution.artifact.AbstractArtifactStagingService

org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService

All Implemented Interfaces:

java.lang.AutoCloseable, FnService, org.apache.beam.vendor.grpc.v1p26p0.io.grpc.BindableService

public class BeamFileSystemArtifactStagingService
extends AbstractArtifactStagingService

This implementation is experimental.

ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase based on beam file system. BeamFileSystemArtifactStagingService requires BeamFileSystemArtifactStagingService.StagingSessionToken in every me call. The manifest is put in BeamFileSystemArtifactStagingService.StagingSessionToken.getBasePath()/BeamFileSystemArtifactStagingService.StagingSessionToken.getSessionId() and artifacts are put in BeamFileSystemArtifactStagingService.StagingSessionToken.getBasePath()/BeamFileSystemArtifactStagingService.StagingSessionToken.getSessionId()/ARTIFACTS.

The returned token is the path to the manifest file.

The manifest file is encoded in ArtifactApi.ProxyManifest.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BeamFileSystemArtifactStagingService.StagingSessionToken Serializable StagingSessionToken used to stage files with BeamFileSystemArtifactStagingService.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ARTIFACTS
static java.lang.String	MANIFEST

Fields inherited from class org.apache.beam.runners.fnexecution.artifact.AbstractArtifactStagingService

NO_ARTIFACTS_STAGED_TOKEN

Constructor Summary

Constructors

Constructor and Description
BeamFileSystemArtifactStagingService()

Method Summary

Modifier and Type	Method and Description
static java.lang.String	generateStagingSessionToken(java.lang.String sessionId, java.lang.String basePath) Generate a stagingSessionToken compatible with BeamFileSystemArtifactStagingService.
java.lang.String	getArtifactUri(java.lang.String stagingSession, java.lang.String encodedFileName)
java.lang.String	getRetrievalToken(java.lang.String stagingSession)
java.nio.channels.WritableByteChannel	openManifest(java.lang.String stagingSession)
java.nio.channels.WritableByteChannel	openUri(java.lang.String uri)
void	removeArtifacts(java.lang.String stagingSessionToken)
void	removeUri(java.lang.String uri)

Methods inherited from class org.apache.beam.runners.fnexecution.artifact.AbstractArtifactStagingService

close, commitManifest, putArtifact

Methods inherited from class org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

bindService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.vendor.grpc.v1p26p0.io.grpc.BindableService

bindService

Field Detail

MANIFEST

public static final java.lang.String MANIFEST

See Also:

Constant Field Values

ARTIFACTS

public static final java.lang.String ARTIFACTS

See Also:

Constant Field Values

Constructor Detail

BeamFileSystemArtifactStagingService

public BeamFileSystemArtifactStagingService()

Method Detail

getArtifactUri

public java.lang.String getArtifactUri(java.lang.String stagingSession,
                                       java.lang.String encodedFileName)
                                throws java.lang.Exception

Specified by:

getArtifactUri in class AbstractArtifactStagingService

Throws:

java.lang.Exception

openUri

public java.nio.channels.WritableByteChannel openUri(java.lang.String uri)
                                              throws java.io.IOException

Specified by:

openUri in class AbstractArtifactStagingService

Throws:

java.io.IOException

removeUri

public void removeUri(java.lang.String uri)
               throws java.io.IOException

Specified by:

removeUri in class AbstractArtifactStagingService

Throws:

java.io.IOException

removeArtifacts

public void removeArtifacts(java.lang.String stagingSessionToken)
                     throws java.lang.Exception

Specified by:

removeArtifacts in class AbstractArtifactStagingService

Throws:

java.lang.Exception

openManifest

public java.nio.channels.WritableByteChannel openManifest(java.lang.String stagingSession)
                                                   throws java.lang.Exception

Specified by:

openManifest in class AbstractArtifactStagingService

Throws:

java.lang.Exception

getRetrievalToken

public java.lang.String getRetrievalToken(java.lang.String stagingSession)
                                   throws java.lang.Exception

Specified by:

getRetrievalToken in class AbstractArtifactStagingService

Throws:

java.lang.Exception

generateStagingSessionToken

public static java.lang.String generateStagingSessionToken(java.lang.String sessionId,
                                                           java.lang.String basePath)

Generate a stagingSessionToken compatible with BeamFileSystemArtifactStagingService.

Parameters:

sessionId - Unique sessionId for artifact staging.

basePath - Base path to upload artifacts.

Returns:

Encoded stagingSessionToken.