org.apache.beam.runners.fnexecution.artifact

Class BeamFileSystemArtifactStagingService

java.lang.Object

org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

org.apache.beam.runners.fnexecution.artifact.BeamFileSystemArtifactStagingService

All Implemented Interfaces:

java.lang.AutoCloseable, FnService, org.apache.beam.vendor.grpc.v1.io.grpc.BindableService

public class BeamFileSystemArtifactStagingService
extends org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase
implements FnService

This implementation is experimental.

ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase based on beam file system. BeamFileSystemArtifactStagingService requires StagingSessionToken in every me call. The manifest is put in StagingSessionToken#getBasePath()/StagingSessionToken#getSessionId() and artifacts are put in StagingSessionToken#getBasePath()/StagingSessionToken#getSessionId()/ARTIFACTS.

The returned token is the path to the manifest file.

The manifest file is encoded in ArtifactApi.ProxyManifest.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ARTIFACTS
static java.lang.String	MANIFEST

Constructor Summary

Constructors

Constructor and Description
BeamFileSystemArtifactStagingService()

Method Summary

Modifier and Type	Method and Description
void	close() .
void	commitManifest(org.apache.beam.model.jobmanagement.v1.ArtifactApi.CommitManifestRequest request, org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.CommitManifestResponse> responseObserver)
static java.lang.String	generateStagingSessionToken(java.lang.String sessionId, java.lang.String basePath) Generate a stagingSessionToken compatible with BeamFileSystemArtifactStagingService.
org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.PutArtifactRequest>	putArtifact(org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.PutArtifactResponse> responseObserver)
void	removeArtifacts(java.lang.String stagingSessionToken)

Methods inherited from class org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

bindService

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.vendor.grpc.v1.io.grpc.BindableService

bindService

Field Detail

MANIFEST

public static final java.lang.String MANIFEST

See Also:

Constant Field Values

ARTIFACTS

public static final java.lang.String ARTIFACTS

See Also:

Constant Field Values

Constructor Detail

BeamFileSystemArtifactStagingService

public BeamFileSystemArtifactStagingService()

Method Detail

putArtifact

public org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.PutArtifactRequest> putArtifact(org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.PutArtifactResponse> responseObserver)

Overrides:

putArtifact in class org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

commitManifest

public void commitManifest(org.apache.beam.model.jobmanagement.v1.ArtifactApi.CommitManifestRequest request,
                           org.apache.beam.vendor.grpc.v1.io.grpc.stub.StreamObserver<org.apache.beam.model.jobmanagement.v1.ArtifactApi.CommitManifestResponse> responseObserver)

Overrides:

commitManifest in class org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceImplBase

close

public void close()
           throws java.lang.Exception

Description copied from interface: FnService

.

There should be no more calls to any service method by the time a call to FnService.close() begins. Specifically, this means that a Server that this service is bound to should have completed a call to the Server.shutdown() method, and all future incoming calls will be rejected.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface FnService

Throws:

java.lang.Exception

generateStagingSessionToken

public static java.lang.String generateStagingSessionToken(java.lang.String sessionId,
                                                           java.lang.String basePath)
                                                    throws java.lang.Exception

Generate a stagingSessionToken compatible with BeamFileSystemArtifactStagingService.

Parameters:

sessionId - Unique sessionId for artifact staging.

basePath - Base path to upload artifacts.

Returns:

Encoded stagingSessionToken.

Throws:

java.lang.Exception

removeArtifacts

public void removeArtifacts(java.lang.String stagingSessionToken)
                     throws java.lang.Exception

Throws:

java.lang.Exception