org.apache.beam.runners.jobsubmission

Class PortablePipelineJarCreator

java.lang.Object

org.apache.beam.runners.jobsubmission.PortablePipelineJarCreator

All Implemented Interfaces:

PortablePipelineRunner

public class PortablePipelineJarCreator
extends java.lang.Object
implements PortablePipelineRunner

PortablePipelineRunner that bundles the input pipeline along with all dependencies, artifacts, etc. required to run the pipeline into a jar that can be executed later.

Each PortablePipelineJarCreator instance is not threadsafe; a new instance is expected to be constructed and run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline, org.apache.beam.runners.fnexecution.provisioning.JobInfo) once per job.

Constructor Summary

Constructors

Constructor and Description
PortablePipelineJarCreator(java.lang.Class mainClass)

Method Summary

Modifier and Type	Method and Description
protected void	copyResourcesFromJar(java.util.jar.JarFile inputJar) Copy resources from inputJar to outputStream.
PortablePipelineResult	run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline, JobInfo jobInfo) Does not actually run the pipeline. Instead bundles the input pipeline along with all dependencies, artifacts, etc.
protected org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline	writeArtifacts(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline, java.lang.String jobName) Stages all dependencies in pipeline into the jar file at outputStream, returning a new pipeline that references these artifacts as classpath artifacts.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PortablePipelineJarCreator

public PortablePipelineJarCreator(java.lang.Class mainClass)

Method Detail

run

public PortablePipelineResult run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline,
                                  JobInfo jobInfo)
                           throws java.lang.Exception

Does not actually run the pipeline. Instead bundles the input pipeline along with all dependencies, artifacts, etc. required to run the pipeline into a jar that can be executed later.

Specified by:

run in interface PortablePipelineRunner

Throws:

java.lang.Exception

copyResourcesFromJar

protected void copyResourcesFromJar(java.util.jar.JarFile inputJar)
                             throws java.io.IOException

Copy resources from inputJar to outputStream.

Throws:

java.io.IOException

writeArtifacts

protected org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline writeArtifacts(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline,
                                                                              java.lang.String jobName)
                                                                       throws java.io.IOException

Stages all dependencies in pipeline into the jar file at outputStream, returning a new pipeline that references these artifacts as classpath artifacts.

Throws:

java.io.IOException