Package org.apache.beam.runners.jobsubmission

Class PortablePipelineJarCreator

java.lang.Object

org.apache.beam.runners.jobsubmission.PortablePipelineJarCreator

All Implemented Interfaces:

PortablePipelineRunner

public class PortablePipelineJarCreator
extends Object
implements PortablePipelineRunner

PortablePipelineRunner that bundles the input pipeline along with all dependencies, artifacts, etc. required to run the pipeline into a jar that can be executed later.

Each PortablePipelineJarCreator instance is not threadsafe; a new instance is expected to be constructed and run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline, org.apache.beam.runners.fnexecution.provisioning.JobInfo) once per job.

Constructor Summary

Constructors

Constructor	Description
PortablePipelineJarCreator(Class mainClass)

Method Summary

Modifier and Type	Method	Description
protected void	copyResourcesFromJar(JarFile inputJar)	Copy resources from inputJar to outputStream.
PortablePipelineResult	run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline, JobInfo jobInfo)	Does not actually run the pipeline. Instead bundles the input pipeline along with all dependencies, artifacts, etc.
protected org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline	writeArtifacts(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline, String jobName)	Stages all dependencies in pipeline into the jar file at outputStream, returning a new pipeline that references these artifacts as classpath artifacts.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PortablePipelineJarCreator

public PortablePipelineJarCreator(Class mainClass)

Method Details

run

public PortablePipelineResult run(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline,
 JobInfo jobInfo)
                           throws Exception

Does not actually run the pipeline. Instead bundles the input pipeline along with all dependencies, artifacts, etc. required to run the pipeline into a jar that can be executed later.

Specified by:

run in interface PortablePipelineRunner

Throws:

Exception

copyResourcesFromJar

protected void copyResourcesFromJar(JarFile inputJar)
                             throws IOException

Copy resources from inputJar to outputStream.

Throws:

IOException

writeArtifacts

protected org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline writeArtifacts(org.apache.beam.model.pipeline.v1.RunnerApi.Pipeline pipeline,
 String jobName)
                                                                       throws IOException

Stages all dependencies in pipeline into the jar file at outputStream, returning a new pipeline that references these artifacts as classpath artifacts.

Throws:

IOException