org.apache.beam.runners.flink

Class FlinkRunner

java.lang.Object

org.apache.beam.sdk.PipelineRunner<PipelineResult>

org.apache.beam.runners.flink.FlinkRunner

public class FlinkRunner
extends PipelineRunner<PipelineResult>

A PipelineRunner that executes the operations in the pipeline by first translating them to a Flink Plan and then executing them either locally or on a Flink cluster, depending on the configuration.

Method Summary

Modifier and Type	Method and Description
protected static java.util.List<java.lang.String>	detectClassPathResourcesToStage(java.lang.ClassLoader classLoader) Attempts to detect all the resources the class loader has access to.
static FlinkRunner	fromOptions(PipelineOptions options) Construct a runner from the provided options.
FlinkPipelineOptions	getPipelineOptions() For testing.
PipelineResult	run(Pipeline pipeline) Processes the given Pipeline, potentially asynchronously, returning a runner-specific type of result.
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

fromOptions

public static FlinkRunner fromOptions(PipelineOptions options)

Construct a runner from the provided options.

Parameters:

options - Properties which configure the runner.

Returns:

The newly created runner.

run

public PipelineResult run(Pipeline pipeline)

Description copied from class: PipelineRunner

Processes the given Pipeline, potentially asynchronously, returning a runner-specific type of result.

Specified by:

run in class PipelineRunner<PipelineResult>

getPipelineOptions

public FlinkPipelineOptions getPipelineOptions()

For testing.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

detectClassPathResourcesToStage

protected static java.util.List<java.lang.String> detectClassPathResourcesToStage(java.lang.ClassLoader classLoader)

Attempts to detect all the resources the class loader has access to. This does not recurse to class loader parents stopping it from pulling in resources from the system class loader.

Parameters:

classLoader - The URLClassLoader to use to detect resources to stage.

Returns:

A list of absolute paths to the resources the class loader uses.

Throws:

java.lang.IllegalArgumentException - If either the class loader is not a URLClassLoader or one of the resources the class loader exposes is not a file resource.