Package org.apache.beam.runners.flink

Class FlinkPortableClientEntryPoint

java.lang.Object
org.apache.beam.runners.flink.FlinkPortableClientEntryPoint
public class FlinkPortableClientEntryPoint extends Object
Flink job entry point to launch a Beam pipeline by executing an external SDK driver program.

Designed for non-interactive Flink REST client and container with Beam job server jar and SDK client (for example when using the FlinkK8sOperator). In the future it would be possible to support driver program execution in a separate (sidecar) container by introducing a client environment abstraction similar to how it exists for SDK workers.

Using this entry point eliminates the need to build jar files with materialized pipeline protos offline. Allows the driver program to access actual execution environment and services, on par with code executed by SDK workers.

The entry point starts the job server and provides the endpoint to the driver program.

The external driver program constructs the Beam pipeline and submits it to the job service.

The job service defers execution of the pipeline to the plan environment and returns the "detached" status to the driver program.

Upon arrival of the job invocation, the entry point executes the runner, which prepares ("executes") the Flink job through the plan environment.

Finally Flink launches the job.

  • Constructor Details

    • FlinkPortableClientEntryPoint

      public FlinkPortableClientEntryPoint(String driverCmd)

  • Method Details

    • main

      public static void main(String[] args) throws Exception
      Main method to be called standalone or by Flink (CLI or REST API).
      Throws:
      Exception