apache_beam.utils.subprocess_server module

class apache_beam.utils.subprocess_server.SubprocessServer(stub_class, cmd, port=None)[source]

Bases: object

An abstract base class for running GRPC Servers as an external process.

This class acts as a context which will start up a server, provides a stub to connect to it, and then shuts the server down. For example:

with SubprocessServer(GrpcStubClass, [executable, arg, ...]) as stub:
    stub.CallService(...)

Creates the server object.

Parameters:
  • stub_class – the auto-generated GRPC client stub class used for connecting to the GRPC service
  • cmd – command (including arguments) for starting up the server, suitable for passing to subprocess.POpen.
  • port – (optional) the port at which the subprocess will serve its service. If not given, one will be randomly chosen and the special string “{{PORT}}” will be substituted in the command line arguments with the chosen port.
start()[source]
start_process()[source]
stop()[source]
stop_process()[source]
local_temp_dir(**kwargs)[source]
class apache_beam.utils.subprocess_server.JavaJarServer(stub_class, path_to_jar, java_arguments, classpath=None)[source]

Bases: apache_beam.utils.subprocess_server.SubprocessServer

MAVEN_CENTRAL_REPOSITORY = 'https://repo.maven.apache.org/maven2'
BEAM_GROUP_ID = 'org.apache.beam'
JAR_CACHE = '/home/runner/.apache_beam/cache/jars'
start_process()[source]
stop_process()[source]
classmethod jar_name(artifact_id, version, classifier=None, appendix=None)[source]
classmethod path_to_maven_jar(artifact_id, group_id, version, repository='https://repo.maven.apache.org/maven2', classifier=None, appendix=None)[source]
classmethod path_to_beam_jar(gradle_target, appendix=None, version='2.51.0', artifact_id=None)[source]
classmethod local_jar(url, cache_dir=None)[source]
classmethod beam_services(replacements)[source]
classmethod make_classpath_jar(main_jar, extra_jars, cache_dir=None)[source]
apache_beam.utils.subprocess_server.is_service_endpoint(path)[source]

Checks whether the path conforms to the ‘beam_services’ PipelineOption.

apache_beam.utils.subprocess_server.pick_port(*ports)[source]

Returns a list of ports, same length as input ports list, but replaces all None or 0 ports with a random free port.