apache_beam.utils.subprocess_server module

class apache_beam.utils.subprocess_server.SubprocessServer(stub_class, cmd, port=None)

Bases: object

An abstract base class for running GRPC Servers as an external process.

This class acts as a context which will start up a server, provides a stub to connect to it, and then shuts the server down. For example:

with SubprocessServer(GrpcStubClass, [executable, arg, ...]) as stub:
    stub.CallService(...)

Creates the server object.

Parameters:

• stub_class – the auto-generated GRPC client stub class used for connecting to the GRPC service
• cmd – command (including arguments) for starting up the server, suitable for passing to subprocess.POpen.
• port – (optional) the port at which the subprocess will serve its service. If not given, one will be randomly chosen and the special string “{{PORT}}” will be substituted in the command line arguments with the chosen port.

start()

start_process()

stop()

stop_process()

local_temp_dir(**kwargs)

class apache_beam.utils.subprocess_server.JavaJarServer(stub_class, path_to_jar, java_arguments, classpath=None)

Bases: apache_beam.utils.subprocess_server.SubprocessServer

MAVEN_CENTRAL_REPOSITORY = 'https://repo.maven.apache.org/maven2'

BEAM_GROUP_ID = 'org.apache.beam'

JAR_CACHE = '/usr/local/google/home/dannymccormick/.apache_beam/cache/jars'

start_process()

stop_process()

classmethod jar_name(artifact_id, version, classifier=None, appendix=None)

classmethod path_to_maven_jar(artifact_id, group_id, version, repository='https://repo.maven.apache.org/maven2', classifier=None, appendix=None)

classmethod path_to_beam_jar(gradle_target, appendix=None, version='2.46.0', artifact_id=None)

classmethod local_jar(url, cache_dir=None)

classmethod beam_services(replacements)

classmethod make_classpath_jar(main_jar, extra_jars, cache_dir=None)

apache_beam.utils.subprocess_server.pick_port(*ports)

Returns a list of ports, same length as input ports list, but replaces all None or 0 ports with a random free port.