apache_beam.internal.pickler module

Pickler for values, functions, and classes.

For internal use only. No backwards compatibility guarantees.

Pickles created by the pickling library contain non-ASCII characters, so we base64-encode the results so that we can put them in a JSON objects. The pickler is used to embed FlatMap callable objects into the workflow JSON description.

The pickler module should be used to pickle functions and modules; for values, the coders.*PickleCoder classes should be used instead.

apache_beam.internal.pickler.save_module(pickler, obj)

apache_beam.internal.pickler.new_save_module_dict(pickler, obj)

apache_beam.internal.pickler.dumps(o, enable_trace=True, use_zlib=False)

For internal use only; no backwards-compatibility guarantees.

apache_beam.internal.pickler.loads(encoded, enable_trace=True, use_zlib=False)

For internal use only; no backwards-compatibility guarantees.

apache_beam.internal.pickler.dump_session(file_path)

For internal use only; no backwards-compatibility guarantees.

Pickle the current python session to be used in the worker.

Note: Due to the inconsistency in the first dump of dill dump_session we create and load the dump twice to have consistent results in the worker and the running session. Check: https://github.com/uqfoundation/dill/issues/195

apache_beam.internal.pickler.load_session(file_path)