apache_beam.runners.interactive.caching.reify module¶
Module for transforms that reifies and unreifies PCollection values with window info.
For internal use only; no backwards-compatibility guarantees.
-
class
apache_beam.runners.interactive.caching.reify.
Reify
(*unused_args, **unused_kwargs)[source]¶ Bases:
apache_beam.transforms.core.DoFn
Reifies elements with window info into windowed values.
Internally used to capture window info with each element into cache for replayability.
-
class
apache_beam.runners.interactive.caching.reify.
Unreify
(*unused_args, **unused_kwargs)[source]¶ Bases:
apache_beam.transforms.core.DoFn
Unreifies elements from windowed values.
Cached values are elements with window info. This unpacks the elements.
-
apache_beam.runners.interactive.caching.reify.
reify_to_cache
(pcoll: apache_beam.pvalue.PCollection, cache_key: str, cache_manager: apache_beam.runners.interactive.cache_manager.CacheManager, reify_label: Optional[str] = None, write_cache_label: Optional[str] = None, is_capture: bool = False) → apache_beam.pvalue.PValue[source]¶ Reifies elements into windowed values and write to cache.
Parameters: - pcoll – The PCollection to be cached.
- cache_key – The key of the cache.
- cache_manager – The cache manager to manage the cache.
- reify_label – (optional) A transform label for the Reify transform.
- write_cache_label – (optional) A transform label for the cache-writing transform.
- is_capture – Whether the cache is capturing a record of recordable sources.
-
apache_beam.runners.interactive.caching.reify.
unreify_from_cache
(pipeline: apache_beam.pipeline.Pipeline, cache_key: str, cache_manager: apache_beam.runners.interactive.cache_manager.CacheManager, element_type: Optional[type] = None, source_label: Optional[str] = None, unreify_label: Optional[str] = None) → apache_beam.pvalue.PCollection[source]¶ Reads from cache and unreifies elements from windowed values.
pipeline: The pipeline that’s reading from the cache. cache_key: The key of the cache. cache_manager: The cache manager to manage the cache. element_type: (optional) The element type of the PCollection’s elements. source_label: (optional) A transform label for the cache-reading transform. unreify_label: (optional) A transform label for the Unreify transform.