apache_beam.runners.interactive.caching.reify module
Module for transforms that reifies and unreifies PCollection values with window info.
For internal use only; no backwards-compatibility guarantees.
- class apache_beam.runners.interactive.caching.reify.Reify(*unused_args, **unused_kwargs)[source]
Bases:
DoFn
Reifies elements with window info into windowed values.
Internally used to capture window info with each element into cache for replayability.
- class apache_beam.runners.interactive.caching.reify.Unreify(*unused_args, **unused_kwargs)[source]
Bases:
DoFn
Unreifies elements from windowed values.
Cached values are elements with window info. This unpacks the elements.
- apache_beam.runners.interactive.caching.reify.reify_to_cache(pcoll: PCollection, cache_key: str, cache_manager: CacheManager, reify_label: str | None = None, write_cache_label: str | None = None, is_capture: bool = False) PValue [source]
Reifies elements into windowed values and write to cache.
- Parameters:
pcoll – The PCollection to be cached.
cache_key – The key of the cache.
cache_manager – The cache manager to manage the cache.
reify_label – (optional) A transform label for the Reify transform.
write_cache_label – (optional) A transform label for the cache-writing transform.
is_capture – Whether the cache is capturing a record of recordable sources.
- apache_beam.runners.interactive.caching.reify.unreify_from_cache(pipeline: Pipeline, cache_key: str, cache_manager: CacheManager, element_type: type | None = None, source_label: str | None = None, unreify_label: str | None = None) PCollection [source]
Reads from cache and unreifies elements from windowed values.
pipeline: The pipeline that’s reading from the cache. cache_key: The key of the cache. cache_manager: The cache manager to manage the cache. element_type: (optional) The element type of the PCollection’s elements. source_label: (optional) A transform label for the cache-reading transform. unreify_label: (optional) A transform label for the Unreify transform.