apache_beam.transforms.resources module¶
A module for defining resource requirements for execution of transforms.
Pipeline authors can use resource hints to provide additional information to runners about the desired aspects of the execution environment.
Resource hints can be specified on a transform level for parts of the pipeline, or globally via –resource_hint pipeline option.
See also: PTransforms.with_resource_hints().
-
class
apache_beam.transforms.resources.
ResourceHint
[source]¶ Bases:
object
A superclass to define resource hints.
-
urn
= None¶
-
classmethod
parse
(value)[source]¶ Describes how to parse the hint. Override to specify a custom parsing logic.
-
classmethod
get_merged_value
(outer_value, inner_value)[source]¶ Reconciles values of a hint when the hint specified on a transform is also defined in an outer context, for example on a composite transform, or specified in the transform’s execution environment. Override to specify a custom merging logic.
-
-
class
apache_beam.transforms.resources.
AcceleratorHint
[source]¶ Bases:
apache_beam.transforms.resources.ResourceHint
Describes desired hardware accelerators in execution environment.
-
urn
= 'beam:resources:accelerator:v1'¶
-
-
class
apache_beam.transforms.resources.
MinRamHint
[source]¶ Bases:
apache_beam.transforms.resources.ResourceHint
Describes min RAM requirements for transform’s execution environment.
-
urn
= 'beam:resources:min_ram_bytes:v1'¶
-
-
class
apache_beam.transforms.resources.
CpuCountHint
[source]¶ Bases:
apache_beam.transforms.resources.ResourceHint
Describes number of CPUs available in transform’s execution environment.
-
urn
= 'beam:resources:cpu_count:v1'¶
-