Package org.apache.beam.sdk.extensions.python.transforms

Class PythonMap<InputT,OutputT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<? extends InputT>,PCollection<OutputT>>

org.apache.beam.sdk.extensions.python.transforms.PythonMap<InputT,OutputT>

All Implemented Interfaces:

Serializable, HasDisplayData

public class PythonMap<InputT,OutputT>
extends PTransform<PCollection<? extends InputT>,PCollection<OutputT>>

Wrapper for invoking external Python Map transforms..

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method	Description
PCollection<OutputT>	expand(PCollection<? extends InputT> input)	Override this method to specify how this PTransform should be expanded on the given InputT.
static <InputT, OutputT> PythonMap<InputT,OutputT>	viaFlatMapFn(String pythonFunction, Coder<?> outputCoder)
static <InputT, OutputT> PythonMap<InputT,OutputT>	viaMapFn(String pythonFunction, Coder<?> outputCoder)
PythonMap<InputT,OutputT>	withExpansionService(String expansionService)
PythonMap<InputT,OutputT>	withExtraPackages(List<String> extraPackages)	Specifies any extra packages required by the Python function.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

viaMapFn

public static <InputT,
OutputT>
PythonMap<InputT,OutputT> viaMapFn(String pythonFunction,
 Coder<?> outputCoder)

viaFlatMapFn

public static <InputT,
OutputT>
PythonMap<InputT,OutputT> viaFlatMapFn(String pythonFunction,
 Coder<?> outputCoder)

withExpansionService

public PythonMap<InputT,OutputT> withExpansionService(String expansionService)

withExtraPackages

public PythonMap<InputT,OutputT> withExtraPackages(List<String> extraPackages)

Specifies any extra packages required by the Python function.

This should only be specified when using the default expansion service, i.e. when not using withExpansionService(String) to provide an expansion service.

The package can either be a PyPi package or the path to a locally available Python package.

Parameters:

extraPackages - a list of PyPi packages. May include the version.

Returns:

A PythonMap with extra packages.

expand

public PCollection<OutputT> expand(PCollection<? extends InputT> input)

Description copied from class: PTransform

Override this method to specify how this PTransform should be expanded on the given InputT.

NOTE: This method should not be called directly. Instead apply the PTransform should be applied to the InputT using the apply method.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

Specified by:

expand in class PTransform<PCollection<? extends InputT>,PCollection<OutputT>>