Interface DoFn<InputT, OutputT, ContextT>

The interface used to apply an elementwise MappingFn to a PCollection.

For simple transformations, PCollection.map or PCollection.flatMap may be simpler to use.

See also https://beam.apache.org/documentation/programming-guide/#pardo

Type Parameters

  • InputT

  • OutputT

  • ContextT = undefined

Hierarchy

  • DoFn

Properties

beamName? finishBundle? process startBundle?

Properties

Optional beamName

beamName?: string

If provided, the default name to use for this operation.

Optional finishBundle

finishBundle?: ((context: ContextT) => void | Iterable<WindowedValue<OutputT>>)

Type declaration

    • (context: ContextT): void | Iterable<WindowedValue<OutputT>>

    • Called once at the end of every bundle, after any process() calls.

      This can be used to clean up expensive initialization and/or flush any elements that were buffered.

      Parameters

      • context: ContextT

      Returns void | Iterable<WindowedValue<OutputT>>

process

process: ((element: InputT, context: ContextT) => void | Iterable<OutputT>)

Type declaration

    • (element: InputT, context: ContextT): void | Iterable<OutputT>

    • Process a single element from the PCollection, returning an iterable of zero or more result elements.

      Also takes as input an optional context element which has the same type as was passed into the parDo at construction time (but which is now "activated" in the sense that side inputs, metrics, etc. are available with runtime values/effects).

      Parameters

      • element: InputT
      • context: ContextT

      Returns void | Iterable<OutputT>

Optional startBundle

startBundle?: ((context: ContextT) => void)

Type declaration

    • (context: ContextT): void

    • Called once at the start of every bundle, before any process() calls.

      This can be used to amortize any expensive initialization.

      Parameters

      • context: ContextT

      Returns void

Settings

Member Visibility

Theme

Generated using TypeDoc