Annotation Interface DoFn.Teardown
A runner will do its best to call this method on any given instance to prevent leaks of
transient resources, however, there may be situations where this is impossible (e.g. process
crash, hardware failure, etc.) or unnecessary (e.g. the pipeline is shutting down and the
process is about to be killed anyway, so all transient resources will be released automatically
by the OS). In these cases, the call may not happen. It will also not be retried, because in
such situations the DoFn instance no longer exists, so there's no instance to retry it on. In
portable execution(with --experiments=beam_fn_api
), the exception thrown calling DoFn.Teardown
will not fail the bundle execution. Instead, an error message will be shown on sdk
harness log.
Thus, all work that depends on input elements, and all externally important side effects,
must be performed in the DoFn.ProcessElement
or DoFn.FinishBundle
methods.
Example things that are a good idea to do in this method:
- Close a network connection that was opened in
DoFn.Setup
- Shut down a helper process that was started in
DoFn.Setup
Example things that MUST NOT be done in this method:
- Flushing a batch of buffered records to a database: this must be done in
DoFn.FinishBundle
. - Deleting temporary files on a distributed filesystem: this must be done using the
pipeline structure, e.g. using the
Wait
transform.
The method annotated with this must satisfy the following constraint:
- It must have zero arguments.