@Experimental(value=WITH_EXCEPTIONS) public class WithFailures extends java.lang.Object
Consuming transforms such as
MapElements.MapWithFailures follow the general pattern of
taking in a user-defined exception handler of type
ProcessFunction<ExceptionElement<InputT>, FailureOutputT> where the input
WithFailures.ExceptionElement contains an exception along with the input element that was being processed
when the exception was raised. This handler is responsible for producing some output element that
captures relevant details of the failure and can be encoded as part of a failure output
PCollection. Transforms can then package together their output and failure collections in a
WithFailures.Result that avoids users needing to interact with
indexing into a
Exception handlers can narrow their scope by rethrowing the passed
WithFailures.ExceptionElement.exception() and catching only specific subclasses of
Unhandled exceptions will generally bubble up to a top-level
Pipeline.PipelineExecutionException that halts progress.
Users can take advantage of
WithFailures.Result.failuresTo(List) for fluent chaining of transforms
that handle exceptions:
PCollection<Integer> input = ... List<PCollection<Map<String, String>> failureCollections = new ArrayList<>(); input.apply(MapElements.via(...).exceptionsVia(...)) .failuresTo(failureCollections) .apply(MapElements.via(...).exceptionsVia(...)) .failuresTo(failureCollections); PCollection<Map<String, String>> failures = PCollectionList.of(failureCollections) .apply("FlattenFailureCollections", Flatten.pCollections());
|Modifier and Type||Class and Description|
A simple handler that extracts information from an exception to a
The value type passed as input to exception handlers.
An intermediate output type for PTransforms that allows an output collection to live alongside a collection of elements that failed the transform.
|Constructor and Description|