Package org.apache.beam.sdk.transforms.errorhandling

Interface ErrorHandler<ErrorT,OutputT extends POutput>

Type Parameters:
ErrorT - The type of the error object. This will usually be a BadRecord, but can be any type
OutputT - The return type of the sink PTransform.

Usage of Error Handlers:

Simple usage with one DLQ 


 PCollection<?> records = ...;
 try (BadRecordErrorHandler<T> errorHandler = pipeline.registerBadRecordErrorHandler(SomeSink.write())) {
  PCollection<?> results = records.apply(SomeIO.write().withErrorHandler(errorHandler));
 }
 results.apply(SomeOtherTransform);
Usage with multiple DLQ stages 

 PCollection<?> records = ...;
 try (BadRecordErrorHandler<T> errorHandler = pipeline.registerBadRecordErrorHandler(SomeSink.write())) {
  PCollection<?> results = records.apply(SomeIO.write().withErrorHandler(errorHandler))
                        .apply(OtherTransform.builder().withErrorHandler(errorHandler));
 }
 results.apply(SomeOtherTransform);
This is marked as serializable despite never being needed on the runner, to enable it to be a parameter of an Autovalue configured PTransform.
All Superinterfaces:
AutoCloseable, Serializable
All Known Implementing Classes:
ErrorHandler.BadRecordErrorHandler, ErrorHandler.DefaultErrorHandler, ErrorHandler.PTransformErrorHandler
public interface ErrorHandler<ErrorT,OutputT extends POutput> extends AutoCloseable, Serializable
An Error Handler is a utility object used for plumbing error PCollections to a configured sink Error Handlers must be closed before a pipeline is run to properly pipe error collections to the sink, and the pipeline will be rejected if any handlers aren't closed.

  • Method Details