org.apache.beam.sdk.transforms.errorhandling

Interface ErrorHandler<ErrorT,OutputT extends POutput>

Type Parameters:

ErrorT - The type of the error object. This will usually be a BadRecord, but can be any type

OutputT - The return type of the sink PTransform.

Usage of Error Handlers:

Simple usage with one DLQ

 PCollection<?> records = ...;
 try (BadRecordErrorHandler<T> errorHandler = pipeline.registerBadRecordErrorHandler(SomeSink.write())) {
  PCollection<?> results = records.apply(SomeIO.write().withErrorHandler(errorHandler));
 }
 results.apply(SomeOtherTransform);
 

Usage with multiple DLQ stages

 PCollection<?> records = ...;
 try (BadRecordErrorHandler<T> errorHandler = pipeline.registerBadRecordErrorHandler(SomeSink.write())) {
  PCollection<?> results = records.apply(SomeIO.write().withErrorHandler(errorHandler))
                        .apply(OtherTransform.builder().withErrorHandler(errorHandler));
 }
 results.apply(SomeOtherTransform);
 

This is marked as serializable despite never being needed on the runner, to enable it to be a parameter of an Autovalue configured PTransform.

All Superinterfaces:

java.lang.AutoCloseable, java.io.Serializable

All Known Implementing Classes:

ErrorHandler.BadRecordErrorHandler, ErrorHandler.DefaultErrorHandler, ErrorHandler.PTransformErrorHandler

public interface ErrorHandler<ErrorT,OutputT extends POutput>
extends java.lang.AutoCloseable, java.io.Serializable

An Error Handler is a utility object used for plumbing error PCollections to a configured sink Error Handlers must be closed before a pipeline is run to properly pipe error collections to the sink, and the pipeline will be rejected if any handlers aren't closed.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	ErrorHandler.BadRecordErrorHandler<OutputT extends POutput>
static class	ErrorHandler.DefaultErrorHandler<ErrorT,OutputT extends POutput> A default, placeholder error handler that exists to allow usage of .addErrorCollection() without effects.
static class	ErrorHandler.PTransformErrorHandler<ErrorT,OutputT extends POutput>

Method Summary

Modifier and Type	Method and Description
void	addErrorCollection(PCollection<ErrorT> errorCollection)
OutputT	getOutput()
boolean	isClosed()

Methods inherited from interface java.lang.AutoCloseable

close

Method Detail

addErrorCollection

void addErrorCollection(PCollection<ErrorT> errorCollection)

isClosed

boolean isClosed()

getOutput

OutputT getOutput()