org.apache.beam.sdk.metrics

Class Lineage

java.lang.Object

org.apache.beam.sdk.metrics.Lineage

public class Lineage
extends java.lang.Object

Standard collection of metrics used to record source and sinks information for lineage tracking.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Lineage.Type Lineage metrics resource types.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	LINEAGE_NAMESPACE

Method Summary

Modifier and Type	Method and Description
void	add(java.lang.String details) Adds the given details as Lineage.
void	add(java.lang.String system, java.lang.Iterable<java.lang.String> segments) Add a FQN (fully-qualified name) to Lineage.
void	add(java.lang.String system, @Nullable java.lang.String subtype, java.lang.Iterable<java.lang.String> segments) Add a FQN (fully-qualified name) to Lineage.
static java.lang.String	getFqName(java.lang.String system, java.lang.Iterable<java.lang.String> segments) Assemble the FQN of given system, and segments.
static java.lang.String	getFqName(java.lang.String system, @Nullable java.lang.String subtype, java.lang.Iterable<java.lang.String> segments) Assemble fully qualified name (FQN).
static Lineage	getSinks() Lineage representing sinks.
static Lineage	getSources() Lineage representing sources and optionally side inputs.
static java.util.Set<java.lang.String>	query(MetricResults results, Lineage.Type type) Query StringSet metrics from MetricResults.
static java.lang.String	wrapSegment(java.lang.String value) Wrap segment to valid segment name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LINEAGE_NAMESPACE

public static final java.lang.String LINEAGE_NAMESPACE

See Also:

Constant Field Values

Method Detail

getSources

public static Lineage getSources()

Lineage representing sources and optionally side inputs.

getSinks

public static Lineage getSinks()

Lineage representing sinks.

wrapSegment

@Internal
public static java.lang.String wrapSegment(java.lang.String value)

Wrap segment to valid segment name.

Specifically, If there are reserved chars (colon, whitespace, dot), escape with backtick. If the segment is already wrapped, return the original.

This helper method is for internal and testing usage only.

getFqName

@Internal
public static java.lang.String getFqName(java.lang.String system,
                                                   @Nullable java.lang.String subtype,
                                                   java.lang.Iterable<java.lang.String> segments)

Assemble fully qualified name (FQN). Format:

• system:segment1.segment2
• system:subtype:segment1.segment2
• system:`segment1.with.dots:clons`.segment2

This helper method is for internal and testing usage only.

getFqName

@Internal
public static java.lang.String getFqName(java.lang.String system,
                                                   java.lang.Iterable<java.lang.String> segments)

Assemble the FQN of given system, and segments.

This helper method is for internal and testing usage only.

add

public void add(java.lang.String system,
                @Nullable java.lang.String subtype,
                java.lang.Iterable<java.lang.String> segments)

Add a FQN (fully-qualified name) to Lineage. Segments will be processed via getFqName(java.lang.String, java.lang.String, java.lang.Iterable<java.lang.String>).

add

public void add(java.lang.String system,
                java.lang.Iterable<java.lang.String> segments)

Add a FQN (fully-qualified name) to Lineage. Segments will be processed via getFqName(java.lang.String, java.lang.String, java.lang.Iterable<java.lang.String>).

add

public void add(java.lang.String details)

Adds the given details as Lineage. For asset level lineage the resource location should be specified as Dataplex FQN https://cloud.google.com/data-catalog/docs/fully-qualified-names

query

public static java.util.Set<java.lang.String> query(MetricResults results,
                                                    Lineage.Type type)

Query StringSet metrics from MetricResults.