org.apache.beam.sdk.extensions.sql.impl.rel

Class BeamJoinRel

java.lang.Object

org.apache.calcite.rel.AbstractRelNode

org.apache.calcite.rel.BiRel

org.apache.calcite.rel.core.Join

org.apache.beam.sdk.extensions.sql.impl.rel.BeamJoinRel

All Implemented Interfaces:

java.lang.Cloneable, BeamRelNode, org.apache.calcite.plan.RelOptNode, org.apache.calcite.rel.RelNode

Direct Known Subclasses:

BeamCoGBKJoinRel, BeamSideInputJoinRel, BeamSideInputLookupJoinRel

public abstract class BeamJoinRel
extends org.apache.calcite.rel.core.Join
implements BeamRelNode

An abstract BeamRelNode to implement Join Rels.

Support for join can be categorized into 4 cases:

• BoundedTable JOIN BoundedTable
• UnboundedTable JOIN UnboundedTable
• BoundedTable JOIN UnboundedTable
• SeekableTable JOIN non SeekableTable

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	BeamJoinRel.ExtractJoinKeys

Nested classes/interfaces inherited from interface org.apache.calcite.rel.RelNode

org.apache.calcite.rel.RelNode.Context

Field Summary

Fields inherited from class org.apache.calcite.rel.core.Join

condition, joinInfo, joinType, variablesSet

Fields inherited from class org.apache.calcite.rel.BiRel

left, right

Fields inherited from class org.apache.calcite.rel.AbstractRelNode

digest, id, rowType, traitSet

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BeamJoinRel(org.apache.calcite.plan.RelOptCluster cluster, org.apache.calcite.plan.RelTraitSet traits, org.apache.calcite.rel.RelNode left, org.apache.calcite.rel.RelNode right, org.apache.calcite.rex.RexNode condition, java.util.Set<org.apache.calcite.rel.core.CorrelationId> variablesSet, org.apache.calcite.rel.core.JoinRelType joinType)

Method Summary

Modifier and Type	Method and Description
BeamCostModel	beamComputeSelfCost(org.apache.calcite.plan.RelOptPlanner planner, org.apache.calcite.rel.metadata.RelMetadataQuery mq) This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl.
protected Schema	buildNullSchema(Schema schema)
static boolean	containsSeekableInput(org.apache.calcite.rel.RelNode relNode) This method returns whether any of the children of the relNode are Seekable.
NodeStats	estimateNodeStats(org.apache.calcite.rel.metadata.RelMetadataQuery mq) This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats.
static PCollection.IsBounded	getBoundednessOfRelNode(org.apache.calcite.rel.RelNode relNode) This method returns the Boundedness of a RelNode.
java.util.List<org.apache.calcite.rel.RelNode>	getPCollectionInputs()
static boolean	isJoinLegal(org.apache.calcite.rel.core.Join join) This method checks if a join is legal and can be converted into Beam SQL.
protected boolean	isSideInputLookupJoin()
protected org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Optional<java.lang.Integer>	nonSeekableInputIndex()
static boolean	seekable(BeamRelNode relNode) check if BeamRelNode implements BeamSeekableTable.
protected org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Optional<java.lang.Integer>	seekableInputIndex()
protected static <K,V> PCollection<KV<K,V>>	setValueCoder(PCollection<KV<K,V>> kvs, Coder<V> valueCoder)

Methods inherited from class org.apache.calcite.rel.core.Join

accept, analyzeCondition, computeSelfCost, copy, copy, createJoinType, deriveJoinRowType, deriveRowType, estimateJoinedRows, estimateRowCount, explainTerms, getChildExps, getCondition, getJoinType, getSystemFieldList, getVariablesSet, isSemiJoin, isSemiJoinDone, isValid

Methods inherited from class org.apache.calcite.rel.BiRel

childrenAccept, getInputs, getLeft, getRight, replaceInput

Methods inherited from class org.apache.calcite.rel.AbstractRelNode

accept, collectVariablesSet, collectVariablesUsed, computeDigest, computeSelfCost, explain, getCluster, getCollationList, getConvention, getCorrelVariable, getDescription, getDigest, getExpectedInputRowType, getId, getInput, getQuery, getRelTypeName, getRows, getRowType, getTable, getTraitSet, getVariablesStopped, isDistinct, isKey, isValid, metadata, onRegister, recomputeDigest, register, sole, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.extensions.sql.impl.rel.BeamRelNode

buildPTransform, getPipelineOptions, isBounded

Methods inherited from interface org.apache.calcite.rel.RelNode

accept, accept, childrenAccept, collectVariablesSet, collectVariablesUsed, computeSelfCost, computeSelfCost, copy, estimateRowCount, explain, getChildExps, getCollationList, getConvention, getCorrelVariable, getExpectedInputRowType, getInput, getInputs, getQuery, getRelTypeName, getRows, getRowType, getTable, getVariablesSet, getVariablesStopped, isDistinct, isKey, isValid, isValid, metadata, onRegister, recomputeDigest, register, replaceInput

Methods inherited from interface org.apache.calcite.plan.RelOptNode

getCluster, getDescription, getDigest, getId, getTraitSet

Constructor Detail

BeamJoinRel

protected BeamJoinRel(org.apache.calcite.plan.RelOptCluster cluster,
                      org.apache.calcite.plan.RelTraitSet traits,
                      org.apache.calcite.rel.RelNode left,
                      org.apache.calcite.rel.RelNode right,
                      org.apache.calcite.rex.RexNode condition,
                      java.util.Set<org.apache.calcite.rel.core.CorrelationId> variablesSet,
                      org.apache.calcite.rel.core.JoinRelType joinType)

Method Detail

getPCollectionInputs

public java.util.List<org.apache.calcite.rel.RelNode> getPCollectionInputs()

Specified by:

getPCollectionInputs in interface BeamRelNode

isSideInputLookupJoin

protected boolean isSideInputLookupJoin()

seekableInputIndex

protected org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Optional<java.lang.Integer> seekableInputIndex()

nonSeekableInputIndex

protected org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Optional<java.lang.Integer> nonSeekableInputIndex()

seekable

public static boolean seekable(BeamRelNode relNode)

check if BeamRelNode implements BeamSeekableTable.

beamComputeSelfCost

public BeamCostModel beamComputeSelfCost(org.apache.calcite.plan.RelOptPlanner planner,
                                         org.apache.calcite.rel.metadata.RelMetadataQuery mq)

Description copied from interface: BeamRelNode

This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl. This is currently only used in SQLTransform Path (and not JDBC path). This is needed when Calcite Query Planner wants to get the cost of a plan. Instead of calling this directly for a node, if we needed that it should be obtained by calling mq.getNonCumulativeCost. This way RelMetadataQuery will call this method instead of ComputeSelfCost if the handler is set correctly (see org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner#convertToBeamRel(String))

Specified by:

beamComputeSelfCost in interface BeamRelNode

estimateNodeStats

public NodeStats estimateNodeStats(org.apache.calcite.rel.metadata.RelMetadataQuery mq)

Description copied from interface: BeamRelNode

This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats. This is currently only used in SQLTransform Path (and not JDBC path). When a RelNode wants to calculate its BeamCost or estimate its NodeStats, it may need NodeStat of its inputs. However, it should not call this directly (because maybe its inputs are not physical yet). It should call BeamSqlRelUtils.getNodeStats(org.apache.calcite.rel.RelNode, org.apache.calcite.rel.metadata.RelMetadataQuery) instead.

Specified by:

estimateNodeStats in interface BeamRelNode

isJoinLegal

public static boolean isJoinLegal(org.apache.calcite.rel.core.Join join)

This method checks if a join is legal and can be converted into Beam SQL. It is used during planning and applying BeamJoinAssociateRule and BeamJoinPushThroughJoinRule

buildNullSchema

protected Schema buildNullSchema(Schema schema)

setValueCoder

protected static <K,V> PCollection<KV<K,V>> setValueCoder(PCollection<KV<K,V>> kvs,
                                                          Coder<V> valueCoder)

getBoundednessOfRelNode

public static PCollection.IsBounded getBoundednessOfRelNode(org.apache.calcite.rel.RelNode relNode)

This method returns the Boundedness of a RelNode. It is used during planning and applying BeamCoGBKJoinRule and BeamSideInputJoinRule

The Volcano planner works in a top-down fashion. It starts by transforming the root and move towards the leafs of the plan. Due to this when transforming a logical join its inputs are still in the logical convention. So, Recursively visit the inputs of the RelNode till BeamIOSourceRel is encountered and propagate the boundedness upwards.

The Boundedness of each child of a RelNode is stored in a list. If any of the children are Unbounded, the RelNode is Unbounded. Else, the RelNode is Bounded.

Parameters:

relNode - the RelNode whose Boundedness has to be determined

Returns:

PCollection.isBounded

containsSeekableInput

public static boolean containsSeekableInput(org.apache.calcite.rel.RelNode relNode)

This method returns whether any of the children of the relNode are Seekable. It is used during planning and applying BeamCoGBKJoinRule and BeamSideInputJoinRule and BeamSideInputLookupJoinRule

Parameters:

relNode - the relNode whose children can be Seekable

Returns:

A boolean