org.apache.beam.sdk.extensions.sql.impl.rel

Class BeamSortRel

java.lang.Object

org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.AbstractRelNode

org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.SingleRel

org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort

org.apache.beam.sdk.extensions.sql.impl.rel.BeamSortRel

All Implemented Interfaces:

java.lang.Cloneable, BeamRelNode, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptNode, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode

public class BeamSortRel
extends org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort
implements BeamRelNode

BeamRelNode to replace a Sort node.

Since Beam does not fully support global sort, it uses Top to implement the Sort algebra. The following types of ORDER BY are supported:

 SELECT * FROM t ORDER BY id DESC LIMIT 10;
 SELECT * FROM t ORDER BY id DESC LIMIT 10 OFFSET 5;
 

but an ORDER BY without a LIMIT is NOT supported. For example, the following will throw an exception:

 SELECT * FROM t ORDER BY id DESC;
 

Constraints

• Due to the constraints of Top, the result of a ORDER BY LIMIT must fit into the memory of a single machine.
• Since WINDOW (HOP, TUMBLE, SESSION, etc.) is always associated with `GroupBy`, it does not make much sense to use ORDER BY with WINDOW.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BeamSortRel.BeamSqlRowComparator

Nested classes/interfaces inherited from interface org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode

org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode.Context

Field Summary

Fields inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort

collation, fetch, offset

Fields inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.SingleRel

input

Fields inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.AbstractRelNode

digest, id, rowType, traitSet

Constructor Summary

Constructors

Constructor and Description

BeamSortRel(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptCluster cluster, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelTraitSet traits, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode child, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelCollation collation, @Nullable org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode offset, @Nullable org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode fetch)

Method Summary

Modifier and Type	Method and Description
BeamCostModel	beamComputeSelfCost(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptPlanner planner, BeamRelMetadataQuery mq) This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl.
PTransform<PCollectionList<Row>,PCollection<Row>>	buildPTransform()
org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort	copy(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelTraitSet traitSet, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode newInput, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelCollation newCollation, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode offset, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode fetch)
NodeStats	estimateNodeStats(BeamRelMetadataQuery mq) This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats.
int	getCount()
boolean	isLimitOnly()

Methods inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort

accept, computeSelfCost, copy, copy, explainTerms, getCollation, getSortExps, isEnforcer

Methods inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.SingleRel

childrenAccept, deriveRowType, estimateRowCount, getInput, getInputs, replaceInput

Methods inherited from class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.AbstractRelNode

accept, collectVariablesSet, collectVariablesUsed, deepEquals, deepHashCode, equals, explain, getCluster, getConvention, getCorrelVariable, getDescription, getDigest, getExpectedInputRowType, getId, getInput, getRelDigest, getRelTypeName, getRowType, getTable, getTraitSet, getVariablesSet, hashCode, isValid, metadata, onRegister, recomputeDigest, register, sole, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.extensions.sql.impl.rel.BeamRelNode

buildPTransform, getPCollectionInputs, getPipelineOptions, isBounded, withErrorsTransformer

Methods inherited from interface org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode

accept, accept, childrenAccept, collectVariablesSet, collectVariablesUsed, computeSelfCost, copy, deepEquals, deepHashCode, estimateRowCount, explain, explain, fieldIsNullable, getConvention, getCorrelVariable, getDigest, getExpectedInputRowType, getInput, getInputs, getRelDigest, getRelTypeName, getRowType, getTable, getVariablesSet, isEnforcer, isValid, metadata, onRegister, recomputeDigest, register, replaceInput

Methods inherited from interface org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptNode

getCluster, getDescription, getId, getTraitSet

Constructor Detail

BeamSortRel

public BeamSortRel(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptCluster cluster,
                   org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelTraitSet traits,
                   org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode child,
                   org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelCollation collation,
                   @Nullable org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode offset,
                   @Nullable org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode fetch)

Method Detail

estimateNodeStats

public NodeStats estimateNodeStats(BeamRelMetadataQuery mq)

Description copied from interface: BeamRelNode

This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats. This is currently only used in SQLTransform Path (and not JDBC path). When a RelNode wants to calculate its BeamCost or estimate its NodeStats, it may need NodeStat of its inputs. However, it should not call this directly (because maybe its inputs are not physical yet). It should call org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils#getNodeStats( org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode, org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.metadata.RelMetadataQuery) instead.

Specified by:

estimateNodeStats in interface BeamRelNode

beamComputeSelfCost

public BeamCostModel beamComputeSelfCost(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelOptPlanner planner,
                                         BeamRelMetadataQuery mq)

Description copied from interface: BeamRelNode

This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl. This is currently only used in SQLTransform Path (and not JDBC path). This is needed when Calcite Query Planner wants to get the cost of a plan. Instead of calling this directly for a node, if we needed that it should be obtained by calling mq.getNonCumulativeCost. This way RelMetadataQuery will call this method instead of ComputeSelfCost if the handler is set correctly (see org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner#convertToBeamRel(String))

Specified by:

beamComputeSelfCost in interface BeamRelNode

isLimitOnly

public boolean isLimitOnly()

getCount

public int getCount()

buildPTransform

public PTransform<PCollectionList<Row>,PCollection<Row>> buildPTransform()

Specified by:

buildPTransform in interface BeamRelNode

copy

public org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort copy(org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.plan.RelTraitSet traitSet,
                                                                                    org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelNode newInput,
                                                                                    org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.RelCollation newCollation,
                                                                                    org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode offset,
                                                                                    org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rex.RexNode fetch)

Specified by:

copy in class org.apache.beam.vendor.calcite.v1_28_0.org.apache.calcite.rel.core.Sort