Package org.apache.beam.sdk.extensions.sql.impl.rel

Class BeamSortRel

java.lang.Object
org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.AbstractRelNode
org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.SingleRel
org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort
org.apache.beam.sdk.extensions.sql.impl.rel.BeamSortRel
All Implemented Interfaces:
Cloneable, BeamRelNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.hint.Hintable, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode
public class BeamSortRel extends org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort implements BeamRelNode
BeamRelNode to replace a Sort node.

Since Beam does not fully support global sort, it uses Top to implement the Sort algebra. The following types of ORDER BY are supported: 


 SELECT * FROM t ORDER BY id DESC LIMIT 10;
 SELECT * FROM t ORDER BY id DESC LIMIT 10 OFFSET 5;

but an ORDER BY without a LIMIT is NOT supported. For example, the following will throw an exception: 


 SELECT * FROM t ORDER BY id DESC;

Constraints

  • Due to the constraints of Top, the result of a ORDER BY LIMIT must fit into the memory of a single machine.
  • Since WINDOW (HOP, TUMBLE, SESSION, etc.) is always associated with `GroupBy`, it does not make much sense to use ORDER BY with WINDOW.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    BeamSortRel.BeamSqlRowComparator
     

    Nested classes/interfaces inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode

    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode.Context

  • Field Summary

    Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort

    collation, fetch, hints, offset

    Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.SingleRel

    input

    Fields inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.AbstractRelNode

    digest, id, rowType, traitSet

  • Constructor Summary

    Constructors
    Constructor
    Description
    BeamSortRel(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptCluster cluster, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traits, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode child, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation collation, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    BeamCostModel
    beamComputeSelfCost(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptPlanner planner, BeamRelMetadataQuery mq)
    This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl.
    PTransform<PCollectionList<Row>,PCollection<Row>>
    buildPTransform()
     
    org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort
    copy(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traitSet, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode newInput, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation newCollation, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)
     
    NodeStats
    estimateNodeStats(BeamRelMetadataQuery mq)
    This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats.
    int
    getCount()
     
    boolean
    isLimitOnly()
     

    Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort

    accept, computeSelfCost, copy, copy, explainTerms, getCollation, getHints, getSortExps, isEnforcer

    Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.SingleRel

    childrenAccept, deriveRowType, estimateRowCount, getInput, getInputs, replaceInput

    Methods inherited from class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.AbstractRelNode

    accept, collectVariablesSet, collectVariablesUsed, deepEquals, deepHashCode, equals, explain, getCluster, getConvention, getCorrelVariable, getDescription, getDigest, getExpectedInputRowType, getId, getInput, getRelDigest, getRelTypeName, getRowType, getTable, getTraitSet, getVariablesSet, hashCode, isValid, metadata, onRegister, recomputeDigest, register, sole, toString

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.beam.sdk.extensions.sql.impl.rel.BeamRelNode

    buildPTransform, getPCollectionInputs, getPipelineOptions, isBounded, withErrorsTransformer

    Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.hint.Hintable

    attachHints, withHints

    Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode

    accept, accept, childrenAccept, collectVariablesSet, collectVariablesUsed, computeSelfCost, copy, deepEquals, deepHashCode, estimateRowCount, explain, explain, fieldIsNullable, getConvention, getCorrelVariable, getDigest, getExpectedInputRowType, getInput, getInputs, getRelDigest, getRelTypeName, getRowType, getTable, getVariablesSet, isEnforcer, isValid, metadata, onRegister, recomputeDigest, register, replaceInput, stripped

    Methods inherited from interface org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptNode

    getCluster, getDescription, getId, getTraitSet

  • Constructor Details

    • BeamSortRel

      public BeamSortRel(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptCluster cluster, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traits, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode child, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation collation, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, @Nullable org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)

  • Method Details

    • estimateNodeStats

      public NodeStats estimateNodeStats(BeamRelMetadataQuery mq)
      Description copied from interface: BeamRelNode
      This method is called by org.apache.beam.sdk.extensions.sql.impl.planner.RelMdNodeStats. This is currently only used in SQLTransform Path (and not JDBC path). When a RelNode wants to calculate its BeamCost or estimate its NodeStats, it may need NodeStat of its inputs. However, it should not call this directly (because maybe its inputs are not physical yet). It should call
      invalid reference 
      org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils#getNodeStats(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.metadata.RelMetadataQuery)
      instead.
      Specified by:
      estimateNodeStats in interface BeamRelNode

    • beamComputeSelfCost

      public BeamCostModel beamComputeSelfCost(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelOptPlanner planner, BeamRelMetadataQuery mq)
      Description copied from interface: BeamRelNode
      This method is called by org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner.NonCumulativeCostImpl. This is currently only used in SQLTransform Path (and not JDBC path). This is needed when Calcite Query Planner wants to get the cost of a plan. Instead of calling this directly for a node, if we needed that it should be obtained by calling mq.getNonCumulativeCost. This way RelMetadataQuery will call this method instead of ComputeSelfCost if the handler is set correctly (see org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner#convertToBeamRel(String))
      Specified by:
      beamComputeSelfCost in interface BeamRelNode

    • isLimitOnly

      public boolean isLimitOnly()

    • getCount

      public int getCount()

    • buildPTransform

      public PTransform<PCollectionList<Row>,PCollection<Row>> buildPTransform()
      Specified by:
      buildPTransform in interface BeamRelNode

    • copy

      public org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort copy(org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.plan.RelTraitSet traitSet, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelNode newInput, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.RelCollation newCollation, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode offset, org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rex.RexNode fetch)
      Specified by:
      copy in class org.apache.beam.vendor.calcite.v1_40_0.org.apache.calcite.rel.core.Sort