t

io.projectglow.sql.expressions

AggregateByIndex

trait AggregateByIndex extends DeclarativeAggregate with HigherOrderFunction

An expression that allows users to aggregate over all array elements at a specific index in an array column. For example, this expression can be used to compute per-sample summary statistics from a genotypes column.

The user must provide the following arguments: - The array for aggregation - The initialValue for each element in the per-index buffer - An update function to update the buffer with a new element - A merge function to combine two buffers

The user may optionally provide an evaluate function. If it's not provided, the identity function is used.

Example usage to calculate average depth across all sites for a sample: aggregate_by_index( genotypes, named_struct('sum', 0l, 'count', 0l), (buf, genotype) -> named_struct('sum', buf.sum + genotype.depth, 'count', buf.count + 1), (buf1, buf2) -> named_struct('sum', buf1.sum + buf2.sum, 'count', buf1.count + buf2.count), buf -> buf.sum / buf.count)

Linear Supertypes
HigherOrderFunction, ExpectsInputTypes, DeclarativeAggregate, Unevaluable, Serializable, Serializable, AggregateFunction, Expression, TreeNode[Expression], Product, Equals, AnyRef, Any
Known Subclasses
UnwrappedAggregateByIndex, WrappedAggregateByIndex
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AggregateByIndex
  2. HigherOrderFunction
  3. ExpectsInputTypes
  4. DeclarativeAggregate
  5. Unevaluable
  6. Serializable
  7. Serializable
  8. AggregateFunction
  9. Expression
  10. TreeNode
  11. Product
  12. Equals
  13. AnyRef
  14. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. implicit class RichAttribute extends AnyRef
    Definition Classes
    DeclarativeAggregate

Abstract Value Members

  1. abstract def arr: Expression

    The array over which we're aggregating

  2. abstract def canEqual(that: Any): Boolean
    Definition Classes
    Equals
  3. abstract def evaluate: Expression

    Function to turn a buffer into the actual output

  4. abstract def initialValue: Expression

    The initial value for each element in the aggregation buffer

  5. abstract def merge: Expression

    Function to merge two buffers' elements

  6. abstract def productArity: Int
    Definition Classes
    Product
  7. abstract def productElement(n: Int): Any
    Definition Classes
    Product
  8. abstract def update: Expression

    Function to update an element of the buffer with a new input element

  9. abstract def withBoundExprs(newUpdate: Expression, newMerge: Expression, newEvaluate: Expression): AggregateByIndex

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def aggBufferAttributes: Seq[AttributeReference]
    Definition Classes
    AggregateByIndex → AggregateFunction
  5. final def aggBufferSchema: StructType
    Definition Classes
    DeclarativeAggregate → AggregateFunction
  6. def apply(number: Int): TreeNode[_]
    Definition Classes
    TreeNode
  7. def argString: String
    Definition Classes
    TreeNode
  8. def argumentTypes: Seq[ADT]
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  9. def arguments: Seq[Expression]
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  10. lazy val argumentsResolved: Boolean
    Definition Classes
    HigherOrderFunction
  11. def asCode: String
    Definition Classes
    TreeNode
  12. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  13. def bind(f: (Expression, Seq[(DataType, Boolean)]) ⇒ LambdaFunction): AggregateByIndex

    To create bound lambda functions, we bind each of the child higher order functions, extract their bound lambda functions, and then copy.

    To create bound lambda functions, we bind each of the child higher order functions, extract their bound lambda functions, and then copy.

    Definition Classes
    AggregateByIndex → HigherOrderFunction
  14. val buffer: AttributeReference
    Attributes
    protected
  15. lazy val canonicalized: Expression
    Definition Classes
    Expression
  16. def checkArgumentDataTypes(): TypeCheckResult
    Definition Classes
    HigherOrderFunction
  17. def checkInputDataTypes(): TypeCheckResult
    Definition Classes
    ExpectsInputTypes → Expression
  18. def children: Seq[Expression]
    Definition Classes
    HigherOrderFunction → TreeNode
  19. def childrenResolved: Boolean
    Definition Classes
    Expression
  20. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  21. def collect[B](pf: PartialFunction[Expression, B]): Seq[B]
    Definition Classes
    TreeNode
  22. def collectFirst[B](pf: PartialFunction[Expression, B]): Option[B]
    Definition Classes
    TreeNode
  23. def collectLeaves(): Seq[Expression]
    Definition Classes
    TreeNode
  24. lazy val containsChild: Set[TreeNode[_]]
    Definition Classes
    TreeNode
  25. def dataType: DataType
    Definition Classes
    AggregateByIndex → Expression
  26. def defaultResult: Option[Literal]
    Definition Classes
    AggregateFunction
  27. lazy val deterministic: Boolean
    Definition Classes
    Expression
  28. final def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode
    Attributes
    protected
    Definition Classes
    Unevaluable → Expression
  29. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  30. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  31. final def eval(input: InternalRow): Any
    Definition Classes
    Unevaluable → Expression
  32. lazy val evaluateExpression: Expression
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  33. def fastEquals(other: TreeNode[_]): Boolean
    Definition Classes
    TreeNode
  34. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  35. def find(f: (Expression) ⇒ Boolean): Option[Expression]
    Definition Classes
    TreeNode
  36. def flatArguments: Iterator[Any]
    Attributes
    protected
    Definition Classes
    Expression
  37. def flatMap[A](f: (Expression) ⇒ TraversableOnce[A]): Seq[A]
    Definition Classes
    TreeNode
  38. final def foldable: Boolean
    Definition Classes
    AggregateFunction → Expression
  39. def foreach(f: (Expression) ⇒ Unit): Unit
    Definition Classes
    TreeNode
  40. def foreachUp(f: (Expression) ⇒ Unit): Unit
    Definition Classes
    TreeNode
  41. def functionTypes: Seq[ADT]
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  42. def functions: Seq[Expression]
    Definition Classes
    AggregateByIndex → HigherOrderFunction
  43. lazy val functionsForEval: Seq[Expression]
    Definition Classes
    HigherOrderFunction
    Annotations
    @transient()
  44. def genCode(ctx: CodegenContext): ExprCode
    Definition Classes
    Expression
  45. def generateTreeString(depth: Int, lastChildren: Seq[Boolean], builder: StringBuilder, verbose: Boolean, prefix: String, addSuffix: Boolean): StringBuilder
    Definition Classes
    TreeNode
  46. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  47. def hashCode(): Int
    Definition Classes
    TreeNode → AnyRef → Any
  48. lazy val initialValues: Seq[Expression]
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  49. def innerChildren: Seq[TreeNode[_]]
    Attributes
    protected
    Definition Classes
    TreeNode
  50. final lazy val inputAggBufferAttributes: Seq[AttributeReference]
    Definition Classes
    DeclarativeAggregate → AggregateFunction
  51. def inputTypes: Seq[AbstractDataType]
    Definition Classes
    HigherOrderFunction → ExpectsInputTypes
  52. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  53. def jsonFields: List[JField]
    Attributes
    protected
    Definition Classes
    TreeNode
  54. def makeCopy(newArgs: Array[AnyRef]): Expression
    Definition Classes
    TreeNode
  55. def map[A](f: (Expression) ⇒ A): Seq[A]
    Definition Classes
    TreeNode
  56. def mapChildren(f: (Expression) ⇒ Expression): Expression
    Definition Classes
    TreeNode
  57. def mapProductIterator[B](f: (Any) ⇒ B)(implicit arg0: ClassTag[B]): Array[B]
    Attributes
    protected
    Definition Classes
    TreeNode
  58. lazy val mergeExpressions: Seq[Expression]
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  59. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  60. def nodeName: String
    Definition Classes
    TreeNode
  61. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  62. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  63. def nullable: Boolean
    Definition Classes
    AggregateByIndex → HigherOrderFunction → Expression
  64. def numberedTreeString: String
    Definition Classes
    TreeNode
  65. val origin: Origin
    Definition Classes
    TreeNode
  66. def otherCopyArgs: Seq[AnyRef]
    Attributes
    protected
    Definition Classes
    TreeNode
  67. def p(number: Int): Expression
    Definition Classes
    TreeNode
  68. def prettyJson: String
    Definition Classes
    TreeNode
  69. def prettyName: String
    Definition Classes
    AggregateByIndex → Expression
  70. def productIterator: Iterator[Any]
    Definition Classes
    Product
  71. def productPrefix: String
    Definition Classes
    Product
  72. def references: AttributeSet
    Definition Classes
    Expression
  73. lazy val resolved: Boolean
    Definition Classes
    HigherOrderFunction → Expression
  74. def semanticEquals(other: Expression): Boolean
    Definition Classes
    Expression
  75. def semanticHash(): Int
    Definition Classes
    Expression
  76. def simpleString: String
    Definition Classes
    Expression → TreeNode
  77. def sql(isDistinct: Boolean): String
    Definition Classes
    AggregateFunction
  78. def sql: String
    Definition Classes
    Expression
  79. def stringArgs: Iterator[Any]
    Attributes
    protected
    Definition Classes
    TreeNode
  80. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  81. def toAggString(isDistinct: Boolean): String
    Definition Classes
    AggregateFunction
  82. def toAggregateExpression(isDistinct: Boolean): AggregateExpression
    Definition Classes
    AggregateFunction
  83. def toAggregateExpression(): AggregateExpression
    Definition Classes
    AggregateFunction
  84. def toJSON: String
    Definition Classes
    TreeNode
  85. def toString(): String
    Definition Classes
    Expression → TreeNode → AnyRef → Any
  86. def transform(rule: PartialFunction[Expression, Expression]): Expression
    Definition Classes
    TreeNode
  87. def transformDown(rule: PartialFunction[Expression, Expression]): Expression
    Definition Classes
    TreeNode
  88. def transformUp(rule: PartialFunction[Expression, Expression]): Expression
    Definition Classes
    TreeNode
  89. def treeString(verbose: Boolean, addSuffix: Boolean): String
    Definition Classes
    TreeNode
  90. def treeString: String
    Definition Classes
    TreeNode
  91. lazy val updateExpressions: Seq[Expression]
    Definition Classes
    AggregateByIndex → DeclarativeAggregate
  92. final def verboseString: String
    Definition Classes
    Expression → TreeNode
  93. def verboseStringWithSuffix: String
    Definition Classes
    TreeNode
  94. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  95. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  96. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  97. def withNewChildren(newChildren: Seq[Expression]): Expression
    Definition Classes
    TreeNode

Inherited from HigherOrderFunction

Inherited from ExpectsInputTypes

Inherited from DeclarativeAggregate

Inherited from Unevaluable

Inherited from Serializable

Inherited from Serializable

Inherited from AggregateFunction

Inherited from Expression

Inherited from TreeNode[Expression]

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped