class BuildSidePartitioner extends JoinPartitioner
Join partitioner for the build side of a large join where the build side of the join does not fit in a single GPU batch.
- Alphabetic
- By Inheritance
- BuildSidePartitioner
- JoinPartitioner
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
BuildSidePartitioner(joinType: JoinType, numPartitions: Int, buildSideIter: Iterator[ColumnarBatch], buildSideTypes: Array[DataType], boundBuildKeys: Seq[Expression], gpuBatchSizeBytes: Long, metrics: Map[String, GpuMetric])
- joinType
type of join being performed
- numPartitions
number of partitions being used in the join
- buildSideIter
iterator of build side batches
- buildSideTypes
schema of the build side batches
- boundBuildKeys
bound join key expressions for the build side
- gpuBatchSizeBytes
target GPU batch size
- metrics
metrics to update
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
addPartition(partIndex: Int, ct: ContiguousTable): Unit
Adds a batch associated with a partition.
Adds a batch associated with a partition.
- partIndex
index of the partition
- ct
contiguous table to add to the specified partition
- Attributes
- protected
- Definition Classes
- JoinPartitioner
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
close(): Unit
- Definition Classes
- BuildSidePartitioner → JoinPartitioner → AutoCloseable
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
getBuildBatch(partitionGroupIndex: Int): LazySpillableColumnarBatch
Returns the batch of data for the specified join group index.
Returns the batch of data for the specified join group index. All of the data across the partitions comprising the join group are concatenated together to produce the batch. The concatenated batches are lazily cached, so the cost of concatenation is only incurred by the first caller for a particular join group.
- partitionGroupIndex
the index of the join group for which to produce the batch
- returns
the batch of data for the join group
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getEmptyPartitions: BitSet
Returns a BitSet where a set bit corresponds to an empty partition at that index.
-
def
getJoinGroups: Array[BitSet]
Returns a BitSet array where each BitSet corresponds to a "join group," a group of partitions that can together be used as a single build batch for a sub-join.
Returns a BitSet array where each BitSet corresponds to a "join group," a group of partitions that can together be used as a single build batch for a sub-join. A set bit in a join group's BitSet corresponds to a partition index that is part of the join group.
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val joinType: JoinType
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val numPartitions: Int
-
val
opTime: GpuMetric
- Attributes
- protected
- Definition Classes
- JoinPartitioner
-
def
partitionBatch(inputBatch: ColumnarBatch): Unit
Hash partitions a batch in preparation for performing a sub-join.
Hash partitions a batch in preparation for performing a sub-join. The input batch will be closed by this method.
- Attributes
- protected
- Definition Classes
- JoinPartitioner
-
val
partitions: Array[JoinPartition]
- Attributes
- protected
- Definition Classes
- JoinPartitioner
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()