Packages

abstract class RapidsMeta[INPUT <: BASE, BASE, OUTPUT <: BASE] extends AnyRef

Holds metadata about a stage in the physical plan that is separate from the plan itself. This is helpful in deciding when to replace part of the plan with a GPU enabled version.

INPUT

the exact type of the class we are wrapping.

BASE

the generic base class for this type of stage, i.e. SparkPlan, Expression, etc.

OUTPUT

when converting to a GPU enabled version of the plan, the generic base type for all GPU enabled versions.

Linear Supertypes
AnyRef, Any
Known Subclasses
AggExprMeta, AppendDataExecV1Meta, AtomicCreateTableAsSelectExecMeta, AtomicReplaceTableAsSelectExecMeta, BaseExprMeta, BinaryAstExprMeta, BinaryExprMeta, CastExprMeta, CastExprMetaBase, ComplexTypeMergingExprMeta, CreatableRelationProviderMeta, DataWritingCommandMeta, DoNotReplaceOrWarnSparkPlanMeta, ExecutedCommandExecMeta, ExprMeta, GeneratorExprMeta, GpuBaseAggregateMeta, GpuCollectLimitMeta, GpuExpandExecMeta, GpuFilterExecMeta, GpuGenerateExecSparkPlanMeta, GpuGetJsonObjectMeta, GpuHashAggregateMeta, GpuMapFromArraysMeta, GpuObjectHashAggregateExecMeta, GpuProjectExecMeta, GpuRegExpReplaceMeta, GpuSampleExecMeta, GpuShuffledHashJoinMeta, GpuSortAggregateExecMeta, GpuSortMergeJoinMeta, GpuSortMeta, GpuSortOrderMeta, GpuStackMeta, GpuTakeOrderedAndProjectExecMeta, GpuTypedImperativeSupportedAggregateExecMeta, ImperativeAggExprMeta, InsertIntoHadoopFsRelationCommandMeta, LiteralExprMeta, OverwriteByExpressionExecV1Meta, PartMeta, QuaternaryExprMeta, ReplicateRowsExprMeta, RuleNotFoundCreatableRelationProviderMeta, RuleNotFoundDataWritingCommandMeta, RuleNotFoundExprMeta, RuleNotFoundPartMeta, RuleNotFoundRunnableCommandMeta, RuleNotFoundScanMeta, RuleNotFoundSparkPlanMeta, RunnableCommandMeta, RuntimeReplaceableUnaryAstExprMeta, RuntimeReplaceableUnaryExprMeta, SaveIntoDataSourceCommandMeta, ScanMeta, SparkPlanMeta, String2TrimExpressionMeta, TernaryExprMeta, TypedImperativeAggExprMeta, UnaryAstExprMeta, UnaryExprMeta, UnaryExprMetaBase, BatchScanExecMeta, CreateDataSourceTableAsSelectCommandMeta, FileSourceScanExecMeta, GetMapValueMeta, GpuAggregateInPandasExecMeta, GpuBroadcastJoinMeta, GpuCustomShuffleReaderMeta, GpuSpecifiedWindowFrameMeta, GpuWindowExpressionMeta, OffsetWindowFunctionMeta, OptimizedCreateHiveTableAsSelectCommandMeta, RapidsCsvScanMeta, RapidsOrcScanMeta, RapidsParquetScanMeta, GpuBaseWindowExecMeta, GpuSpecifiedWindowFrameMetaBase, GpuWindowExecMeta, GpuWindowExpressionMetaBase, GpuWindowSpecDefinitionMeta, GpuCreateHiveTableAsSelectCommandMeta, GpuInsertIntoHiveTableMeta, ConvertUTCTimestampExprMetaBase, FromUTCTimestampExprMeta, FromUnixTimeMeta, GpuConvMeta, GpuGetArrayStructFieldsMeta, GpuRLikeMeta, GpuRegExpExtractAllMeta, GpuRegExpExtractMeta, GpuSequenceMeta, GpuStringSplitMeta, GpuStringToMapMeta, GpuStructsToJsonMeta, InMemoryTableScanMeta, StringSplitRegExpMeta, SubstringIndexMeta, ToUTCTimestampExprMeta, UnixTimeExprMeta, GpuBroadcastHashJoinMeta, GpuBroadcastHashJoinMetaBase, GpuBroadcastMeta, GpuBroadcastNestedLoopJoinMeta, GpuBroadcastNestedLoopJoinMetaBase, GpuShuffleMeta, GpuShuffleMetaBase, GpuSubqueryBroadcastMeta, GpuSubqueryBroadcastMetaBase, GpuFlatMapCoGroupsInPandasExecMeta, GpuFlatMapGroupsInPandasExecMeta, GpuMapInPandasExecMetaBase, GpuWindowInPandasExecMetaBase, GpuMapInPandasExecMeta
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RapidsMeta
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new RapidsMeta(wrapped: INPUT, conf: RapidsConf, parent: Option[RapidsMeta[_, _, _]], rule: DataFromReplacementRule)

    wrapped

    what we are wrapping

    conf

    the config

    parent

    the parent of this node, if there is one.

    rule

    holds information related to the config for this object, typically this is the rule used to wrap the stage.

Abstract Value Members

  1. abstract val childDataWriteCmds: Seq[DataWritingCommandMeta[_]]

    The wrapped data writing commands that should be examined

  2. abstract val childExprs: Seq[BaseExprMeta[_]]

    The wrapped expressions that should be examined

  3. abstract val childParts: Seq[PartMeta[_]]

    The wrapped partitioning that should be examined

  4. abstract val childPlans: Seq[SparkPlanMeta[_]]

    The wrapped plans that should be examined

  5. abstract val childScans: Seq[ScanMeta[_]]

    The wrapped scans that should be examined

  6. abstract def convertToGpu(): OUTPUT

    Convert what this wraps to a GPU enabled version.

  7. abstract def tagSelfForGpu(): Unit

    Do any extra checks and tag yourself if you are compatible or not.

    Do any extra checks and tag yourself if you are compatible or not. Be aware that this may already have been marked as incompatible for a number of reasons.

    All of your children should have already been tagged so if there are situations where you may need to disqualify your children for various reasons you may do it here too.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def addConvertedDataType(expression: Expression, typeMeta: DataTypeMeta): Unit

    Call this method to record information about type conversions via DataTypeMeta.

  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def canDataWriteCmdsBeReplaced: Boolean

    Returns true iff all of the data writing commands can be replaced.

  7. def canExprTreeBeReplaced: Boolean

    Returns true iff all of the expressions and their children could be replaced.

  8. def canPartsBeReplaced: Boolean

    Returns true iff all of the partitioning can be replaced.

  9. def canScansBeReplaced: Boolean

    Returns true iff all of the scans can be replaced.

  10. final def canThisBeReplaced: Boolean

    Returns true iff this could be replaced.

  11. var cannotBeReplacedReasons: Option[Set[String]]
    Attributes
    protected
  12. var cannotRunOnGpuBecauseOfCost: Boolean
    Attributes
    protected
  13. var cannotRunOnGpuBecauseOfSparkPlan: Boolean
    Attributes
    protected
  14. def checkTimeZoneId(sessionZoneId: ZoneId): Unit
  15. val childRunnableCmds: Seq[RunnableCommandMeta[_]]

    The wrapped runnable commands that should be examined.

  16. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  17. val conf: RapidsConf
  18. def confKey: String
  19. def convertToCpu(): BASE

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled.

    Keep this on the CPU, but possibly convert its children under it to run on the GPU if enabled. By default this just returns what is wrapped by this. For some types of operators/stages, like SparkPlan, each part of the query can be converted independent of other parts. As such in a subclass this should be overridden to do the correct thing.

  20. final def costPreventsRunningOnGpu(): Unit
  21. final val disabledMsg: Option[String]
  22. final def entirePlanExcludedReasons: Set[String]

    Returns the list of reasons the entire plan can't be replaced.

    Returns the list of reasons the entire plan can't be replaced. An empty set means the entire plan is ok to be replaced, do the normal checking per exec and children.

  23. final def entirePlanWillNotWork(because: String): Unit

    Call this if there is a condition found that the entire plan is not allowed to run on the GPU.

  24. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  25. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  26. def explain(all: Boolean): String
  27. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  28. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  29. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  30. final val incompatDoc: Option[String]
  31. def indent(append: StringBuilder, depth: Int): Unit
    Attributes
    protected
  32. def initReasons(): Unit
  33. def isDisabledByDefault: Boolean
  34. def isIncompat: Boolean
  35. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  36. final def mustBeReplaced(because: String): Unit
  37. final def mustThisBeReplaced: Boolean

    Returns true iff this must be replaced because its children have already been replaced and this needs to also be replaced for compatibility.

  38. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  39. def noReplacementPossibleMessage(reasons: String): String
  40. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  41. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  42. final val operationName: String
  43. val parent: Option[RapidsMeta[_, _, _]]
  44. def print(strBuilder: StringBuilder, depth: Int, all: Boolean): Unit

    Create a string representation of this in append.

    Create a string representation of this in append.

    strBuilder

    where to place the string representation.

    depth

    how far down the tree this is.

    all

    should all the data be printed or just what does not work on the GPU?

  45. val printWrapped: Boolean

    When converting this to a string should we include the string representation of what this wraps too? This is off by default.

    When converting this to a string should we include the string representation of what this wraps too? This is off by default.

    Attributes
    protected
  46. final def recursiveCostPreventsRunningOnGpu(): Unit

    Recursively force a section of the plan back onto CPU, stopping once a plan is reached that is already on CPU.

  47. final def recursiveSparkPlanPreventsRunningOnGpu(): Unit
  48. final def recursiveSparkPlanRemoved(): Unit
  49. def replaceMessage: String
  50. final def shouldBeRemoved(because: String): Unit
  51. final def shouldThisBeRemoved: Boolean

    Returns true if this node should be removed.

  52. def supportsColumnar: Boolean

    Return true if the resulting node in the plan will support columnar execution

  53. def suppressWillWorkOnGpuInfo: Boolean
  54. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  55. final def tagForGpu(): Unit

    Tag all of the children to see if they are GPU compatible first.

    Tag all of the children to see if they are GPU compatible first. Do basic common verification for the operators, and then call tagSelfForGpu

  56. def toString(): String
    Definition Classes
    RapidsMeta → AnyRef → Any
  57. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  58. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  59. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  60. final def willNotWorkOnGpu(because: String): Unit

    Call this to indicate that this should not be replaced with a GPU enabled version

    Call this to indicate that this should not be replaced with a GPU enabled version

    because

    why it should not be replaced.

  61. val wrapped: INPUT

Inherited from AnyRef

Inherited from Any

Ungrouped