Packages

object TypeSig

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TypeSig
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val ARRAY: TypeSig

    ARRAY type support, but not very useful on its own because no child types under it are supported

  5. val BINARY: TypeSig
  6. val BOOLEAN: TypeSig
  7. val BYTE: TypeSig
  8. val CALENDAR: TypeSig
  9. val DATE: TypeSig
  10. val DAYTIME: TypeSig

    DayTimeIntervalType of Spark 3.2.0+ support

  11. val DECIMAL_128: TypeSig

    Full support for 128 bit DECIMAL.

    Full support for 128 bit DECIMAL. In the future we expect to have other types with slightly less than full DECIMAL support. This are things like math operations where we cannot replicate the overflow behavior of Spark. These will be added when needed.

  12. val DECIMAL_64: TypeSig
  13. val DOUBLE: TypeSig
  14. val FLOAT: TypeSig
  15. val INT: TypeSig
  16. val LONG: TypeSig
  17. val MAP: TypeSig

    MAP type support, but not very useful on its own because no child types under it are supported

  18. val NULL: TypeSig
  19. val SHORT: TypeSig
  20. val STRING: TypeSig
  21. val STRUCT: TypeSig

    STRUCT type support, but only matches empty structs unless you add child types to it.

  22. val TIMESTAMP: TypeSig
  23. val UDT: TypeSig

    User Defined Type (We don't support these in the plugin yet)

  24. val YEARMONTH: TypeSig

    YearMonthIntervalType of Spark 3.2.0+ support

  25. val all: TypeSig

    All types nested and not nested

  26. val ansiIntervals: TypeSig

    ANSI year-month and day-time interval for Spark 320+

  27. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  28. val astTypes: TypeSig

    All types that can appear in AST expressions

  29. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  30. val commonCudfTypes: TypeSig

    A signature for types that are generally supported by the plugin/CUDF.

    A signature for types that are generally supported by the plugin/CUDF. Please make sure to check what Spark actually supports instead of blindly using this in a signature.

  31. def commonCudfTypesLit(): TypeSig

    Create a TypeSig that supports only literals of common primitive CUDF types.

  32. val commonCudfTypesWithNested: TypeSig

    commonCudfTypes plus decimal, null and nested types.

  33. val comparable: TypeSig

    All types that Spark supports for comparison operators (really everything but MAP according to https://spark.apache.org/docs/latest/api/sql/index.html#_12), e.g.

    All types that Spark supports for comparison operators (really everything but MAP according to https://spark.apache.org/docs/latest/api/sql/index.html#_12), e.g. "<=>", "=", "==".

  34. val comparisonAstTypes: TypeSig

    All AST types that work for comparisons

  35. val cpuAtomics: TypeSig

    All values that correspond to Spark's AtomicType

  36. val cpuNumeric: TypeSig

    All numeric types fp + integral + DECIMAL_128

  37. def decimal(maxPrecision: Int): TypeSig
  38. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  39. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  40. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  41. val fp: TypeSig

    All floating point types

  42. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  43. def getDataType(expr: Expression): Option[DataType]
  44. val gpuAtomics: TypeSig

    All values that correspond to Spark's AtomicType but supported by GPU

  45. val gpuNumeric: TypeSig

    All numeric types fp + integral + DECIMAL_128

  46. val gpuNumericAndInterval: TypeSig

    numeric + CALENDAR but only for GPU

  47. val gpuOrderable: TypeSig

    All types that CUDF supports sorting/ordering on.

  48. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  49. val implicitCastsAstTypes: TypeSig

    All types that can appear in an implicit cast AST expression

  50. val integral: TypeSig

    All integer types

  51. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  52. def lit(dataTypes: TypeEnum.ValueSet): TypeSig

    Create a TypeSig that only supports literals of certain given types.

  53. def lit(dataType: TypeEnum.Value): TypeSig

    Create a TypeSig that only supports a literal of the given type.

  54. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  55. val none: TypeSig

    No types supported at all

  56. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  57. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  58. val numericAndInterval: TypeSig

    numeric + CALENDAR

  59. val orderable: TypeSig

    All types that Spark supports sorting/ordering on (really everything but MAP)

  60. def psNote(dataType: TypeEnum.Value, note: String): TypeSig

    Create a TypeSig that has partial support for the given type.

  61. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  62. def toString(): String
    Definition Classes
    AnyRef → Any
  63. val unionOfPandasUdfOut: TypeSig

    Different types of Pandas UDF support different sets of output type.

    Different types of Pandas UDF support different sets of output type. Please refer to https://github.com/apache/spark/blob/master/python/pyspark/sql/udf.py#L98 for more details.

    It is impossible to specify the exact type signature for each Pandas UDF type in a single expression 'PythonUDF'.

    So here comes the union of all the sets of supported type, to cover all the cases.

  64. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  65. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  66. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped