class DeltaOptions extends DeltaWriteOptions with DeltaReadOptions with Serializable
Options for the Delta data source.
- Alphabetic
- By Inheritance
- DeltaOptions
- Serializable
- DeltaReadOptions
- DeltaWriteOptions
- DeltaWriteOptionsImpl
- DeltaOptionParser
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def canMergeSchema: Boolean
Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration.
Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration. Automerging is off when table ACLs are enabled. We always respect the DataFrame writer configuration over the session config.
- Definition Classes
- DeltaWriteOptionsImpl
- def canOverwriteSchema: Boolean
Whether to allow overwriting the schema of a Delta table in an overwrite mode operation.
Whether to allow overwriting the schema of a Delta table in an overwrite mode operation. If ACLs are enabled, we can't change the schema of an operation through a write, which requires MODIFY permissions, when schema changes require OWN permissions.
- Definition Classes
- DeltaWriteOptionsImpl
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- val excludeRegex: Option[Regex]
- Definition Classes
- DeltaReadOptions
- val failOnDataLoss: Boolean
- Definition Classes
- DeltaReadOptions
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- val ignoreChanges: Boolean
- Definition Classes
- DeltaReadOptions
- val ignoreDeletes: Boolean
- Definition Classes
- DeltaReadOptions
- val ignoreFileDeletion: Boolean
- Definition Classes
- DeltaReadOptions
- def isDynamicPartitionOverwriteMode: Boolean
Whether to only overwrite partitions that have data written into it at runtime.
Whether to only overwrite partitions that have data written into it at runtime.
- Definition Classes
- DeltaWriteOptionsImpl
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val maxBytesPerTrigger: Option[Long]
- Definition Classes
- DeltaReadOptions
- val maxFilesPerTrigger: Option[Int]
- Definition Classes
- DeltaReadOptions
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val optimizeWrite: Option[Boolean]
Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.
Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.
- Definition Classes
- DeltaWriteOptions
- val options: CaseInsensitiveMap[String]
- Attributes
- protected[delta]
- Definition Classes
- DeltaOptions → DeltaOptionParser
- val partitionOverwriteModeInOptions: Boolean
Whether partitionOverwriteMode is provided as a DataFrameWriter option.
Whether partitionOverwriteMode is provided as a DataFrameWriter option.
- Definition Classes
- DeltaWriteOptionsImpl
- val readChangeFeed: Boolean
- Definition Classes
- DeltaReadOptions
- def rearrangeOnly: Boolean
Whether to write new data to the table or just rearrange data that is already part of the table.
Whether to write new data to the table or just rearrange data that is already part of the table. This option declares that the data being written by this job does not change any data in the table and merely rearranges existing data. This makes sure streaming queries reading from this table will not see any new changes
- Definition Classes
- DeltaWriteOptionsImpl
- val replaceWhere: Option[String]
- Definition Classes
- DeltaWriteOptions
- val sqlConf: SQLConf
- Attributes
- protected
- Definition Classes
- DeltaOptions → DeltaOptionParser
- val startingTimestamp: Option[String]
- Definition Classes
- DeltaReadOptions
- val startingVersion: Option[DeltaStartingVersion]
- Definition Classes
- DeltaReadOptions
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toBoolean(input: String, name: String): Boolean
- Definition Classes
- DeltaOptionParser
- def toString(): String
- Definition Classes
- AnyRef → Any
- val txnAppId: Option[String]
- Definition Classes
- DeltaWriteOptionsImpl
- val txnVersion: Option[Long]
- Definition Classes
- DeltaWriteOptionsImpl
- val userMetadata: Option[String]
- Definition Classes
- DeltaWriteOptions
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()