trait AlterDeltaTableCommand extends DeltaCommand
A super trait for alter table commands that modify Delta tables.
- Alphabetic
- By Inheritance
- AlterDeltaTableCommand
- DeltaCommand
- DeltaLogging
- DatabricksLogging
- DeltaProgressReporter
- Logging
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def table: DeltaTableV2
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def buildBaseRelation(spark: SparkSession, txn: OptimisticTransaction, actionType: String, rootPath: Path, inputLeafFiles: Seq[String], nameToAddFileMap: Map[String, AddFile]): HadoopFsRelation
Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.
Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def checkDependentExpressions(sparkSession: SparkSession, columnParts: Seq[String], newMetadata: Metadata, protocol: Protocol, operationName: String): Unit
Check if the column to change has any dependent expressions:
Check if the column to change has any dependent expressions:
- generated column expressions
- check constraints
- Attributes
- protected
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def commitLarge(spark: SparkSession, txn: OptimisticTransaction, actions: Iterator[Action], op: Operation, context: Map[String, String], metrics: Map[String, String]): Long
Create a large commit on the Delta log by directly writing an iterator of FileActions to the LogStore.
Create a large commit on the Delta log by directly writing an iterator of FileActions to the LogStore. This function only commits the next possible version and will not check whether the commit is retry-able. If the next version has already been committed, then this function will fail. This bypasses all optimistic concurrency checks. We assume that transaction conflicts should be rare because this method is typically used to create new tables (e.g. CONVERT TO DELTA) or apply some commands which rarely receive other transactions (e.g. CLONE/RESTORE).
- Attributes
- protected
- Definition Classes
- DeltaCommand
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def generateCandidateFileMap(basePath: Path, candidateFiles: Seq[AddFile]): Map[String, AddFile]
Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update.
Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update. We expect file names to be unique, because each file contains a UUID.
- Attributes
- protected
- Definition Classes
- DeltaCommand
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def getDeltaLog(spark: SparkSession, path: Option[String], tableIdentifier: Option[TableIdentifier], operationName: String): DeltaLog
Utility method to return the DeltaLog of an existing Delta table referred by either the given path or
Utility method to return the DeltaLog of an existing Delta table referred by either the given path or
- spark
SparkSession reference to use.
- path
Table location. Expects a non-empty tableIdentifier or path.
- tableIdentifier
Table identifier. Expects a non-empty tableIdentifier or path.
- operationName
Operation that is getting the DeltaLog, used in error messages.
- returns
DeltaLog of the table
- Attributes
- protected
- Definition Classes
- DeltaCommand
- Exceptions thrown
AnalysisExceptionIf either no Delta table exists at the given path/identifier or there is neither path nor tableIdentifier is provided.
- def getTouchedFile(basePath: Path, filePath: String, nameToAddFileMap: Map[String, AddFile]): AddFile
Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.
Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.
- filePath
The path to a file. Can be either absolute or relative
- nameToAddFileMap
Map generated through
generateCandidateFileMap()
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def isCatalogTable(analyzer: Analyzer, tableIdent: TableIdentifier): Boolean
Use the analyzer to see whether the provided TableIdentifier is for a path based table or not
Use the analyzer to see whether the provided TableIdentifier is for a path based table or not
- analyzer
The session state analyzer to call
- tableIdent
Table Identifier to determine whether is path based or not
- returns
Boolean where true means that the table is a table in a metastore and false means the table is a path based table
- Definition Classes
- DeltaCommand
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isPathIdentifier(tableIdent: TableIdentifier): Boolean
Checks if the given identifier can be for a delta table's path
Checks if the given identifier can be for a delta table's path
- tableIdent
Table Identifier for which to check
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logConsole(line: String): Unit
- Definition Classes
- DatabricksLogging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def parsePredicates(spark: SparkSession, predicate: String): Seq[Expression]
Converts string predicates into Expressions relative to a transaction.
Converts string predicates into Expressions relative to a transaction.
- Attributes
- protected
- Definition Classes
- DeltaCommand
- Exceptions thrown
AnalysisExceptionif a non-partition column is referenced.
- def recordDeltaEvent(deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty, data: AnyRef = null, path: Option[Path] = None): Unit
Used to record the occurrence of a single event or report detailed, operation specific statistics.
Used to record the occurrence of a single event or report detailed, operation specific statistics.
- path
Used to log the path of the delta table when
deltaLogis null.
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordDeltaOperation[A](deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: => A): A
Used to report the duration as well as the success or failure of an operation on a
deltaLog.Used to report the duration as well as the success or failure of an operation on a
deltaLog.- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordDeltaOperationForTablePath[A](tablePath: String, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: => A): A
Used to report the duration as well as the success or failure of an operation on a
tahoePath.Used to report the duration as well as the success or failure of an operation on a
tahoePath.- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordEvent(metric: MetricDefinition, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
- Definition Classes
- DatabricksLogging
- def recordFrameProfile[T](group: String, name: String)(thunk: => T): T
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordOperation[S](opType: OpType, opTarget: String = null, extraTags: Map[TagDefinition, String], isSynchronous: Boolean = true, alwaysRecordStats: Boolean = false, allowAuthTags: Boolean = false, killJvmIfStuck: Boolean = false, outputMetric: MetricDefinition = null, silent: Boolean = true)(thunk: => S): S
- Definition Classes
- DatabricksLogging
- def recordProductEvent(metric: MetricDefinition with CentralizableMetric, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
- Definition Classes
- DatabricksLogging
- def recordProductUsage(metric: MetricDefinition with CentralizableMetric, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
- Definition Classes
- DatabricksLogging
- def recordUsage(metric: MetricDefinition, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
- Definition Classes
- DatabricksLogging
- def removeFilesFromPaths(deltaLog: DeltaLog, nameToAddFileMap: Map[String, AddFile], filesToRewrite: Seq[String], operationTimestamp: Long): Seq[RemoveFile]
This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.
This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.
- deltaLog
The DeltaLog of the table that is being operated on
- nameToAddFileMap
A map generated using
generateCandidateFileMap.- filesToRewrite
Absolute paths of the files that were touched. We will search for these in
candidateFiles. Obtained as the output of theinput_file_namefunction.- operationTimestamp
The timestamp of the operation
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def resolveIdentifier(analyzer: Analyzer, identifier: TableIdentifier): LogicalPlan
Use the analyzer to resolve the identifier provided
Use the analyzer to resolve the identifier provided
- analyzer
The session state analyzer to call
- identifier
Table Identifier to determine whether is path based or not
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def startTransaction(): OptimisticTransaction
- Attributes
- protected
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def updateAndCheckpoint(spark: SparkSession, deltaLog: DeltaLog, commitSize: Int, attemptVersion: Long, txnId: String): Snapshot
Update the table now that the commit has been made, and write a checkpoint.
Update the table now that the commit has been made, and write a checkpoint.
- Attributes
- protected
- Definition Classes
- DeltaCommand
- def verifyPartitionPredicates(spark: SparkSession, partitionColumns: Seq[String], predicates: Seq[Expression]): Unit
- Definition Classes
- DeltaCommand
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- def withDmqTag[T](thunk: => T): T
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def withStatusCode[T](statusCode: String, defaultMessage: String, data: Map[String, Any] = Map.empty)(body: => T): T
Report a log to indicate some command is running.
Report a log to indicate some command is running.
- Definition Classes
- DeltaProgressReporter