class OptimisticTransaction extends OptimisticTransactionImpl with DeltaLogging
Used to perform a set of reads in a transaction and then commit a set of updates to the state of the log. All reads from the DeltaLog, MUST go through this instance rather than directly to the DeltaLog otherwise they will not be check for logical conflicts with concurrent updates.
This class is not thread-safe.
- Alphabetic
- By Inheritance
- OptimisticTransaction
- OptimisticTransactionImpl
- DeltaScanGenerator
- DeltaScanGeneratorBase
- SQLMetricsReporting
- TransactionalWrite
- DeltaLogging
- DatabricksLogging
- DeltaProgressReporter
- Logging
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new OptimisticTransaction(deltaLog: DeltaLog)(implicit clock: Clock)
Creates a new OptimisticTransaction.
Creates a new OptimisticTransaction.
- deltaLog
The Delta Log for the table this transaction is modifying.
- new OptimisticTransaction(deltaLog: DeltaLog, snapshot: Snapshot)(implicit clock: Clock)
- deltaLog
The Delta Log for the table this transaction is modifying.
- snapshot
The snapshot that this transaction is reading at.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def canDowngradeToSnapshotIsolation(preparedActions: Seq[Action], op: Operation): Boolean
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def checkForConflicts(checkVersion: Long, currentTransactionInfo: CurrentTransactionInfo, attemptNumber: Int, commitIsolationLevel: IsolationLevel): (Long, CurrentTransactionInfo)
Looks at actions that have happened since the txn started and checks for logical conflicts with the read/writes.
Looks at actions that have happened since the txn started and checks for logical conflicts with the read/writes. Resolve conflicts and returns a tuple representing the commit version to attempt next and the commit summary which we need to commit.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def checkForConflictsAgainstVersion(currentTransactionInfo: CurrentTransactionInfo, otherCommitVersion: Long, commitIsolationLevel: IsolationLevel): CurrentTransactionInfo
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def checkPartitionColumns(partitionSchema: StructType, output: Seq[Attribute], colsDropped: Boolean): Unit
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- val checkUnsupportedDataType: Boolean
Whether to check unsupported data type when updating the table schema
Whether to check unsupported data type when updating the table schema
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- implicit val clock: Clock
- Definition Classes
- OptimisticTransaction → OptimisticTransactionImpl
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def commit(actions: Seq[Action], op: Operation): Long
Modifies the state of the log by adding a new commit that is based on a read at the given
lastVersion.Modifies the state of the log by adding a new commit that is based on a read at the given
lastVersion. In the case of a conflict with a concurrent writer this method will throw an exception.- actions
Set of actions to commit
- op
Details of operation that is performing this transactional commit
- Definition Classes
- OptimisticTransactionImpl
- Annotations
- @throws(classOf[ConcurrentModificationException])
- val commitAttemptStartTime: Long
Tracks the start time since we started trying to write a particular commit.
Tracks the start time since we started trying to write a particular commit. Used for logging duration of retried transactions.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val commitEndNano: Long
The transaction commit end time.
The transaction commit end time.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val commitInfo: CommitInfo
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val commitStartNano: Long
The transaction commit start time.
The transaction commit start time.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val committed: Boolean
Tracks if this transaction has already committed.
Tracks if this transaction has already committed.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def convertEmptyToNullIfNeeded(plan: SparkPlan, partCols: Seq[Attribute], constraints: Seq[Constraint]): SparkPlan
If there is any string partition column and there are constraints defined, add a projection to convert empty string to null for that column.
If there is any string partition column and there are constraints defined, add a projection to convert empty string to null for that column. The empty strings will be converted to null eventually even without this convert, but we want to do this earlier before check constraints so that empty strings are correctly rejected. Note that this should not cause the downstream logic in
FileFormatWriterto add duplicate conversions because the logic there checks the partition column using the original plan's output. When the plan is modified with additional projections, the partition column check won't match and will not add more conversion.- plan
The original SparkPlan.
- partCols
The partition columns.
- constraints
The defined constraints.
- returns
A SparkPlan potentially modified with an additional projection on top of
plan
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- val deltaLog: DeltaLog
- Definition Classes
- OptimisticTransaction → OptimisticTransactionImpl → TransactionalWrite
- def doCommit(attemptVersion: Long, currentTransactionInfo: CurrentTransactionInfo, attemptNumber: Int, isolationLevel: IsolationLevel): Boolean
Commit
actionsusingattemptVersionversion number.Commit
actionsusingattemptVersionversion number. Throws a FileAlreadyExistsException if any conflicts are detected.- returns
if a checkpoint should be made once this commit is written
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def doCommitRetryIteratively(attemptVersion: Long, currentTransactionInfo: CurrentTransactionInfo, isolationLevel: IsolationLevel): (Long, CurrentTransactionInfo, Boolean)
Commit the txn represented by
currentTransactionInfousingattemptVersionversion number.Commit the txn represented by
currentTransactionInfousingattemptVersionversion number. If there are any conflicts that are found, we will retry a fixed number of times.- returns
the real version that was committed, txn info, and if a checkpoint should be made.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def filesForScan(projection: Seq[Attribute], filters: Seq[Expression]): DeltaScan
Returns aDeltaScan based on the given filters and projections.
Returns aDeltaScan based on the given filters and projections.
- Definition Classes
- OptimisticTransactionImpl → DeltaScanGeneratorBase
- def filesWithStatsForScan(partitionFilters: Seq[Expression]): DataFrame
Returns a DataFrame for the given partition filters.
Returns a DataFrame for the given partition filters. The schema of returned DataFrame is nearly the same as
AddFile, except that thestatsfield is parsed to a struct from a json string.- Definition Classes
- OptimisticTransactionImpl → DeltaScanGeneratorBase
- def filterFiles(partitions: Set[Map[String, String]]): Seq[AddFile]
Returns files within the given partitions.
Returns files within the given partitions.
- Definition Classes
- OptimisticTransactionImpl
- def filterFiles(filters: Seq[Expression]): Seq[AddFile]
Returns files matching the given predicates.
Returns files matching the given predicates.
- Definition Classes
- OptimisticTransactionImpl
- def filterFiles(): Seq[AddFile]
Returns files matching the given predicates.
Returns files matching the given predicates.
- Definition Classes
- OptimisticTransactionImpl
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def getCommitter(outputPath: Path): DelayedCommitProtocol
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def getDefaultIsolationLevel(): IsolationLevel
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def getDeltaScanGenerator(index: TahoeLogFileIndex): DeltaScanGenerator
Returns the DeltaScanGenerator for the given log, which will be used to generate DeltaScans.
Returns the DeltaScanGenerator for the given log, which will be used to generate DeltaScans. Every time this method is called on a log, the returned generator generator will read a snapshot that is pinned on the first access for that log.
Internally, if the given log is the same as the log associated with this transaction, then it returns this transaction, otherwise it will return a snapshot of given log
- Definition Classes
- OptimisticTransactionImpl
- def getIsolationLevelToUse(preparedActions: Seq[Action], op: Operation): IsolationLevel
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def getMetric(name: String): Option[SQLMetric]
Returns the metric with
nameregistered for the given transaction if it exists.Returns the metric with
nameregistered for the given transaction if it exists.- Definition Classes
- SQLMetricsReporting
- def getMetricsForOperation(operation: Operation): Map[String, String]
Get the metrics for an operation based on collected SQL Metrics and filtering out the ones based on the metric parameters for that operation.
Get the metrics for an operation based on collected SQL Metrics and filtering out the ones based on the metric parameters for that operation.
- Definition Classes
- SQLMetricsReporting
- def getNextAttemptVersion(previousAttemptVersion: Long): Long
Returns the next attempt version given the last attempted version
Returns the next attempt version given the last attempted version
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def getOperationMetrics(op: Operation): Option[Map[String, String]]
Return the operation metrics for the operation if it is enabled
Return the operation metrics for the operation if it is enabled
- Definition Classes
- OptimisticTransactionImpl
- def getPartitioningColumns(partitionSchema: StructType, output: Seq[Attribute]): Seq[Attribute]
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def getUserMetadata(op: Operation): Option[String]
Return the user-defined metadata for the operation.
Return the user-defined metadata for the operation.
- Definition Classes
- OptimisticTransactionImpl
- val hasWritten: Boolean
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logConsole(line: String): Unit
- Definition Classes
- DatabricksLogging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Definition Classes
- OptimisticTransactionImpl → Logging
- def logError(msg: => String): Unit
- Definition Classes
- OptimisticTransactionImpl → Logging
- def logInfo(msg: => String): Unit
- Definition Classes
- OptimisticTransactionImpl → Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- lazy val logPrefix: String
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Definition Classes
- OptimisticTransactionImpl → Logging
- def logWarning(msg: => String): Unit
- Definition Classes
- OptimisticTransactionImpl → Logging
- def makeOutputNullable(output: Seq[Attribute]): Seq[Attribute]
Makes the output attributes nullable, so that we don't write unreadable parquet files.
Makes the output attributes nullable, so that we don't write unreadable parquet files.
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def mapColumnAttributes(output: Seq[Attribute], mappingMode: DeltaColumnMappingMode): Seq[Attribute]
Replace the output attributes with the physical mapping information.
Replace the output attributes with the physical mapping information.
- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def metadata: Metadata
Returns the metadata for this transaction.
Returns the metadata for this transaction. The metadata refers to the metadata of the snapshot at the transaction's read version unless updated during the transaction.
- Definition Classes
- OptimisticTransactionImpl → TransactionalWrite
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- val newMetadata: Option[Metadata]
Stores the updated metadata (if any) that will result from this txn.
Stores the updated metadata (if any) that will result from this txn.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val newProtocol: Option[Protocol]
Stores the updated protocol (if any) that will result from this txn.
Stores the updated protocol (if any) that will result from this txn.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def normalizeData(deltaLog: DeltaLog, data: Dataset[_]): (QueryExecution, Seq[Attribute], Seq[Constraint], Set[String])
Normalize the schema of the query, and return the QueryExecution to execute.
Normalize the schema of the query, and return the QueryExecution to execute. If the table has generated columns and users provide these columns in the output, we will also return constraints that should be respected. If any constraints are returned, the caller should apply these constraints when writing data.
Note: The output attributes of the QueryExecution may not match the attributes we return as the output schema. This is because streaming queries create
IncrementalExecution, which cannot be further modified. We can however have the Parquet writer use the physical plan fromIncrementalExecutionand the output schema provided through the attributes.- Attributes
- protected
- Definition Classes
- TransactionalWrite
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def performCDCPartition(inputData: Dataset[_]): (DataFrame, StructType)
Returns a tuple of (data, partition schema).
Returns a tuple of (data, partition schema). For CDC writes, a
is_cdcis added to the front of the partition schema.column is added to the data andis_cdc=true/false- Attributes
- protected
- Definition Classes
- TransactionalWrite
- def performCdcMetadataCheck(): Unit
Checks if the new schema contains any CDC columns (which is invalid) and throws the appropriate error
Checks if the new schema contains any CDC columns (which is invalid) and throws the appropriate error
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def postCommit(commitVersion: Long, needsCheckpoint: Boolean): Unit
Perform post-commit operations
Perform post-commit operations
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val postCommitHooks: ArrayBuffer[PostCommitHook]
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def prepareCommit(actions: Seq[Action], op: Operation): Seq[Action]
Prepare for a commit by doing all necessary pre-commit checks and modifications to the actions.
Prepare for a commit by doing all necessary pre-commit checks and modifications to the actions.
- returns
The finalized set of actions.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def protocol: Protocol
The protocol of the snapshot that this transaction is reading at.
The protocol of the snapshot that this transaction is reading at.
- Definition Classes
- OptimisticTransactionImpl → TransactionalWrite
- val readFiles: HashSet[AddFile]
Tracks specific files that have been seen by this transaction.
Tracks specific files that have been seen by this transaction.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val readPredicates: ArrayBuffer[Expression]
Tracks the data that could have been seen by recording the partition predicates by which files have been queried by this transaction.
Tracks the data that could have been seen by recording the partition predicates by which files have been queried by this transaction.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val readSnapshots: ConcurrentHashMap[(String, Path), Snapshot]
Tracks the first-access snapshots of other Delta logs read by this transaction.
Tracks the first-access snapshots of other Delta logs read by this transaction. The snapshots are keyed by the log's unique id.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val readTheWholeTable: Boolean
Whether the whole table was read during the transaction.
Whether the whole table was read during the transaction.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val readTxn: ArrayBuffer[String]
Tracks the appIds that have been seen by this transaction.
Tracks the appIds that have been seen by this transaction.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def readVersion: Long
The version that this transaction is reading from.
The version that this transaction is reading from.
- Definition Classes
- OptimisticTransactionImpl
- def readWholeTable(): Unit
Mark the entire table as tainted by this transaction.
Mark the entire table as tainted by this transaction.
- Definition Classes
- OptimisticTransactionImpl
- def recordDeltaEvent(deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty, data: AnyRef = null, path: Option[Path] = None): Unit
Used to record the occurrence of a single event or report detailed, operation specific statistics.
Used to record the occurrence of a single event or report detailed, operation specific statistics.
- path
Used to log the path of the delta table when
deltaLogis null.
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordDeltaOperation[A](deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: => A): A
Used to report the duration as well as the success or failure of an operation on a
deltaLog.Used to report the duration as well as the success or failure of an operation on a
deltaLog.- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordDeltaOperationForTablePath[A](tablePath: String, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: => A): A
Used to report the duration as well as the success or failure of an operation on a
tahoePath.Used to report the duration as well as the success or failure of an operation on a
tahoePath.- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordEvent(metric: MetricDefinition, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
- Definition Classes
- DatabricksLogging
- def recordFrameProfile[T](group: String, name: String)(thunk: => T): T
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def recordOperation[S](opType: OpType, opTarget: String = null, extraTags: Map[TagDefinition, String], isSynchronous: Boolean = true, alwaysRecordStats: Boolean = false, allowAuthTags: Boolean = false, killJvmIfStuck: Boolean = false, outputMetric: MetricDefinition = null, silent: Boolean = true)(thunk: => S): S
- Definition Classes
- DatabricksLogging
- def recordProductEvent(metric: MetricDefinition with CentralizableMetric, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
- Definition Classes
- DatabricksLogging
- def recordProductUsage(metric: MetricDefinition with CentralizableMetric, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
- Definition Classes
- DatabricksLogging
- def recordUsage(metric: MetricDefinition, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
- Definition Classes
- DatabricksLogging
- def registerPostCommitHook(hook: PostCommitHook): Unit
Register a hook that will be executed once a commit is successful.
Register a hook that will be executed once a commit is successful.
- Definition Classes
- OptimisticTransactionImpl
- def registerSQLMetrics(spark: SparkSession, metrics: Map[String, SQLMetric]): Unit
Register SQL metrics for an operation by appending the supplied metrics map to the operationSQLMetrics map.
Register SQL metrics for an operation by appending the supplied metrics map to the operationSQLMetrics map.
- Definition Classes
- SQLMetricsReporting
- def runPostCommitHooks(version: Long, committedActions: Seq[Action]): Unit
Executes the registered post commit hooks.
Executes the registered post commit hooks.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def shouldCheckpoint(committedVersion: Long): Boolean
Returns true if we should checkpoint the version that has just been committed.
Returns true if we should checkpoint the version that has just been committed.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- val snapshot: Snapshot
- Definition Classes
- OptimisticTransaction → OptimisticTransactionImpl → TransactionalWrite
- val snapshotToScan: Snapshot
The snapshot that the scan is being generated on.
The snapshot that the scan is being generated on.
- Definition Classes
- OptimisticTransactionImpl → DeltaScanGeneratorBase
- def spark: SparkSession
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def statsCollector: Column
Gets the stats collector for the table at the snapshot this transaction has.
Gets the stats collector for the table at the snapshot this transaction has.
- Definition Classes
- OptimisticTransactionImpl
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def txnExecutionTimeMs: Option[Long]
The end to end execution time of this transaction.
The end to end execution time of this transaction.
- Definition Classes
- OptimisticTransactionImpl
- val txnId: String
Unique identifier for the transaction
Unique identifier for the transaction
- Definition Classes
- OptimisticTransactionImpl
- val txnStartNano: Long
The transaction start time.
The transaction start time.
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def txnStartTimeNs: Long
Start time of txn in nanoseconds
Start time of txn in nanoseconds
- Definition Classes
- OptimisticTransactionImpl
- def txnVersion(id: String): Long
Returns the latest version that has committed for the idempotent transaction with given
id.Returns the latest version that has committed for the idempotent transaction with given
id.- Definition Classes
- OptimisticTransactionImpl
- def updateMetadata(_metadata: Metadata): Unit
Records an update to the metadata that should be committed with this transaction.
Records an update to the metadata that should be committed with this transaction. Note that this must be done before writing out any files so that file writing and checks happen with the final metadata for the table.
IMPORTANT: It is the responsibility of the caller to ensure that files currently present in the table are still valid under the new metadata.
- Definition Classes
- OptimisticTransactionImpl
- def updateMetadataForNewTable(metadata: Metadata): Unit
Records an update to the metadata that should be committed with this transaction and when this transaction is logically creating a new table, e.g.
Records an update to the metadata that should be committed with this transaction and when this transaction is logically creating a new table, e.g. replacing a previous table with new metadata. Note that this must be done before writing out any files so that file writing and checks happen with the final metadata for the table.
IMPORTANT: It is the responsibility of the caller to ensure that files currently present in the table are still valid under the new metadata.
- Definition Classes
- OptimisticTransactionImpl
- def updateMetadataInternal(_metadata: Metadata): Unit
Do the actual checks and works to update the metadata and save it into the
newMetadatafield, which will be added to the actions to commit in prepareCommit.Do the actual checks and works to update the metadata and save it into the
newMetadatafield, which will be added to the actions to commit in prepareCommit.- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- def verifyNewMetadata(metadata: Metadata): Unit
- Attributes
- protected
- Definition Classes
- OptimisticTransactionImpl
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- def withDmqTag[T](thunk: => T): T
- Attributes
- protected
- Definition Classes
- DeltaLogging
- def withFilesRead(files: Seq[AddFile]): Unit
Mark the given files as read within this transaction.
Mark the given files as read within this transaction.
- Definition Classes
- OptimisticTransactionImpl
- def withStatusCode[T](statusCode: String, defaultMessage: String, data: Map[String, Any] = Map.empty)(body: => T): T
Report a log to indicate some command is running.
Report a log to indicate some command is running.
- Definition Classes
- DeltaProgressReporter
- def writeFiles(inputData: Dataset[_], writeOptions: Option[DeltaOptions], additionalConstraints: Seq[Constraint]): Seq[FileAction]
Writes out the dataframe after performing schema validation.
Writes out the dataframe after performing schema validation. Returns a list of actions to append these files to the reservoir.
- Definition Classes
- TransactionalWrite
- def writeFiles(data: Dataset[_]): Seq[FileAction]
- Definition Classes
- TransactionalWrite
- def writeFiles(data: Dataset[_], writeOptions: Option[DeltaOptions]): Seq[FileAction]
- Definition Classes
- TransactionalWrite
- def writeFiles(data: Dataset[_], additionalConstraints: Seq[Constraint]): Seq[FileAction]
- Definition Classes
- TransactionalWrite