Type Members

1.  sealed trait Action extends AnyRef

   Represents a single change to the state of a Delta table.

   Represents a single change to the state of a Delta table. An order sequence of actions can be replayed using InMemoryLogReplay to derive the state of the table at a given point in time.

2.  case class AddCDCFile(path: String, partitionValues: Map[String, String], size: Long, tags: Map[String, String] = null) extends FileAction with Product with Serializable

   A change file containing CDC data for the Delta version it's within.

   A change file containing CDC data for the Delta version it's within. Non-CDC readers should ignore this, CDC readers should scan all ChangeFiles in a version rather than computing changes from AddFile and RemoveFile actions.

3.  case class AddFile(path: String, partitionValues: Map[String, String], size: Long, modificationTime: Long, dataChange: Boolean, stats: String = null, tags: Map[String, String] = null) extends FileAction with Product with Serializable

   Adds a new file to the table.

   Adds a new file to the table. When multiple AddFile file actions are seen with the same path only the metadata from the last one is kept.

4.  case class CommitInfo(version: Option[Long], timestamp: Timestamp, userId: Option[String], userName: Option[String], operation: String, operationParameters: Map[String, String], job: Option[JobInfo], notebook: Option[NotebookInfo], clusterId: Option[String], readVersion: Option[Long], isolationLevel: Option[String], isBlindAppend: Option[Boolean], operationMetrics: Option[Map[String, String]], userMetadata: Option[String], tags: Option[Map[String, String]], engineInfo: Option[String], txnId: Option[String]) extends Action with CommitMarker with Product with Serializable

   Holds provenance information about changes to the table.

   Holds provenance information about changes to the table. This Action is not stored in the checkpoint and has reduced compatibility guarantees. Information stored in it is best effort (i.e. can be falsified by the writer).

   isBlindAppend

   Whether this commit has blindly appended without caring about existing files

   engineInfo

   The information for the engine that makes the commit. If a commit is made by Delta Lake 1.1.0 or above, it will be Apache-Spark/x.y.z Delta-Lake/x.y.z.

5.  trait CommitMarker extends AnyRef

   Interface for objects that represents the information for a commit.

   Interface for objects that represents the information for a commit. Commits can be referred to using a version and timestamp. The timestamp of a commit comes from the remote storage lastModifiedTime, and can be adjusted for clock skew. Hence we have the method withTimestamp.

6.  sealed trait FileAction extends Action

   Actions pertaining to the addition and removal of files.

7.  case class Format(provider: String = "parquet", options: Map[String, String] = Map.empty) extends Product with Serializable
8.  class InMemoryLogReplay extends LogReplay

   Replays a history of action, resolving them to produce the current state of the table.

   Replays a history of action, resolving them to produce the current state of the table. The protocol for resolution is as follows:

   • The most recent AddFile and accompanying metadata for any path wins.
   • RemoveFile deletes a corresponding AddFile and is retained as a tombstone until minFileRetentionTimestamp has passed.
   • The most recent version for any appId in a SetTransaction wins.
   • The most recent Metadata wins.
   • The most recent Protocol version wins.
   • For each path, this class should always output only one FileAction (either AddFile or RemoveFile)

   This class is not thread safe.

9.  class InvalidProtocolVersionException extends RuntimeException

   Thrown when the protocol version of a table is greater than supported by this client.

10.  case class JobInfo(jobId: String, jobName: String, runId: String, jobOwnerId: String, triggerType: String) extends Product with Serializable
11.  class JsonMapSerializer extends JsonSerializer[Map[String, String]]

    Serializes Maps containing JSON strings without extra escaping.

12.  trait LogReplay extends AnyRef

    Replays a history of actions, resolving them to produce the current state of the table.

13.  case class Metadata(id: String = if (Utils.isTesting) "testId" else java.util.UUID.randomUUID().toString, name: String = null, description: String = null, format: Format = Format(), schemaString: String = null, partitionColumns: Seq[String] = Nil, configuration: Map[String, String] = Map.empty, createdTime: Option[Long] = None) extends Action with Product with Serializable

    Updates the metadata of the table.

    Updates the metadata of the table. Only the last update to the Metadata of a table is kept. It is the responsibility of the writer to ensure that any data already present in the table is still valid after any change.

14.  case class NotebookInfo(notebookId: String) extends Product with Serializable
15.  case class Protocol(minReaderVersion: Int = Action.readerVersion, minWriterVersion: Int = Action.writerVersion) extends Action with Product with Serializable

    Used to block older clients from reading or writing the log when backwards incompatible changes are made to the protocol.

    Used to block older clients from reading or writing the log when backwards incompatible changes are made to the protocol. Readers and writers are responsible for checking that they meet the minimum versions before performing any other operations.

    Since this action allows us to explicitly block older clients in the case of a breaking change to the protocol, clients should be tolerant of messages and fields that they do not understand.

16.  class ProtocolDowngradeException extends RuntimeException with DeltaThrowable
17.  case class RemoveFile(path: String, deletionTimestamp: Option[Long], dataChange: Boolean = true, extendedFileMetadata: Option[Boolean] = None, partitionValues: Map[String, String] = null, size: Option[Long] = None, tags: Map[String, String] = null) extends FileAction with Product with Serializable

    Logical removal of a given file from the reservoir.

    Logical removal of a given file from the reservoir. Acts as a tombstone before a file is deleted permanently.

    Note that for protocol compatibility reasons, the fields partitionValues, size, and tags are only present when the extendedFileMetadata flag is true. New writers should generally be setting this flag, but old writers (and FSCK) won't, so readers must check this flag before attempting to consume those values.

    Since old tables would not have extendedFileMetadata and size field, we should make them nullable by setting their type Option.

18.  case class SetTransaction(appId: String, version: Long, lastUpdated: Option[Long]) extends Action with Product with Serializable

    Sets the committed version for a given application.

    Sets the committed version for a given application. Used to make operations like streaming append idempotent.

19.  case class SingleAction(txn: SetTransaction = null, add: AddFile = null, remove: RemoveFile = null, metaData: Metadata = null, protocol: Protocol = null, cdc: AddCDCFile = null, commitInfo: CommitInfo = null) extends Product with Serializable

    A serialization helper to create a common action envelope.