Type Members

1.  class CdcAddFileIndex extends TahoeBatchFileIndex

   A TahoeFileIndex for scanning a sequence of added files as CDC.

   A TahoeFileIndex for scanning a sequence of added files as CDC. Similar to TahoeBatchFileIndex, with a bit of special handling to attach the log version and CDC type on a per-file basis.

2.  class DelayedCommitProtocol extends FileCommitProtocol with Serializable with Logging

   Writes out the files to path and returns a list of them in addedStatuses.

   Writes out the files to path and returns a list of them in addedStatuses. Includes special handling for partitioning on CDC_PARTITION_COL for compatibility between enabled and disabled CDC; partitions with a value of false in this column produce no corresponding partitioning directory.

3.  class DeltaSourceSnapshot extends SnapshotIterator with StateCache

   Converts a Snapshot into the initial set of files read when starting a new streaming query.

   Converts a Snapshot into the initial set of files read when starting a new streaming query. The list of files that represent the table at the time the query starts are selected by: - Adding version and index to each file to enable splitting of the initial state into multiple batches. - Filtering files that don't match partition predicates, while preserving the aforementioned indexing.

4.  case class PinnedTahoeFileIndex(spark: SparkSession, deltaLog: DeltaLog, path: Path, snapshot: Snapshot) extends TahoeFileIndex with Product with Serializable

   A TahoeFileIndex that generates the list of files from the given Snapshot.

5.  trait SQLMetricsReporting extends AnyRef

   This trait is used to register SQL metrics for a Delta Operation.

   This trait is used to register SQL metrics for a Delta Operation. Registering will allow the metrics to be instrumented via the CommitInfo and is accessible via DescribeHistory

6.  trait SnapshotIterator extends AnyRef
7.  class TahoeBatchFileIndex extends TahoeFileIndex

   A TahoeFileIndex that generates the list of files from a given list of files that are within a version range of DeltaLog.

8.  class TahoeChangeFileIndex extends TahoeFileIndex

   A TahoeFileIndex for scanning a sequence of CDC files.

   A TahoeFileIndex for scanning a sequence of CDC files. Similar to TahoeBatchFileIndex, the equivalent for reading AddFile actions.

9.  abstract class TahoeFileIndex extends FileIndex

   A FileIndex that generates the list of files managed by the Tahoe protocol.

10.  case class TahoeLogFileIndex(spark: SparkSession, deltaLog: DeltaLog, path: Path, snapshotAtAnalysis: Snapshot, partitionFilters: Seq[Expression] = Nil, isTimeTravelQuery: Boolean = false) extends TahoeFileIndex with Product with Serializable

    A TahoeFileIndex that generates the list of files from DeltaLog with given partition filters.

11.  class TahoeRemoveFileIndex extends TahoeFileIndex

    A TahoeFileIndex for scanning a sequence of removed files as CDC.

    A TahoeFileIndex for scanning a sequence of removed files as CDC. Similar to TahoeBatchFileIndex, the equivalent for reading AddFile actions.

12.  trait TransactionalWrite extends DeltaLogging

    Adds the ability to write files out as part of a transaction.

    Adds the ability to write files out as part of a transaction. Checks are performed to ensure that the data being written matches either the current metadata or the new metadata being set by this transaction.