Type Members

1.  class AbortableStreamBasedCheckpointFileManager extends AbstractFileContextBasedCheckpointFileManager with Logging
2.  class BindingParquetOutputCommitter extends ParquetOutputCommitter with Logging with StreamCapabilities

   This Parquet Committer subclass dynamically binds to the factory-configured output committer, and is intended to allow callers to use any 'PathOutputCommitter', even if not a subclass of 'ParquetOutputCommitter'.

   This Parquet Committer subclass dynamically binds to the factory-configured output committer, and is intended to allow callers to use any 'PathOutputCommitter', even if not a subclass of 'ParquetOutputCommitter'.

   The Parquet parquet.enable.summary-metadata option will only be supported if the instantiated committer itself supports it.

3.  class PathOutputCommitProtocol extends HadoopMapReduceCommitProtocol with Serializable

   Spark Commit protocol for Path Output Committers.

   Spark Commit protocol for Path Output Committers. This committer will work with the FileOutputCommitter and subclasses. All implementations *must* be serializable.

   Rather than ask the FileOutputFormat for a committer, it uses the org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory factory API to create the committer.

   In setupCommitter the factory is identified and instantiated; this factory then creates the actual committer implementation.

   Dynamic Partition support will be determined once the committer is instantiated in the setupJob/setupTask methods. If this class was instantiated with dynamicPartitionOverwrite set to true, then the instantiated committer must either be an instance of FileOutputCommitter or it must implement the StreamCapabilities interface and declare that it has the capability mapreduce.job.committer.dynamic.partitioning. That feature is available on Hadoop releases with the Intermediate Manifest Committer for GCS and ABFS; it is not supported by the S3A committers.