io.qdrant.spark

Class QdrantStreamingWriter

java.lang.Object

io.qdrant.spark.QdrantStreamingWriter

All Implemented Interfaces:

org.apache.spark.sql.connector.write.streaming.StreamingWrite

public class QdrantStreamingWriter
extends Object
implements org.apache.spark.sql.connector.write.streaming.StreamingWrite

A class that implements the Spark StreamingWrite interface for writing data to Qdrant.

Constructor Summary

Constructors

Constructor and Description
QdrantStreamingWriter(QdrantOptions options, org.apache.spark.sql.types.StructType schema) Constructor for QdrantStreamingWriter.

Method Summary

Modifier and Type	Method and Description
void	abort(long epochId, org.apache.spark.sql.connector.write.WriterCommitMessage[] messages) Aborts the write operation for the given epochId and messages.
void	commit(long epochId, org.apache.spark.sql.connector.write.WriterCommitMessage[] messages) Commits the write operation for the given epochId and messages.
org.apache.spark.sql.connector.write.streaming.StreamingDataWriterFactory	createStreamingWriterFactory(org.apache.spark.sql.connector.write.PhysicalWriteInfo info) Creates a StreamingDataWriterFactory for writing data to Qdrant.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.write.streaming.StreamingWrite

useCommitCoordinator

Constructor Detail

QdrantStreamingWriter

public QdrantStreamingWriter(QdrantOptions options,
                             org.apache.spark.sql.types.StructType schema)

Constructor for QdrantStreamingWriter.

Parameters:

options - The options for writing data to Qdrant.

schema - The schema of the data to be written.

Method Detail

createStreamingWriterFactory

public org.apache.spark.sql.connector.write.streaming.StreamingDataWriterFactory createStreamingWriterFactory(org.apache.spark.sql.connector.write.PhysicalWriteInfo info)

Creates a StreamingDataWriterFactory for writing data to Qdrant.

Specified by:

createStreamingWriterFactory in interface org.apache.spark.sql.connector.write.streaming.StreamingWrite

Parameters:

info - The PhysicalWriteInfo object containing information about the write operation.

Returns:

A QdrantDataWriterFactory object for writing data to Qdrant.

commit

public void commit(long epochId,
                   org.apache.spark.sql.connector.write.WriterCommitMessage[] messages)

Commits the write operation for the given epochId and messages.

Specified by:

commit in interface org.apache.spark.sql.connector.write.streaming.StreamingWrite

Parameters:

epochId - The epochId of the write operation.

messages - The WriterCommitMessage objects containing information about the write operation.

abort

public void abort(long epochId,
                  org.apache.spark.sql.connector.write.WriterCommitMessage[] messages)

Aborts the write operation for the given epochId and messages.

Specified by:

abort in interface org.apache.spark.sql.connector.write.streaming.StreamingWrite

Parameters:

epochId - The epochId of the write operation.

messages - The WriterCommitMessage objects containing information about the write operation.