io.qdrant.spark

Class QdrantDataWriter

java.lang.Object

io.qdrant.spark.QdrantDataWriter

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, org.apache.spark.sql.connector.write.DataWriter<org.apache.spark.sql.catalyst.InternalRow>

public class QdrantDataWriter
extends Object
implements org.apache.spark.sql.connector.write.DataWriter<org.apache.spark.sql.catalyst.InternalRow>, Serializable

A DataWriter implementation that writes data to Qdrant, a vector search engine. This class takes QdrantOptions and StructType as input and writes data to QdrantGRPC. It implements the DataWriter interface and overrides its methods write, commit, abort and close. It also has a private method write that is used to upload a batch of points to Qdrant. The class uses a Point class to represent a data point and an ArrayList to store the points.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
QdrantDataWriter(QdrantOptions options, org.apache.spark.sql.types.StructType schema)

Method Summary

Modifier and Type	Method and Description
void	abort()
void	close()
org.apache.spark.sql.connector.write.WriterCommitMessage	commit()
void	write(int retries)
void	write(org.apache.spark.sql.catalyst.InternalRow record)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.write.DataWriter

currentMetricsValues

Constructor Detail

QdrantDataWriter

public QdrantDataWriter(QdrantOptions options,
                        org.apache.spark.sql.types.StructType schema)

Method Detail

write

public void write(org.apache.spark.sql.catalyst.InternalRow record)

Specified by:

write in interface org.apache.spark.sql.connector.write.DataWriter<org.apache.spark.sql.catalyst.InternalRow>

commit

public org.apache.spark.sql.connector.write.WriterCommitMessage commit()

Specified by:

commit in interface org.apache.spark.sql.connector.write.DataWriter<org.apache.spark.sql.catalyst.InternalRow>

write

public void write(int retries)

abort

public void abort()

Specified by:

abort in interface org.apache.spark.sql.connector.write.DataWriter<org.apache.spark.sql.catalyst.InternalRow>

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable