Type Members

1.  trait BaseResponseCallback extends AnyRef

   A basic callback.

   A basic callback. This is extended by RpcResponseCallback and MergedBlockMetaResponseCallback so that both RpcRequests and MergedBlockMetaRequests can be handled in TransportResponseHandler a similar way.

   Since

   3.2.0

2.  class ChunkFetchFailureException extends RuntimeException

   General exception caused by a remote exception while fetching a chunk.

3.  trait ChunkReceivedCallback extends AnyRef

   Callback for the result of a single chunk result.

   Callback for the result of a single chunk result. For a single stream, the callbacks are guaranteed to be called by the same thread in the same order as the requests for chunks were made.

   Note that if a general stream failure occurs, all outstanding chunk requests may be failed.

4.  trait MergedBlockMetaResponseCallback extends BaseResponseCallback

   Callback for the result of a single org.apache.spark.network.protocol.MergedBlockMetaRequest.

   Callback for the result of a single org.apache.spark.network.protocol.MergedBlockMetaRequest.

   Since

   3.2.0

5.  trait RpcResponseCallback extends BaseResponseCallback

   Callback for the result of a single RPC.

   Callback for the result of a single RPC. This will be invoked once with either success or failure.

6.  trait StreamCallback extends AnyRef

   Callback for streaming data.

   Callback for streaming data. Stream data will be offered to the ByteBuffer) method as it arrives. Once all the stream data is received, #onComplete(String) will be called.

   The network library guarantees that a single thread will call these methods at a time, but different call may be made by different threads.

7.  trait StreamCallbackWithID extends StreamCallback
8.  class StreamInterceptor[T <: Message] extends Interceptor

   An interceptor that is registered with the frame decoder to feed stream data to a callback.

9.  class TransportClient extends Closeable

   Client for fetching consecutive chunks of a pre-negotiated stream.

   Client for fetching consecutive chunks of a pre-negotiated stream. This API is intended to allow efficient transfer of a large amount of data, broken up into chunks with size ranging from hundreds of KB to a few MB.

   Note that while this client deals with the fetching of chunks from a stream (i.e., data plane), the actual setup of the streams is done outside the scope of the transport layer. The convenience method "sendRPC" is provided to enable control plane communication between the client and server to perform this setup.

   For example, a typical workflow might be: client.sendRPC(new OpenFile("/foo")) --> returns StreamId = 100 client.fetchChunk(streamId = 100, chunkIndex = 0, callback) client.fetchChunk(streamId = 100, chunkIndex = 1, callback) ... client.sendRPC(new CloseStream(100))

   Construct an instance of TransportClient using TransportClientFactory. A single TransportClient may be used for multiple streams, but any given stream must be restricted to a single client, in order to avoid out-of-order responses.

   NB: This class is used to make requests to the server, while TransportResponseHandler is responsible for handling responses from the server.

   Concurrency: thread safe and can be called from multiple threads.

10.  trait TransportClientBootstrap extends AnyRef

    A bootstrap which is executed on a TransportClient before it is returned to the user.

    A bootstrap which is executed on a TransportClient before it is returned to the user. This enables an initial exchange of information (e.g., SASL authentication tokens) on a once-per- connection basis.

    Since connections (and TransportClients) are reused as much as possible, it is generally reasonable to perform an expensive bootstrapping operation, as they often share a lifespan with the JVM itself.

11.  class TransportClientFactory extends Closeable

    Factory for creating TransportClients by using createClient.

    Factory for creating TransportClients by using createClient.

    The factory maintains a connection pool to other hosts and should return the same TransportClient for the same remote host. It also shares a single worker thread pool for all TransportClients.

    TransportClients will be reused whenever possible. Prior to completing the creation of a new TransportClient, all given TransportClientBootstraps will be run.

12.  class TransportResponseHandler extends MessageHandler[ResponseMessage]

    Handler that processes server responses, in response to requests issued from a TransportClient.

    Handler that processes server responses, in response to requests issued from a TransportClient. It works by tracking the list of outstanding requests (and their callbacks).

    Concurrency: thread safe and can be called from multiple threads.