Package io.confluent.parallelconsumer

Class ParallelConsumerOptions<K,V>

java.lang.Object

io.confluent.parallelconsumer.ParallelConsumerOptions<K,V>

public class ParallelConsumerOptions<K,V>
extends java.lang.Object

The options for the ParallelEoSStreamProcessor system.

See Also:

builder(), ParallelConsumerOptions.ParallelConsumerOptionsBuilder

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ParallelConsumerOptions.CommitMode	The type of commit to be made, with either a transactions configured Producer where messages produced are committed back to the Broker along with the offsets they originated from, or with the faster simpler Consumer offset system either synchronously or asynchronously
static class	ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>
static class	ParallelConsumerOptions.ProcessingOrder	The ordering guarantee to use.

Method Summary

Modifier and Type	Method	Description
static <K, V> ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>	builder()
ParallelConsumerOptions.CommitMode	getCommitMode()	The ParallelConsumerOptions.CommitMode to be used
org.apache.kafka.clients.consumer.Consumer<K,V>	getConsumer()	Required parameter for all use.
java.time.Duration	getDefaultMessageRetryDelay()	When a message fails, how long the system should wait before trying that message again.
int	getMaxConcurrency()	Controls the maximum degree of concurrency to occur.
ParallelConsumerOptions.ProcessingOrder	getOrdering()	The ParallelConsumerOptions.ProcessingOrder type to use
org.apache.kafka.clients.producer.Producer<K,V>	getProducer()	Supplying a producer is only needed if using the produce flows.
boolean	isProducerSupplied()
protected boolean	isUsingTransactionalProducer()
ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>	toBuilder()
java.lang.String	toString()
void	validate()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

validate

public void validate()

isUsingTransactionalProducer

protected boolean isUsingTransactionalProducer()

isProducerSupplied

public boolean isProducerSupplied()

builder

public static <K, V> ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> builder()

toBuilder

public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> toBuilder()

getConsumer

public org.apache.kafka.clients.consumer.Consumer<K,V> getConsumer()

Required parameter for all use.

getProducer

public org.apache.kafka.clients.producer.Producer<K,V> getProducer()

Supplying a producer is only needed if using the produce flows.

See Also:

ParallelStreamProcessor

getOrdering

public ParallelConsumerOptions.ProcessingOrder getOrdering()

The ParallelConsumerOptions.ProcessingOrder type to use

getCommitMode

public ParallelConsumerOptions.CommitMode getCommitMode()

The ParallelConsumerOptions.CommitMode to be used

getMaxConcurrency

public int getMaxConcurrency()

Controls the maximum degree of concurrency to occur. Used to limit concurrent calls to external systems to a maximum to prevent overloading them or to a degree, using up quotas.

A note on quotas - if your quota is expressed as maximum concurrent calls, this works well. If it's limited in total requests / sec, this may still overload the system. See towards the distributed rate limiting feature for this to be properly addressed: https://github.com/confluentinc/parallel-consumer/issues/24 Add distributed rate limiting support #24.

In the core module, this sets the number of threads to use in the core's thread pool.

It's recommended to set this quite high, much higher than core count, as it's expected that these threads will spend most of their time blocked waiting for IO. For automatic setting of this variable, look out for issue https://github.com/confluentinc/parallel-consumer/issues/21 Dynamic concurrency control with flow control or tcp congestion control theory #21.

getDefaultMessageRetryDelay

public java.time.Duration getDefaultMessageRetryDelay()

When a message fails, how long the system should wait before trying that message again.

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object