io.confluent.parallelconsumer.ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>

Enclosing class:: ParallelConsumerOptions<K,V>

public static class ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>
extends Object

Method Summary

Modifier and Type	Method	Description
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`batchSize(Integer batchSize)`	The maximum number of messages to attempt to pass into the user functions.
`ParallelConsumerOptions<K,V>`	`build()`
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`commitMode(ParallelConsumerOptions.CommitMode commitMode)`	The `ParallelConsumerOptions.CommitMode` to be used
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`consumer(org.apache.kafka.clients.consumer.Consumer<K,V> consumer)`	Required parameter for all use.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`defaultMessageRetryDelay(Duration defaultMessageRetryDelay)`	When a message fails, how long the system should wait before trying that message again.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`managedExecutorService(String managedExecutorService)`	Path to Managed executor service for Java EE
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`managedThreadFactory(String managedThreadFactory)`	Path to Managed thread factory for Java EE
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`maxConcurrency(int maxConcurrency)`	Controls the maximum degree of concurrency to occur.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`maxFailureHistory(int maxFailureHistory)`
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`offsetCommitTimeout(Duration offsetCommitTimeout)`	Controls how long to block while waiting for offsets to be committed.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`ordering(ParallelConsumerOptions.ProcessingOrder ordering)`	The `ParallelConsumerOptions.ProcessingOrder` type to use
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`producer(org.apache.kafka.clients.producer.Producer<K,V> producer)`	Supplying a producer is only needed if using the produce flows.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`retryDelayProvider(Function<RecordContext<K,V>,Duration> retryDelayProvider)`	When present, use this to generate the retry delay, instead of `ParallelConsumerOptions.getDefaultMessageRetryDelay()`.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`sendTimeout(Duration sendTimeout)`	Controls how long to block while waiting for the `Producer.send(org.apache.kafka.clients.producer.ProducerRecord<K, V>)` to complete for any ProducerRecords returned from the user-function.
`ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>`	`thresholdForTimeSpendInQueueWarning(Duration thresholdForTimeSpendInQueueWarning)`	Configure the amount of delay a record experiences, before a warning is logged.
`String`	`toString()`

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details
- consumer
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> consumer(org.apache.kafka.clients.consumer.Consumer<K,V> consumer)
  
  Required parameter for all use.
  
  Returns:
  
  this.
- producer
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> producer(org.apache.kafka.clients.producer.Producer<K,V> producer)
  
  Supplying a producer is only needed if using the produce flows.
  
  Returns:
  
  this.
  
  See Also:
  
  ParallelStreamProcessor
- managedExecutorService
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> managedExecutorService(String managedExecutorService)
  
  Path to Managed executor service for Java EE
  
  Returns:
  
  this.
- managedThreadFactory
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> managedThreadFactory(String managedThreadFactory)
  
  Path to Managed thread factory for Java EE
  
  Returns:
  
  this.
- ordering
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> ordering(ParallelConsumerOptions.ProcessingOrder ordering)
  
  The ParallelConsumerOptions.ProcessingOrder type to use
  
  Returns:
  
  this.
- commitMode
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> commitMode(ParallelConsumerOptions.CommitMode commitMode)
  
  The ParallelConsumerOptions.CommitMode to be used
  
  Returns:
  
  this.
- maxConcurrency
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> maxConcurrency(int maxConcurrency)
  
  Controls the maximum degree of concurrency to occur. Used to limit concurrent calls to external systems to a maximum to prevent overloading them or to a degree, using up quotas.
  When using ParallelConsumerOptions.getBatchSize(), this is over and above the batch size setting. So for example, a ParallelConsumerOptions.getMaxConcurrency() of 2 and a batch size of 3 would result in at most 15 records being processed at once.
  A note on quotas - if your quota is expressed as maximum concurrent calls, this works well. If it's limited in total requests / sec, this may still overload the system. See towards the distributed rate limiting feature for this to be properly addressed: https://github.com/confluentinc/parallel-consumer/issues/24 Add distributed rate limiting support #24.
  In the core module, this sets the number of threads to use in the core's thread pool.
  It's recommended to set this quite high, much higher than core count, as it's expected that these threads will spend most of their time blocked waiting for IO. For automatic setting of this variable, look out for issue https://github.com/confluentinc/parallel-consumer/issues/21 Dynamic concurrency control with flow control or tcp congestion control theory #21.
  
  Returns:
  
  this.
- defaultMessageRetryDelay
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> defaultMessageRetryDelay(Duration defaultMessageRetryDelay)
  
  When a message fails, how long the system should wait before trying that message again. Note that this will not be exact, and is just a target.
  
  Returns:
  
  this.
- retryDelayProvider
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> retryDelayProvider(Function<RecordContext<K,V>,Duration> retryDelayProvider)
  
  When present, use this to generate the retry delay, instead of ParallelConsumerOptions.getDefaultMessageRetryDelay().
  Overrides defaultMessageRetryDelay(java.time.Duration), even if it's set.
  
  Returns:
  
  this.
- sendTimeout
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> sendTimeout(Duration sendTimeout)
  
  Controls how long to block while waiting for the Producer.send(org.apache.kafka.clients.producer.ProducerRecord<K, V>) to complete for any ProducerRecords returned from the user-function. Only relevant if using one of the produce-flows and providing a ParallelConsumerOptions.producer. If the timeout occurs the record will be re-processed in the user-function.
  Consider aligning the value with the ParallelConsumerOptions.producer-options to avoid unnecessary re-processing and duplicates on slow Producer.send(org.apache.kafka.clients.producer.ProducerRecord<K, V>) calls.
  
  Returns:
  
  this.
  
  See Also:
  
  ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG
- offsetCommitTimeout
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> offsetCommitTimeout(Duration offsetCommitTimeout)
  
  Controls how long to block while waiting for offsets to be committed. Only relevant if using ParallelConsumerOptions.CommitMode.PERIODIC_CONSUMER_SYNC commit-mode.
  
  Returns:
  
  this.
- batchSize
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> batchSize(Integer batchSize)
  
  The maximum number of messages to attempt to pass into the user functions.
  Batch sizes may sometimes be less than this size, but will never be more.
  The system will treat the messages as a set, so if an error is thrown by the user code, then all messages will be marked as failed and be retried (Note that when they are retried, there is no guarantee they will all be in the same batch again). So if you're going to process messages individually, then don't set a batch size.
  Otherwise, if you're going to process messages in sub sets from this batch, it's better to instead adjust the ParallelConsumerOptions.getBatchSize() instead to the actual desired size, and process them as a whole.
  Note that there is no relationship between the ConsumerConfig setting of ConsumerConfig.MAX_POLL_RECORDS_CONFIG and this configured batch size, as this library introduces a large layer of indirection between the managed consumer, and the managed queues we use.
  This indirection effectively disconnects the processing of messages from "polling" them from the managed client, as we do not wait to process them before calling poll again. We simply call poll as much as we need to, in order to keep our queues full of enough work to satisfy demand.
  If we have enough, then we actively manage pausing our subscription so that we can continue calling poll without pulling in even more messages.
  
  Returns:
  
  this.
  
  See Also:
  
  ParallelConsumerOptions.getBatchSize()
- thresholdForTimeSpendInQueueWarning
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> thresholdForTimeSpendInQueueWarning(Duration thresholdForTimeSpendInQueueWarning)
  
  Configure the amount of delay a record experiences, before a warning is logged.
  
  Returns:
  
  this.
- maxFailureHistory
  
  public ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V> maxFailureHistory(int maxFailureHistory)
  
  Returns:
  
  this.
- build
  
  public ParallelConsumerOptions<K,V> build()
- toString
  
  public String toString()
  
  Overrides:
  
  toString in class Object

Class ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,​V>

Method Summary

Methods inherited from class java.lang.Object

Method Details

Class ParallelConsumerOptions.ParallelConsumerOptionsBuilder<K,V>