Package org.apache.pulsar.broker.service.persistent

Class PersistentStickyKeyDispatcherMultipleConsumers

java.lang.Object

org.apache.pulsar.broker.service.EntryFilterSupport

org.apache.pulsar.broker.service.AbstractBaseDispatcher

org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.AbstractPersistentDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers

All Implemented Interfaces:

AsyncCallbacks.ReadEntriesCallback, Dispatcher, StickyKeyDispatcher

public class PersistentStickyKeyDispatcherMultipleConsumers
extends PersistentDispatcherMultipleConsumers
implements StickyKeyDispatcher

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers

PersistentDispatcherMultipleConsumers.ReadType

Field Summary

Fields inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers

BLOCKED_DISPATCHER_ON_CURSOR_DATA_CAN_NOT_FULLY_PERSIST_UPDATER, BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER, cursor, dispatchMessagesThread, dispatchRateLimiter, havePendingRead, havePendingReplayRead, lastIndividualDeletedRangeFromCursorRecovery, lastNumberOfEntriesProcessed, minReplayedPosition, name, readBatchSize, readFailureBackoff, redeliveryMessages, redeliveryTracker, shouldRewindBeforeReadingOrReplaying, skipNextBackoff, topic, TOTAL_AVAILABLE_PERMITS_UPDATER, totalAvailablePermits, totalUnackedMessages

Fields inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

consumerList, consumerSet, currentConsumerRoundRobinIndex, FALSE, IS_CLOSED_UPDATER, TRUE

Fields inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

dispatchThrottlingOnBatchMessageEnabled, serviceConfig

Fields inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

entryFilters, filterContext, hasFilter, subscription

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Void>	addConsumer(Consumer consumer)
protected Set<? extends Position>	asyncReplayEntries(Set<? extends Position> positions)
protected boolean	canReplayMessages()	The dispatcher will skip replaying messages when all messages in the replay queue are filtered out when skipNextReplayToTriggerLookAhead=true.
protected void	clearComponentsAfterRemovedAllConsumers()
protected Predicate<Position>	createFilterForReplay()	Creates a filter for replaying messages.
protected boolean	doesntHavePendingRead()	For Key_Shared subscription, the dispatcher will not read more entries while there are pending reads or pending replay reads.
Map<Consumer,List<org.apache.pulsar.client.api.Range>>	getConsumerKeyHashRanges()
org.apache.pulsar.common.api.proto.KeySharedMode	getKeySharedMode()
protected int	getMaxEntriesReadLimit()	Sets a hard limit on the number of entries to read from the Managed Ledger.
StickyKeyConsumerSelector	getSelector()
protected int	getStickyKeyHash(Entry entry)
org.apache.pulsar.common.api.proto.CommandSubscribe.SubType	getType()
protected void	handleNormalReadNotAllowed()	When a normal read is not allowed, the dispatcher will reschedule a read with a backoff.
boolean	hasSameKeySharedPolicy(org.apache.pulsar.common.api.proto.KeySharedMeta ksm)
boolean	isAllowOutOfOrderDelivery()
protected boolean	isNormalReadAllowed()	For Key_Shared subscription, the dispatcher will not attempt to read more entries if the replay queue size has reached the limit or if there are no consumers with permits.
void	markDeletePositionMoveForward()
void	removeConsumer(Consumer consumer)
protected boolean	trySendMessagesToConsumers(PersistentDispatcherMultipleConsumers.ReadType readType, List<Entry> entries)	Dispatch the messages to the Consumers.

Methods inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers

acquireSendInProgress, addEntryToReplay, addMessageToReplay, addMessageToReplay, addUnAckedMessages, afterAckMessages, asyncReplayEntriesInOrder, blockDispatcherOnUnackedMsgs, calculateToRead, cancelPendingRead, canUnsubscribe, checkAndResumeIfPaused, checkAndUnblockIfStuck, clearDelayedMessages, close, consumerFlow, createReadEntriesSkipConditionForNormalRead, cursorIsReset, disconnectActiveConsumers, disconnectAllConsumers, getBucketDelayedIndexStats, getConsumers, getCursor, getDelayedTrackerMemoryUsage, getFirstAvailableConsumerPermits, getFirstPositionInReplay, getMessagesToReplayNow, getName, getNumberOfDelayedMessages, getNumberOfMessagesInReplay, getRateLimiter, getReadMoreEntriesCallCount, getRedeliveryTracker, getSubscription, getTopic, getTotalUnackedMessages, initializeDispatchRateLimiterIfNeeded, internalRemoveConsumer, isAtleastOneConsumerAvailable, isBlockedDispatcherOnUnackedMsgs, isClassic, isConsumerAvailable, isConsumerConnected, isConsumersExceededOnSubscription, isHavePendingRead, isHavePendingReplayRead, isSendInProgress, readEntriesComplete, readEntriesFailed, readMoreEntries, readMoreEntriesAsync, redeliverUnacknowledgedMessages, redeliverUnacknowledgedMessages, releaseSendInProgress, reScheduleRead, reScheduleReadInMs, reScheduleReadWithBackoff, reset, resetCloseFuture, sendMessagesToConsumers, shouldPauseDeliveryForDelayTracker, trackDelayedDelivery, unBlockDispatcherOnUnackedMsgs

Methods inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

getNextConsumer, getRandomConsumer, isClosed

Methods inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

acquirePermitsForDeliveredMessages, checkAndApplyReachedEndOfTopicOrTopicMigration, checkAndApplyReachedEndOfTopicOrTopicMigration, computeReadLimits, filterEntriesForConsumer, filterEntriesForConsumer, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, getSubscriptionName, isConsumersExceededOnSubscription, peekStickyKey, updateMessagesToRead, updatePendingBytesToDispatch

Methods inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

runFiltersForEntry

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.broker.service.Dispatcher

addUnAckedMessages, afterAckMessages, canUnsubscribe, checkAndResumeIfPaused, checkAndUnblockIfStuck, clearDelayedMessages, close, close, consumerFlow, cursorIsReset, disconnectActiveConsumers, disconnectAllConsumers, disconnectAllConsumers, disconnectAllConsumers, getConsumers, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, getNumberOfDelayedMessages, getRateLimiter, getRedeliveryTracker, initializeDispatchRateLimiterIfNeeded, isClosed, isConsumerConnected, redeliverUnacknowledgedMessages, redeliverUnacknowledgedMessages, reset, resetCloseFuture, trackDelayedDelivery, updateRateLimiter

Methods inherited from interface org.apache.pulsar.broker.service.StickyKeyDispatcher

getNumberOfMessagesInReplay, getRecentlyJoinedConsumers, isClassic

Method Details

getSelector

public StickyKeyConsumerSelector getSelector()

Specified by:

getSelector in interface StickyKeyDispatcher

addConsumer

public CompletableFuture<Void> addConsumer(Consumer consumer)

Specified by:

addConsumer in interface Dispatcher

Overrides:

addConsumer in class PersistentDispatcherMultipleConsumers

removeConsumer

public void removeConsumer(Consumer consumer)
                    throws BrokerServiceException

Specified by:

removeConsumer in interface Dispatcher

Overrides:

removeConsumer in class PersistentDispatcherMultipleConsumers

Throws:

BrokerServiceException

clearComponentsAfterRemovedAllConsumers

protected void clearComponentsAfterRemovedAllConsumers()

Overrides:

clearComponentsAfterRemovedAllConsumers in class PersistentDispatcherMultipleConsumers

trySendMessagesToConsumers

protected boolean trySendMessagesToConsumers(PersistentDispatcherMultipleConsumers.ReadType readType,
 List<Entry> entries)

Description copied from class: PersistentDispatcherMultipleConsumers

Dispatch the messages to the Consumers.

Overrides:

trySendMessagesToConsumers in class PersistentDispatcherMultipleConsumers

Returns:

true if you want to trigger a new read. This method is overridden by other classes, please take a look to other implementations if you need to change it.

createFilterForReplay

protected Predicate<Position> createFilterForReplay()

Creates a filter for replaying messages. The filter is stateful and shouldn't be cached or reused.

Overrides:

createFilterForReplay in class PersistentDispatcherMultipleConsumers

See Also:

• PersistentDispatcherMultipleConsumers.createFilterForReplay()

getStickyKeyHash

protected int getStickyKeyHash(Entry entry)

Overrides:

getStickyKeyHash in class PersistentDispatcherMultipleConsumers

markDeletePositionMoveForward

public void markDeletePositionMoveForward()

Specified by:

markDeletePositionMoveForward in interface Dispatcher

canReplayMessages

protected boolean canReplayMessages()

The dispatcher will skip replaying messages when all messages in the replay queue are filtered out when skipNextReplayToTriggerLookAhead=true. The flag gets resetted after the call. If we're stuck on replay, we want to move forward reading on the topic (until the configured look ahead limits kick in), instead of keep replaying the same old messages, since the consumer that these messages are routing to might be busy at the moment. Please see ServiceConfiguration.getKeySharedLookAheadMsgInReplayThresholdPerConsumer() and ServiceConfiguration.getKeySharedLookAheadMsgInReplayThresholdPerSubscription() for configuring the limits.

Overrides:

canReplayMessages in class PersistentDispatcherMultipleConsumers

Returns:

true if replaying entries is currently enabled

doesntHavePendingRead

protected boolean doesntHavePendingRead()

For Key_Shared subscription, the dispatcher will not read more entries while there are pending reads or pending replay reads.

Overrides:

doesntHavePendingRead in class PersistentDispatcherMultipleConsumers

Returns:

true if there are no pending reads or pending replay reads

isNormalReadAllowed

protected boolean isNormalReadAllowed()

For Key_Shared subscription, the dispatcher will not attempt to read more entries if the replay queue size has reached the limit or if there are no consumers with permits.

Overrides:

isNormalReadAllowed in class PersistentDispatcherMultipleConsumers

getMaxEntriesReadLimit

protected int getMaxEntriesReadLimit()

Description copied from class: PersistentDispatcherMultipleConsumers

Sets a hard limit on the number of entries to read from the Managed Ledger. Subclasses can override this method to set a different limit. By default, this method does not impose an additional limit.

Overrides:

getMaxEntriesReadLimit in class PersistentDispatcherMultipleConsumers

Returns:

the maximum number of entries to read from the Managed Ledger

handleNormalReadNotAllowed

protected void handleNormalReadNotAllowed()

When a normal read is not allowed, the dispatcher will reschedule a read with a backoff.

Overrides:

handleNormalReadNotAllowed in class PersistentDispatcherMultipleConsumers

getType

public org.apache.pulsar.common.api.proto.CommandSubscribe.SubType getType()

Specified by:

getType in interface Dispatcher

Overrides:

getType in class PersistentDispatcherMultipleConsumers

asyncReplayEntries

protected Set<? extends Position> asyncReplayEntries(Set<? extends Position> positions)

Overrides:

asyncReplayEntries in class PersistentDispatcherMultipleConsumers

getKeySharedMode

public org.apache.pulsar.common.api.proto.KeySharedMode getKeySharedMode()

Specified by:

getKeySharedMode in interface StickyKeyDispatcher

isAllowOutOfOrderDelivery

public boolean isAllowOutOfOrderDelivery()

Specified by:

isAllowOutOfOrderDelivery in interface StickyKeyDispatcher

hasSameKeySharedPolicy

public boolean hasSameKeySharedPolicy(org.apache.pulsar.common.api.proto.KeySharedMeta ksm)

Specified by:

hasSameKeySharedPolicy in interface StickyKeyDispatcher

getConsumerKeyHashRanges

public Map<Consumer,List<org.apache.pulsar.client.api.Range>> getConsumerKeyHashRanges()

Specified by:

getConsumerKeyHashRanges in interface StickyKeyDispatcher