Package org.apache.pulsar.broker.service.persistent

Class PersistentStickyKeyDispatcherMultipleConsumersClassic

java.lang.Object

org.apache.pulsar.broker.service.EntryFilterSupport

org.apache.pulsar.broker.service.AbstractBaseDispatcher

org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.AbstractPersistentDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic

org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumersClassic

All Implemented Interfaces:

AsyncCallbacks.ReadEntriesCallback, Dispatcher, StickyKeyDispatcher

public class PersistentStickyKeyDispatcherMultipleConsumersClassic
extends PersistentDispatcherMultipleConsumersClassic
implements StickyKeyDispatcher

This is the "classic" dispatcher implementation for the Key_Shared subscription that was used before Pulsar 4.0.0 and PIP-379. When `subscriptionKeySharedUseClassicPersistentImplementation=true`, this dispatcher will be used. The main purpose is to provide a way to rollback to the old behavior in case of issues with the preferred implementation.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic

PersistentDispatcherMultipleConsumersClassic.ReadType

Field Summary

Fields inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic

BLOCKED_DISPATCHER_ON_CURSOR_DATA_CAN_NOT_FULLY_PERSIST_UPDATER, BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER, cursor, dispatchMessagesThread, dispatchRateLimiter, havePendingRead, havePendingReplayRead, lastIndividualDeletedRangeFromCursorRecovery, minReplayedPosition, name, readBatchSize, readFailureBackoff, redeliveryMessages, redeliveryTracker, shouldRewindBeforeReadingOrReplaying, topic, TOTAL_AVAILABLE_PERMITS_UPDATER, totalAvailablePermits, totalUnackedMessages

Fields inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

consumerList, consumerSet, currentConsumerRoundRobinIndex, FALSE, IS_CLOSED_UPDATER, TRUE

Fields inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

dispatchThrottlingOnBatchMessageEnabled, serviceConfig

Fields inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

entryFilters, filterContext, hasFilter, subscription

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Void>	addConsumer(Consumer consumer)
protected Set<? extends Position>	asyncReplayEntries(Set<? extends Position> positions)
protected NavigableSet<Position>	filterOutEntriesWillBeDiscarded(NavigableSet<Position> src)	This is a mode method designed for Key_Shared mode.
Map<Consumer,List<org.apache.pulsar.client.api.Range>>	getConsumerKeyHashRanges()
org.apache.pulsar.common.api.proto.KeySharedMode	getKeySharedMode()
protected NavigableSet<Position>	getMessagesToReplayNow(int maxMessagesToRead)
LinkedHashMap<Consumer,Position>	getRecentlyJoinedConsumers()
StickyKeyConsumerSelector	getSelector()
protected int	getStickyKeyHash(Entry entry)
org.apache.pulsar.common.api.proto.CommandSubscribe.SubType	getType()
protected boolean	hasConsumersNeededNormalRead()	In Key_Shared mode, the consumer will not receive any entries from a normal reading if it is included in recentlyJoinedConsumers, they can only receive entries from replay reads.
boolean	hasSameKeySharedPolicy(org.apache.pulsar.common.api.proto.KeySharedMeta ksm)
boolean	isAllowOutOfOrderDelivery()
void	markDeletePositionMoveForward()
void	removeConsumer(Consumer consumer)
protected boolean	trySendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType, List<Entry> entries)	Dispatch the messages to the Consumers.

Methods inherited from class org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic

acquireSendInProgress, addMessageToReplay, addMessageToReplay, addUnAckedMessages, afterAckMessages, asyncReplayEntriesInOrder, blockDispatcherOnUnackedMsgs, calculateToRead, cancelPendingRead, canUnsubscribe, checkAndResumeIfPaused, checkAndUnblockIfStuck, clearDelayedMessages, close, consumerFlow, cursorIsReset, disconnectActiveConsumers, disconnectAllConsumers, getBucketDelayedIndexStats, getConsumers, getCursor, getDelayedTrackerMemoryUsage, getFirstAvailableConsumerPermits, getName, getNumberOfDelayedMessages, getNumberOfMessagesInReplay, getRateLimiter, getRedeliveryTracker, getSubscription, getTopic, getTotalUnackedMessages, initializeDispatchRateLimiterIfNeeded, isAtleastOneConsumerAvailable, isBlockedDispatcherOnUnackedMsgs, isClassic, isConsumerAvailable, isConsumerConnected, isConsumersExceededOnSubscription, isHavePendingRead, isHavePendingReplayRead, isSendInProgress, readEntriesComplete, readEntriesFailed, readMoreEntries, readMoreEntriesAsync, redeliverUnacknowledgedMessages, redeliverUnacknowledgedMessages, releaseSendInProgress, reScheduleRead, reset, resetCloseFuture, sendMessagesToConsumers, shouldPauseDeliveryForDelayTracker, trackDelayedDelivery, unBlockDispatcherOnUnackedMsgs

Methods inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

getNextConsumer, getRandomConsumer, isClosed

Methods inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

acquirePermitsForDeliveredMessages, checkAndApplyReachedEndOfTopicOrTopicMigration, checkAndApplyReachedEndOfTopicOrTopicMigration, computeReadLimits, filterEntriesForConsumer, filterEntriesForConsumer, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, getSubscriptionName, isConsumersExceededOnSubscription, peekStickyKey, updateMessagesToRead, updatePendingBytesToDispatch

Methods inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

runFiltersForEntry

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.broker.service.Dispatcher

addUnAckedMessages, afterAckMessages, canUnsubscribe, checkAndResumeIfPaused, checkAndUnblockIfStuck, clearDelayedMessages, close, close, consumerFlow, cursorIsReset, disconnectActiveConsumers, disconnectAllConsumers, disconnectAllConsumers, disconnectAllConsumers, getConsumers, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, getNumberOfDelayedMessages, getRateLimiter, getRedeliveryTracker, initializeDispatchRateLimiterIfNeeded, isClosed, isConsumerConnected, redeliverUnacknowledgedMessages, redeliverUnacknowledgedMessages, reset, resetCloseFuture, trackDelayedDelivery, updateRateLimiter

Methods inherited from interface org.apache.pulsar.broker.service.StickyKeyDispatcher

getNumberOfMessagesInReplay, isClassic

Method Details

getSelector

public StickyKeyConsumerSelector getSelector()

Specified by:

getSelector in interface StickyKeyDispatcher

addConsumer

public CompletableFuture<Void> addConsumer(Consumer consumer)

Specified by:

addConsumer in interface Dispatcher

Overrides:

addConsumer in class PersistentDispatcherMultipleConsumersClassic

removeConsumer

public void removeConsumer(Consumer consumer)
                    throws BrokerServiceException

Specified by:

removeConsumer in interface Dispatcher

Overrides:

removeConsumer in class PersistentDispatcherMultipleConsumersClassic

Throws:

BrokerServiceException

trySendMessagesToConsumers

protected boolean trySendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType,
 List<Entry> entries)

Description copied from class: PersistentDispatcherMultipleConsumersClassic

Dispatch the messages to the Consumers.

Overrides:

trySendMessagesToConsumers in class PersistentDispatcherMultipleConsumersClassic

Returns:

true if you want to trigger a new read. This method is overridden by other classes, please take a look to other implementations if you need to change it.

markDeletePositionMoveForward

public void markDeletePositionMoveForward()

Specified by:

markDeletePositionMoveForward in interface Dispatcher

getMessagesToReplayNow

protected NavigableSet<Position> getMessagesToReplayNow(int maxMessagesToRead)

Overrides:

getMessagesToReplayNow in class PersistentDispatcherMultipleConsumersClassic

filterOutEntriesWillBeDiscarded

protected NavigableSet<Position> filterOutEntriesWillBeDiscarded(NavigableSet<Position> src)

Description copied from class: PersistentDispatcherMultipleConsumersClassic

This is a mode method designed for Key_Shared mode. Filter out the entries that will be discarded due to the order guarantee mechanism of Key_Shared mode. This method is in order to avoid the scenario below: - Get positions from the Replay queue. - Read entries from BK. - The order guarantee mechanism of Key_Shared mode filtered out all the entries. - Delivery non entry to the client, but we did a BK read.

Overrides:

filterOutEntriesWillBeDiscarded in class PersistentDispatcherMultipleConsumersClassic

hasConsumersNeededNormalRead

protected boolean hasConsumersNeededNormalRead()

In Key_Shared mode, the consumer will not receive any entries from a normal reading if it is included in recentlyJoinedConsumers, they can only receive entries from replay reads. If all entries in PersistentDispatcherMultipleConsumersClassic.redeliveryMessages have been filtered out due to the order guarantee mechanism, Broker need a normal read to make the consumers not included in @link #recentlyJoinedConsumers} will not be stuck. See https://github.com/apache/pulsar/pull/7105.

Overrides:

hasConsumersNeededNormalRead in class PersistentDispatcherMultipleConsumersClassic

getType

public org.apache.pulsar.common.api.proto.CommandSubscribe.SubType getType()

Specified by:

getType in interface Dispatcher

Overrides:

getType in class PersistentDispatcherMultipleConsumersClassic

asyncReplayEntries

protected Set<? extends Position> asyncReplayEntries(Set<? extends Position> positions)

Overrides:

asyncReplayEntries in class PersistentDispatcherMultipleConsumersClassic

getKeySharedMode

public org.apache.pulsar.common.api.proto.KeySharedMode getKeySharedMode()

Specified by:

getKeySharedMode in interface StickyKeyDispatcher

isAllowOutOfOrderDelivery

public boolean isAllowOutOfOrderDelivery()

Specified by:

isAllowOutOfOrderDelivery in interface StickyKeyDispatcher

hasSameKeySharedPolicy

public boolean hasSameKeySharedPolicy(org.apache.pulsar.common.api.proto.KeySharedMeta ksm)

Specified by:

hasSameKeySharedPolicy in interface StickyKeyDispatcher

getRecentlyJoinedConsumers

public LinkedHashMap<Consumer,Position> getRecentlyJoinedConsumers()

Specified by:

getRecentlyJoinedConsumers in interface StickyKeyDispatcher

getConsumerKeyHashRanges

public Map<Consumer,List<org.apache.pulsar.client.api.Range>> getConsumerKeyHashRanges()

Specified by:

getConsumerKeyHashRanges in interface StickyKeyDispatcher

getStickyKeyHash

protected int getStickyKeyHash(Entry entry)

Overrides:

getStickyKeyHash in class PersistentDispatcherMultipleConsumersClassic