Package org.apache.pulsar.broker.service.persistent

Class PersistentDispatcherMultipleConsumersClassic

java.lang.Object

org.apache.pulsar.broker.service.EntryFilterSupport

org.apache.pulsar.broker.service.AbstractBaseDispatcher

org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.AbstractPersistentDispatcherMultipleConsumers

org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic

All Implemented Interfaces:

AsyncCallbacks.ReadEntriesCallback, Dispatcher

Direct Known Subclasses:

PersistentStickyKeyDispatcherMultipleConsumersClassic

public class PersistentDispatcherMultipleConsumersClassic
extends AbstractPersistentDispatcherMultipleConsumers

This is the "classic" dispatcher implementation for the Shared subscription that was used before Pulsar 4.0.0 and PIP-379. When `subscriptionSharedUseClassicPersistentImplementation=true`, this dispatcher will be used. The main purpose is to provide a way to rollback to the old behavior in case of issues with the preferred implementation.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static enum	PersistentDispatcherMultipleConsumersClassic.ReadType

Field Summary

Fields

Modifier and Type	Field	Description
protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic>	BLOCKED_DISPATCHER_ON_CURSOR_DATA_CAN_NOT_FULLY_PERSIST_UPDATER	A signature that relate to the check of "Dispatching has paused on cursor data can fully persist".
protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic>	BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER
protected final ManagedCursor	cursor
protected final ExecutorService	dispatchMessagesThread
protected Optional<DispatchRateLimiter>	dispatchRateLimiter
protected boolean	havePendingRead
protected boolean	havePendingReplayRead
protected com.google.common.collect.Range<Position>	lastIndividualDeletedRangeFromCursorRecovery
protected Position	minReplayedPosition
protected final String	name
protected int	readBatchSize
protected final org.apache.pulsar.common.util.Backoff	readFailureBackoff
protected final MessageRedeliveryController	redeliveryMessages
protected final RedeliveryTracker	redeliveryTracker
protected boolean	shouldRewindBeforeReadingOrReplaying
protected final PersistentTopic	topic
protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic>	TOTAL_AVAILABLE_PERMITS_UPDATER
protected int	totalAvailablePermits
protected int	totalUnackedMessages

Fields inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

consumerList, consumerSet, currentConsumerRoundRobinIndex, FALSE, IS_CLOSED_UPDATER, TRUE

Fields inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

dispatchThrottlingOnBatchMessageEnabled, serviceConfig

Fields inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

entryFilters, filterContext, hasFilter, subscription

Constructor Summary

Constructors

Constructor	Description
PersistentDispatcherMultipleConsumersClassic(PersistentTopic topic, ManagedCursor cursor, Subscription subscription)
PersistentDispatcherMultipleConsumersClassic(PersistentTopic topic, ManagedCursor cursor, Subscription subscription, boolean allowOutOfOrderDelivery)

Method Summary

Modifier and Type	Method	Description
protected void	acquireSendInProgress()
CompletableFuture<Void>	addConsumer(Consumer consumer)
protected boolean	addMessageToReplay(long ledgerId, long entryId)
protected boolean	addMessageToReplay(long ledgerId, long entryId, long stickyKeyHash)
void	addUnAckedMessages(int numberOfMessages)
void	afterAckMessages(Throwable exOfDeletion, Object ctxOfDeletion)	A callback hook after acknowledge messages.
protected Set<? extends Position>	asyncReplayEntries(Set<? extends Position> positions)
protected Set<? extends Position>	asyncReplayEntriesInOrder(Set<? extends Position> positions)
void	blockDispatcherOnUnackedMsgs()
protected org.apache.commons.lang3.tuple.Pair<Integer,Long>	calculateToRead(int currentTotalAvailablePermits)
protected void	cancelPendingRead()
boolean	canUnsubscribe(Consumer consumer)
boolean	checkAndResumeIfPaused()	Trigger a new "readMoreEntries" if the dispatching has been paused before.
boolean	checkAndUnblockIfStuck()	Checks if dispatcher is stuck and unblocks the dispatch if needed.
CompletableFuture<Void>	clearDelayedMessages()
CompletableFuture<Void>	close(boolean disconnectConsumers, Optional<BrokerLookupData> assignedBrokerLookupData)
void	consumerFlow(Consumer consumer, int additionalNumberOfMessages)	Indicates that this consumer is now ready to receive more messages.
void	cursorIsReset()
CompletableFuture<Void>	disconnectActiveConsumers(boolean isResetCursor)	Disconnect active consumers.
CompletableFuture<Void>	disconnectAllConsumers(boolean isResetCursor, Optional<BrokerLookupData> assignedBrokerLookupData)
protected NavigableSet<Position>	filterOutEntriesWillBeDiscarded(NavigableSet<Position> src)	This is a mode method designed for Key_Shared mode.
Map<String,org.apache.pulsar.common.policies.data.stats.TopicMetricBean>	getBucketDelayedIndexStats()
CopyOnWriteArrayList<Consumer>	getConsumers()
ManagedCursor	getCursor()
long	getDelayedTrackerMemoryUsage()
protected int	getFirstAvailableConsumerPermits()
protected NavigableSet<Position>	getMessagesToReplayNow(int maxMessagesToRead)
String	getName()
long	getNumberOfDelayedMessages()
long	getNumberOfMessagesInReplay()
Optional<DispatchRateLimiter>	getRateLimiter()
RedeliveryTracker	getRedeliveryTracker()
protected int	getStickyKeyHash(Entry entry)
Subscription	getSubscription()
PersistentTopic	getTopic()
int	getTotalUnackedMessages()
org.apache.pulsar.common.api.proto.CommandSubscribe.SubType	getType()
protected boolean	hasConsumersNeededNormalRead()	This is a mode method designed for Key_Shared mode, to avoid unnecessary stuck.
boolean	initializeDispatchRateLimiterIfNeeded()
protected boolean	isAtleastOneConsumerAvailable()	returns true only if AbstractDispatcherMultipleConsumers.consumerList has atleast one unblocked consumer and have available permits.
boolean	isBlockedDispatcherOnUnackedMsgs()
boolean	isClassic()
boolean	isConsumerAvailable(Consumer consumer)
boolean	isConsumerConnected()
protected boolean	isConsumersExceededOnSubscription()	Determine whether the number of consumers on the subscription reaches the threshold.
boolean	isHavePendingRead()
boolean	isHavePendingReplayRead()
protected boolean	isSendInProgress()
final void	readEntriesComplete(List<Entry> entries, Object ctx)
void	readEntriesFailed(ManagedLedgerException exception, Object ctx)
void	readMoreEntries()
void	readMoreEntriesAsync()	We should not call readMoreEntries() recursively in the same thread as there is a risk of StackOverflowError.
void	redeliverUnacknowledgedMessages(Consumer consumer, long consumerEpoch)
void	redeliverUnacknowledgedMessages(Consumer consumer, List<Position> positions)
protected void	releaseSendInProgress()
void	removeConsumer(Consumer consumer)
protected void	reScheduleRead()
void	reset()	mark dispatcher open to serve new incoming requests.
void	resetCloseFuture()
protected final boolean	sendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType, List<Entry> entries, boolean needAcquireSendInProgress)
protected boolean	shouldPauseDeliveryForDelayTracker()
boolean	trackDelayedDelivery(long ledgerId, long entryId, org.apache.pulsar.common.api.proto.MessageMetadata msgMetadata)	Check with dispatcher if the message should be added to the delayed delivery tracker.
protected boolean	trySendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType, List<Entry> entries)	Dispatch the messages to the Consumers.
void	unBlockDispatcherOnUnackedMsgs()

Methods inherited from class org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers

getNextConsumer, getRandomConsumer, isClosed

Methods inherited from class org.apache.pulsar.broker.service.AbstractBaseDispatcher

acquirePermitsForDeliveredMessages, checkAndApplyReachedEndOfTopicOrTopicMigration, checkAndApplyReachedEndOfTopicOrTopicMigration, computeReadLimits, filterEntriesForConsumer, filterEntriesForConsumer, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, getSubscriptionName, isConsumersExceededOnSubscription, peekStickyKey, updateMessagesToRead, updatePendingBytesToDispatch

Methods inherited from class org.apache.pulsar.broker.service.EntryFilterSupport

runFiltersForEntry

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.broker.service.Dispatcher

close, disconnectAllConsumers, disconnectAllConsumers, getFilterAcceptedMsgCount, getFilterProcessedMsgCount, getFilterRejectedMsgCount, getFilterRescheduledMsgCount, isClosed, markDeletePositionMoveForward, updateRateLimiter

Field Details

topic

protected final PersistentTopic topic

cursor

protected final ManagedCursor cursor

lastIndividualDeletedRangeFromCursorRecovery

protected volatile com.google.common.collect.Range<Position> lastIndividualDeletedRangeFromCursorRecovery

redeliveryMessages

protected final MessageRedeliveryController redeliveryMessages

redeliveryTracker

protected final RedeliveryTracker redeliveryTracker

havePendingRead

protected volatile boolean havePendingRead

havePendingReplayRead

protected volatile boolean havePendingReplayRead

minReplayedPosition

protected volatile Position minReplayedPosition

shouldRewindBeforeReadingOrReplaying

protected boolean shouldRewindBeforeReadingOrReplaying

name

protected final String name

TOTAL_AVAILABLE_PERMITS_UPDATER

protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic> TOTAL_AVAILABLE_PERMITS_UPDATER

totalAvailablePermits

protected volatile int totalAvailablePermits

readBatchSize

protected volatile int readBatchSize

readFailureBackoff

protected final org.apache.pulsar.common.util.Backoff readFailureBackoff

totalUnackedMessages

protected volatile int totalUnackedMessages

BLOCKED_DISPATCHER_ON_CURSOR_DATA_CAN_NOT_FULLY_PERSIST_UPDATER

protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic> BLOCKED_DISPATCHER_ON_CURSOR_DATA_CAN_NOT_FULLY_PERSIST_UPDATER

A signature that relate to the check of "Dispatching has paused on cursor data can fully persist". Note: It is a tool that helps determine whether it should trigger a new reading after acknowledgments to avoid too many CPU circles, see afterAckMessages(Throwable, Object) for more details. Do not use this to confirm whether the delivery should be paused, please call shouldPauseOnAckStatePersist(org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumersClassic.ReadType).

BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER

protected static final AtomicIntegerFieldUpdater<PersistentDispatcherMultipleConsumersClassic> BLOCKED_DISPATCHER_ON_UNACKMSG_UPDATER

dispatchRateLimiter

protected Optional<DispatchRateLimiter> dispatchRateLimiter

dispatchMessagesThread

protected final ExecutorService dispatchMessagesThread

Constructor Details

PersistentDispatcherMultipleConsumersClassic

public PersistentDispatcherMultipleConsumersClassic(PersistentTopic topic,
 ManagedCursor cursor,
 Subscription subscription)

PersistentDispatcherMultipleConsumersClassic

public PersistentDispatcherMultipleConsumersClassic(PersistentTopic topic,
 ManagedCursor cursor,
 Subscription subscription,
 boolean allowOutOfOrderDelivery)

Method Details

addConsumer

public CompletableFuture<Void> addConsumer(Consumer consumer)

isConsumersExceededOnSubscription

protected boolean isConsumersExceededOnSubscription()

Description copied from class: AbstractBaseDispatcher

Determine whether the number of consumers on the subscription reaches the threshold.

Specified by:

isConsumersExceededOnSubscription in class AbstractBaseDispatcher

Returns:

removeConsumer

public void removeConsumer(Consumer consumer)
                    throws BrokerServiceException

Throws:

BrokerServiceException

consumerFlow

public void consumerFlow(Consumer consumer,
 int additionalNumberOfMessages)

Description copied from interface: Dispatcher

Indicates that this consumer is now ready to receive more messages.

readMoreEntriesAsync

public void readMoreEntriesAsync()

We should not call readMoreEntries() recursively in the same thread as there is a risk of StackOverflowError.

Specified by:

readMoreEntriesAsync in class AbstractPersistentDispatcherMultipleConsumers

readMoreEntries

public void readMoreEntries()

reScheduleRead

protected void reScheduleRead()

Specified by:

reScheduleRead in class AbstractBaseDispatcher

calculateToRead

protected org.apache.commons.lang3.tuple.Pair<Integer,Long> calculateToRead(int currentTotalAvailablePermits)

asyncReplayEntries

protected Set<? extends Position> asyncReplayEntries(Set<? extends Position> positions)

asyncReplayEntriesInOrder

protected Set<? extends Position> asyncReplayEntriesInOrder(Set<? extends Position> positions)

isConsumerConnected

public boolean isConsumerConnected()

Specified by:

isConsumerConnected in interface Dispatcher

Overrides:

isConsumerConnected in class AbstractDispatcherMultipleConsumers

getConsumers

public CopyOnWriteArrayList<Consumer> getConsumers()

Specified by:

getConsumers in interface Dispatcher

Overrides:

getConsumers in class AbstractDispatcherMultipleConsumers

canUnsubscribe

public boolean canUnsubscribe(Consumer consumer)

Specified by:

canUnsubscribe in interface Dispatcher

Overrides:

canUnsubscribe in class AbstractDispatcherMultipleConsumers

close

public CompletableFuture<Void> close(boolean disconnectConsumers,
 Optional<BrokerLookupData> assignedBrokerLookupData)

disconnectAllConsumers

public CompletableFuture<Void> disconnectAllConsumers(boolean isResetCursor,
 Optional<BrokerLookupData> assignedBrokerLookupData)

cancelPendingRead

protected void cancelPendingRead()

Overrides:

cancelPendingRead in class AbstractDispatcherMultipleConsumers

disconnectActiveConsumers

public CompletableFuture<Void> disconnectActiveConsumers(boolean isResetCursor)

Description copied from interface: Dispatcher

Disconnect active consumers.

resetCloseFuture

public void resetCloseFuture()

Specified by:

resetCloseFuture in interface Dispatcher

Overrides:

resetCloseFuture in class AbstractBaseDispatcher

reset

public void reset()

Description copied from interface: Dispatcher

mark dispatcher open to serve new incoming requests.

getType

public org.apache.pulsar.common.api.proto.CommandSubscribe.SubType getType()

Specified by:

getType in interface Dispatcher

Overrides:

getType in class AbstractDispatcherMultipleConsumers

readEntriesComplete

public final void readEntriesComplete(List<Entry> entries,
 Object ctx)

acquireSendInProgress

protected void acquireSendInProgress()

releaseSendInProgress

protected void releaseSendInProgress()

isSendInProgress

protected boolean isSendInProgress()

sendMessagesToConsumers

protected final boolean sendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType,
 List<Entry> entries,
 boolean needAcquireSendInProgress)

trySendMessagesToConsumers

protected boolean trySendMessagesToConsumers(PersistentDispatcherMultipleConsumersClassic.ReadType readType,
 List<Entry> entries)

Dispatch the messages to the Consumers.

Returns:

true if you want to trigger a new read. This method is overridden by other classes, please take a look to other implementations if you need to change it.

readEntriesFailed

public void readEntriesFailed(ManagedLedgerException exception,
 Object ctx)

isAtleastOneConsumerAvailable

protected boolean isAtleastOneConsumerAvailable()

returns true only if AbstractDispatcherMultipleConsumers.consumerList has atleast one unblocked consumer and have available permits.

Returns:

getFirstAvailableConsumerPermits

protected int getFirstAvailableConsumerPermits()

isConsumerAvailable

public boolean isConsumerAvailable(Consumer consumer)

Specified by:

isConsumerAvailable in class AbstractDispatcherMultipleConsumers

redeliverUnacknowledgedMessages

public void redeliverUnacknowledgedMessages(Consumer consumer,
 long consumerEpoch)

redeliverUnacknowledgedMessages

public void redeliverUnacknowledgedMessages(Consumer consumer,
 List<Position> positions)

addUnAckedMessages

public void addUnAckedMessages(int numberOfMessages)

afterAckMessages

public void afterAckMessages(Throwable exOfDeletion,
 Object ctxOfDeletion)

Description copied from interface: Dispatcher

A callback hook after acknowledge messages.

Parameters:

exOfDeletion - the ex of ManagedCursor.asyncDelete(org.apache.bookkeeper.mledger.Position, org.apache.bookkeeper.mledger.AsyncCallbacks.DeleteCallback, java.lang.Object), ManagedCursor.asyncClearBacklog(org.apache.bookkeeper.mledger.AsyncCallbacks.ClearBacklogCallback, java.lang.Object) or {@link ManagedCursor#asyncSkipEntries)}.

ctxOfDeletion - the param ctx of calling ManagedCursor.asyncDelete(org.apache.bookkeeper.mledger.Position, org.apache.bookkeeper.mledger.AsyncCallbacks.DeleteCallback, java.lang.Object), ManagedCursor.asyncClearBacklog(org.apache.bookkeeper.mledger.AsyncCallbacks.ClearBacklogCallback, java.lang.Object) or {@link ManagedCursor#asyncSkipEntries)}.

checkAndResumeIfPaused

public boolean checkAndResumeIfPaused()

Description copied from interface: Dispatcher

Trigger a new "readMoreEntries" if the dispatching has been paused before. This method is only implemented in AbstractPersistentDispatcherMultipleConsumers right now, other implementations do not necessary implement this method.

Returns:

did a resume.

isBlockedDispatcherOnUnackedMsgs

public boolean isBlockedDispatcherOnUnackedMsgs()

Specified by:

isBlockedDispatcherOnUnackedMsgs in class AbstractPersistentDispatcherMultipleConsumers

blockDispatcherOnUnackedMsgs

public void blockDispatcherOnUnackedMsgs()

Specified by:

blockDispatcherOnUnackedMsgs in class AbstractPersistentDispatcherMultipleConsumers

unBlockDispatcherOnUnackedMsgs

public void unBlockDispatcherOnUnackedMsgs()

Specified by:

unBlockDispatcherOnUnackedMsgs in class AbstractPersistentDispatcherMultipleConsumers

getTotalUnackedMessages

public int getTotalUnackedMessages()

Specified by:

getTotalUnackedMessages in class AbstractPersistentDispatcherMultipleConsumers

getName

public String getName()

Specified by:

getName in class AbstractPersistentDispatcherMultipleConsumers

getRedeliveryTracker

public RedeliveryTracker getRedeliveryTracker()

getRateLimiter

public Optional<DispatchRateLimiter> getRateLimiter()

initializeDispatchRateLimiterIfNeeded

public boolean initializeDispatchRateLimiterIfNeeded()

trackDelayedDelivery

public boolean trackDelayedDelivery(long ledgerId,
 long entryId,
 org.apache.pulsar.common.api.proto.MessageMetadata msgMetadata)

Description copied from interface: Dispatcher

Check with dispatcher if the message should be added to the delayed delivery tracker. Return true if the message should be delayed and ignored at this point.

getMessagesToReplayNow

protected NavigableSet<Position> getMessagesToReplayNow(int maxMessagesToRead)

filterOutEntriesWillBeDiscarded

protected NavigableSet<Position> filterOutEntriesWillBeDiscarded(NavigableSet<Position> src)

This is a mode method designed for Key_Shared mode. Filter out the entries that will be discarded due to the order guarantee mechanism of Key_Shared mode. This method is in order to avoid the scenario below: - Get positions from the Replay queue. - Read entries from BK. - The order guarantee mechanism of Key_Shared mode filtered out all the entries. - Delivery non entry to the client, but we did a BK read.

hasConsumersNeededNormalRead

protected boolean hasConsumersNeededNormalRead()

This is a mode method designed for Key_Shared mode, to avoid unnecessary stuck. See detail PersistentStickyKeyDispatcherMultipleConsumersClassic.hasConsumersNeededNormalRead().

shouldPauseDeliveryForDelayTracker

protected boolean shouldPauseDeliveryForDelayTracker()

getNumberOfDelayedMessages

public long getNumberOfDelayedMessages()

clearDelayedMessages

public CompletableFuture<Void> clearDelayedMessages()

cursorIsReset

public void cursorIsReset()

addMessageToReplay

protected boolean addMessageToReplay(long ledgerId,
 long entryId,
 long stickyKeyHash)

addMessageToReplay

protected boolean addMessageToReplay(long ledgerId,
 long entryId)

checkAndUnblockIfStuck

public boolean checkAndUnblockIfStuck()

Description copied from interface: Dispatcher

Checks if dispatcher is stuck and unblocks the dispatch if needed.

getTopic

public PersistentTopic getTopic()

Specified by:

getTopic in class AbstractPersistentDispatcherMultipleConsumers

getDelayedTrackerMemoryUsage

public long getDelayedTrackerMemoryUsage()

Specified by:

getDelayedTrackerMemoryUsage in class AbstractPersistentDispatcherMultipleConsumers

getBucketDelayedIndexStats

public Map<String,org.apache.pulsar.common.policies.data.stats.TopicMetricBean> getBucketDelayedIndexStats()

Specified by:

getBucketDelayedIndexStats in class AbstractPersistentDispatcherMultipleConsumers

isClassic

public boolean isClassic()

Specified by:

isClassic in class AbstractPersistentDispatcherMultipleConsumers

getCursor

public ManagedCursor getCursor()

Specified by:

getCursor in class AbstractPersistentDispatcherMultipleConsumers

getStickyKeyHash

protected int getStickyKeyHash(Entry entry)

getSubscription

public Subscription getSubscription()

Specified by:

getSubscription in class AbstractPersistentDispatcherMultipleConsumers

getNumberOfMessagesInReplay

public long getNumberOfMessagesInReplay()

Specified by:

getNumberOfMessagesInReplay in class AbstractPersistentDispatcherMultipleConsumers

isHavePendingRead

public boolean isHavePendingRead()

Specified by:

isHavePendingRead in class AbstractPersistentDispatcherMultipleConsumers

isHavePendingReplayRead

public boolean isHavePendingReplayRead()

Specified by:

isHavePendingReplayRead in class AbstractPersistentDispatcherMultipleConsumers