Package org.apache.pulsar.broker.qos

Class AsyncTokenBucket

java.lang.Object
org.apache.pulsar.broker.qos.AsyncTokenBucket
Direct Known Subclasses:
DynamicRateAsyncTokenBucket
public abstract class AsyncTokenBucket extends Object
An asynchronous token bucket algorithm implementation that is optimized for performance with highly concurrent use. CAS (compare-and-swap) operations are used and multiple levels of CAS fields are used to minimize contention when using CAS fields. The LongAdder class is used in the hot path to hold the sum of consumed tokens. It is eventually consistent, meaning that the tokens are not updated on every call to the "consumeTokens" method.

Main usage flow: 1. Tokens are consumed by invoking the "consumeTokens" or "consumeTokensAndCheckIfContainsTokens" methods. 2. The "consumeTokensAndCheckIfContainsTokens" or "containsTokens" methods return false if there are no tokens available, indicating a need for throttling. 3. In case of throttling, the application should throttle in a way that is suitable for the use case and then call the "calculateThrottlingDuration" method to calculate the duration of the required pause. 4. After the pause duration, the application should verify if there are any available tokens by invoking the containsTokens method. If tokens are available, the application should cease throttling. However, if tokens are not available, the application should maintain the throttling and recompute the throttling duration. In a concurrent environment, it is advisable to use a throttling queue to ensure fair distribution of resources across throttled connections or clients. Once the throttling duration has elapsed, the application should select the next connection or client from the throttling queue to unthrottle. Before unthrottling, the application should check for available tokens. If tokens are still not available, the application should continue with throttling and repeat the throttling loop.

This class does not produce side effects outside its own scope. It functions similarly to a stateful function, akin to a counter function. In essence, it is a sophisticated counter. It can serve as a foundational component for constructing higher-level asynchronous rate limiter implementations, which require side effects for throttling.

To achieve optimal performance, pass a DefaultMonotonicSnapshotClock instance as the clock .

  • Field Details

    • DEFAULT_SNAPSHOT_CLOCK

      public static final MonotonicSnapshotClock DEFAULT_SNAPSHOT_CLOCK

    • tokens

      protected volatile long tokens
      This field represents the number of tokens in the bucket. It is eventually consistent, as the pendingConsumedTokens are subtracted from the total number of tokens at most once during each "tick" or "increment", when time advances according to the configured resolution.

    • resolutionNanos

      protected final long resolutionNanos
      The resolution in nanoseconds. This is the amount of time that must pass before the tokens are updated.

  • Constructor Details

  • Method Details

    • switchToConsistentTokensView

      public static void switchToConsistentTokensView()

    • resetToDefaultEventualConsistentTokensView

      public static void resetToDefaultEventualConsistentTokensView()

    • builder

      public static FinalRateAsyncTokenBucketBuilder builder()

    • builderForDynamicRate

      public static DynamicRateAsyncTokenBucketBuilder builderForDynamicRate()

    • getRatePeriodNanos

      protected abstract long getRatePeriodNanos()

    • getTargetAmountOfTokensAfterThrottling

      protected abstract long getTargetAmountOfTokensAfterThrottling()

    • consumeTokens

      public void consumeTokens(long consumeTokens)
      Eventually consume tokens from the bucket. The number of tokens is eventually consistent with the configured granularity of resolutionNanos.
      Parameters:
      consumeTokens - the number of tokens to consume

    • consumeTokensAndCheckIfContainsTokens

      public boolean consumeTokensAndCheckIfContainsTokens(long consumeTokens)
      Eventually consume tokens from the bucket and check if tokens remain available. The number of tokens is eventually consistent with the configured granularity of resolutionNanos. Therefore, the returned result is not definite.
      Parameters:
      consumeTokens - the number of tokens to consume
      Returns:
      true if there is tokens remains, false if tokens are all consumed. The answer isn't definite since the comparison is made with eventually consistent token value.

    • tokens

      protected long tokens(boolean forceUpdateTokens)
      Returns the current token balance. When forceUpdateTokens is true, the tokens balance is updated before returning. If forceUpdateTokens is false, the tokens balance could be updated if the last updated happened more than resolutionNanos nanoseconds ago.
      Parameters:
      forceUpdateTokens - if true, the tokens balance is updated before returning
      Returns:
      the current token balance

    • calculateThrottlingDuration

      public long calculateThrottlingDuration()
      Calculate the required throttling duration in nanoseconds to fill up the bucket with the minimum amount of tokens. This method shouldn't be called from the hot path since it calculates a consistent value for the tokens which isn't necessary on the hotpath.

    • getCapacity

      public abstract long getCapacity()

    • getTokens

      public final long getTokens()
      Returns the current number of tokens in the bucket. The token balance is updated if the configured resolutionNanos has passed since the last update.

    • getRate

      public abstract long getRate()

    • containsTokens

      public boolean containsTokens()
      Checks if the bucket contains tokens. The token balance is updated before the comparison if the configured resolutionNanos has passed since the last update. It's possible that the returned result is not definite since the token balance is eventually consistent.
      Returns:
      true if the bucket contains tokens, false otherwise

    • containsTokens

      public boolean containsTokens(boolean forceUpdateTokens)
      Checks if the bucket contains tokens. The token balance is updated before the comparison if the configured resolutionNanos has passed since the last update. The token balance is also updated when forceUpdateTokens is true. It's possible that the returned result is not definite since the token balance is eventually consistent.
      Parameters:
      forceUpdateTokens - if true, the token balance is updated before the comparison
      Returns:
      true if the bucket contains tokens, false otherwise