Package org.apache.pulsar.broker.service

Class BrokerService

java.lang.Object
org.apache.pulsar.broker.service.BrokerService
All Implemented Interfaces:
Closeable, AutoCloseable
public class BrokerService extends Object implements Closeable

  • Field Details

  • Constructor Details

  • Method Details

    • addTopicEventListener

      public void addTopicEventListener(TopicEventsListener... listeners)

    • removeTopicEventListener

      public void removeTopicEventListener(TopicEventsListener... listeners)

    • startProtocolHandlers

      public void startProtocolHandlers(Map<String,Map<InetSocketAddress,io.netty.channel.ChannelInitializer<io.netty.channel.socket.SocketChannel>>> protocolHandlers)

    • getTopicStats

      public Map<String,org.apache.pulsar.common.policies.data.stats.TopicStatsImpl> getTopicStats(NamespaceBundle bundle)

    • start

      public void start() throws Exception
      Throws:
      Exception

    • startClearInvalidateTopicNameCacheTask

      protected void startClearInvalidateTopicNameCacheTask()

    • startStatsUpdater

      protected void startStatsUpdater(int statsUpdateInitialDelayInSecs, int statsUpdateFrequencyInSecs)

    • initializeHealthChecker

      protected void initializeHealthChecker()

    • checkHealth

      public CompletableFuture<Void> checkHealth()

    • startDeduplicationSnapshotMonitor

      protected void startDeduplicationSnapshotMonitor()

    • startInactivityMonitor

      protected void startInactivityMonitor()

    • startMessageExpiryMonitor

      protected void startMessageExpiryMonitor()

    • startCheckReplicationPolicies

      protected void startCheckReplicationPolicies()

    • startCompactionMonitor

      protected void startCompactionMonitor()

    • startConsumedLedgersMonitor

      protected void startConsumedLedgersMonitor()

    • startBacklogQuotaChecker

      protected void startBacklogQuotaChecker()

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • closeAndRemoveReplicationClient

      public CompletableFuture<Void> closeAndRemoveReplicationClient(String clusterName)

    • closeAsync

      public CompletableFuture<Void> closeAsync()

    • unloadNamespaceBundlesGracefully

      public void unloadNamespaceBundlesGracefully()
      It unloads all owned namespacebundles gracefully.
      • First it makes current broker unavailable and isolates from the clusters so, it will not serve any new requests.
      • Second it starts unloading namespace bundle one by one without closing the connection in order to avoid disruption for other namespacebundles which are sharing the same connection from the same client.

    • unloadNamespaceBundlesGracefully

      public void unloadNamespaceBundlesGracefully(int maxConcurrentUnload, boolean closeWithoutWaitingClientDisconnect)

    • getTopicIfExists

      public CompletableFuture<Optional<Topic>> getTopicIfExists(String topic)

    • getOrCreateTopic

      public CompletableFuture<Topic> getOrCreateTopic(String topic)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(String topic, boolean createIfMissing)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(String topic, boolean createIfMissing, Map<String,String> properties)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(org.apache.pulsar.common.naming.TopicName topicName, boolean createIfMissing, Map<String,String> properties)
      Retrieves or creates a topic based on the specified parameters. 0. If disable PersistentTopics or NonPersistentTopics, it will return a failed future with NotAllowedException. 1. If topic future exists in the cache returned directly regardless of whether it fails or timeout. 2. If the topic metadata exists, the topic is created regardless of createIfMissing. 3. If the topic metadata not exists, and createIfMissing is false, returns an empty Optional in a CompletableFuture. And this empty future not be added to the map. 4. Otherwise, use computeIfAbsent. It returns the existing topic or creates and adds a new topicFuture. Any exceptions will remove the topicFuture from the map.
      Parameters:
      topicName - The name of the topic, potentially including partition information.
      createIfMissing - If true, creates the topic if it does not exist.
      properties - Topic configuration properties used during creation.
      Returns:
      CompletableFuture with an Optional of the topic if found or created, otherwise empty.

    • deleteTopic

      public CompletableFuture<Void> deleteTopic(String topic, boolean forceDelete)

    • getManagedLedgerFactoryForTopic

      public CompletableFuture<ManagedLedgerFactory> getManagedLedgerFactoryForTopic(org.apache.pulsar.common.naming.TopicName topicName)

    • getManagedLedgerFactoryForTopic

      public ManagedLedgerFactory getManagedLedgerFactoryForTopic(org.apache.pulsar.common.naming.TopicName topicName, String storageClassName)

    • deleteTopicAuthenticationWithRetry

      public void deleteTopicAuthenticationWithRetry(String topic, CompletableFuture<Void> future, int count)

    • getReplicationClient

      public org.apache.pulsar.client.api.PulsarClient getReplicationClient(String cluster, Optional<org.apache.pulsar.common.policies.data.ClusterData> clusterDataOp)

    • getClusterPulsarAdmin

      public PulsarAdmin getClusterPulsarAdmin(String cluster, Optional<org.apache.pulsar.common.policies.data.ClusterData> clusterDataOp)

    • loadOrCreatePersistentTopic

      protected CompletableFuture<Optional<Topic>> loadOrCreatePersistentTopic(String topic, boolean createIfMissing, Map<String,String> properties, @Nullable org.apache.pulsar.common.policies.data.TopicPolicies topicPolicies)
      It creates a topic async and returns CompletableFuture. It also throttles down configured max-concurrent topic loading and puts them into queue once in-process topics are created.
      Parameters:
      topic - persistent-topic name
      Returns:
      CompletableFuture
      Throws:
      RuntimeException

    • fetchTopicPropertiesAsync

      protected CompletableFuture<Map<String,String>> fetchTopicPropertiesAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • createPersistentTopic0

      public void createPersistentTopic0(String topic, boolean createIfMissing, CompletableFuture<Optional<Topic>> topicFuture, Map<String,String> properties)

    • getManagedLedgerConfig

      public CompletableFuture<ManagedLedgerConfig> getManagedLedgerConfig(@Nonnull org.apache.pulsar.common.naming.TopicName topicName)

    • refreshTopicToStatsMaps

      public void refreshTopicToStatsMaps(NamespaceBundle oldBundle)

    • getOfflineTopicStat

      public org.apache.pulsar.common.policies.data.PersistentOfflineTopicStats getOfflineTopicStat(org.apache.pulsar.common.naming.TopicName topicName)

    • cacheOfflineTopicStats

      public void cacheOfflineTopicStats(org.apache.pulsar.common.naming.TopicName topicName, org.apache.pulsar.common.policies.data.PersistentOfflineTopicStats offlineTopicStats)

    • invalidateOfflineTopicStatCache

      public void invalidateOfflineTopicStatCache(org.apache.pulsar.common.naming.TopicName topicName)

    • getTopicReference

      public Optional<Topic> getTopicReference(String topic)
      Get a reference to a topic that is currently loaded in the broker. This method will not make the broker attempt to load the topic if it's not already.

    • updateRates

      public void updateRates()

    • getDimensionMetrics

      public void getDimensionMetrics(Consumer<io.netty.buffer.ByteBuf> consumer)

    • getTopicMetrics

      public List<org.apache.pulsar.common.stats.Metrics> getTopicMetrics()

    • getBundleStats

      public Map<String,org.apache.pulsar.policies.data.loadbalancer.NamespaceBundleStats> getBundleStats()

    • getLookupRequestSemaphore

      public Semaphore getLookupRequestSemaphore()

    • checkGC

      public void checkGC()

    • checkClusterMigration

      public void checkClusterMigration()

    • checkMessageExpiry

      public void checkMessageExpiry()

    • checkReplicationPolicies

      public void checkReplicationPolicies()

    • checkCompaction

      public void checkCompaction()

    • checkMessageDeduplicationInfo

      public void checkMessageDeduplicationInfo()

    • checkInactiveSubscriptions

      public void checkInactiveSubscriptions()

    • forEachTopic

      public void forEachTopic(Consumer<Topic> consumer)
      Iterates over all loaded topics in the broker.

    • forEachPersistentTopic

      public void forEachPersistentTopic(Consumer<PersistentTopic> consumer)

    • getBacklogQuotaManager

      public BacklogQuotaManager getBacklogQuotaManager()

    • monitorBacklogQuota

      public void monitorBacklogQuota()

    • isTopicNsOwnedByBrokerAsync

      public CompletableFuture<Boolean> isTopicNsOwnedByBrokerAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • checkTopicNsOwnership

      public CompletableFuture<Void> checkTopicNsOwnership(String topic)

    • unloadServiceUnit

      public CompletableFuture<Integer> unloadServiceUnit(NamespaceBundle serviceUnit, boolean disconnectClients, boolean closeWithoutWaitingClientDisconnect, long timeout, TimeUnit unit)

    • cleanUnloadedTopicFromCache

      public void cleanUnloadedTopicFromCache(NamespaceBundle serviceUnit)

    • getAuthorizationService

      public AuthorizationService getAuthorizationService()

    • removeTopicFromCache

      public CompletableFuture<Void> removeTopicFromCache(Topic topic)

    • getNumberOfNamespaceBundles

      public long getNumberOfNamespaceBundles()

    • pulsar

      public PulsarService pulsar()

    • executor

      public io.netty.channel.EventLoopGroup executor()

    • isAuthenticationEnabled

      public boolean isAuthenticationEnabled()

    • isAuthorizationEnabled

      public boolean isAuthorizationEnabled()

    • getKeepAliveIntervalSeconds

      public int getKeepAliveIntervalSeconds()

    • generateUniqueProducerName

      public String generateUniqueProducerName()

    • getTopicStats

      public Map<String,org.apache.pulsar.common.policies.data.stats.TopicStatsImpl> getTopicStats()

    • getAuthenticationService

      public AuthenticationService getAuthenticationService()

    • getAllTopicsFromNamespaceBundle

      public List<Topic> getAllTopicsFromNamespaceBundle(String namespace, String bundle)

    • registerConfigurationListener

      public <T> void registerConfigurationListener(String configKey, Consumer<T> listener)
      Allows a listener to listen on update of ServiceConfiguration change, so listener can take appropriate action if any specific config-field value has been changed. On notification, listener should first check if config value has been changed and after taking appropriate action, listener should update config value with new value if it has been changed (so, next time listener can compare values on configMap change). Note: The new value that the may accept could be a null value.
      Type Parameters:
      T -
      Parameters:
      configKey - : configuration field name
      listener - : listener which takes appropriate action on config-value change

    • registerCustomDynamicConfiguration

      public void registerCustomDynamicConfiguration(String key, Predicate<String> validator)
      Allows the third-party plugin to register a custom dynamic configuration.

    • getDelayedDeliveryTrackerFactory

      public DelayedDeliveryTrackerFactory getDelayedDeliveryTrackerFactory()

    • getDynamicConfiguration

      public List<String> getDynamicConfiguration()

    • getRuntimeConfiguration

      public Map<String,String> getRuntimeConfiguration()

    • isDynamicConfiguration

      public boolean isDynamicConfiguration(String key)

    • validateDynamicConfiguration

      public boolean validateDynamicConfiguration(String key, String value)

    • fetchPartitionedTopicMetadataCheckAllowAutoCreationAsync

      public CompletableFuture<org.apache.pulsar.common.partition.PartitionedTopicMetadata> fetchPartitionedTopicMetadataCheckAllowAutoCreationAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • fetchPartitionedTopicMetadataAsync

      public CompletableFuture<org.apache.pulsar.common.partition.PartitionedTopicMetadata> fetchPartitionedTopicMetadataAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • fetchPartitionedTopicMetadataAsync

      public CompletableFuture<org.apache.pulsar.common.partition.PartitionedTopicMetadata> fetchPartitionedTopicMetadataAsync(org.apache.pulsar.common.naming.TopicName topicName, boolean refreshCacheAndGet)

    • getTopicOrderedExecutor

      public org.apache.bookkeeper.common.util.OrderedExecutor getTopicOrderedExecutor()

    • addUnAckedMessages

      public void addUnAckedMessages(AbstractPersistentDispatcherMultipleConsumers dispatcher, int numberOfMessages)
      If per-broker unacked message reached to limit then it blocks dispatcher if its unacked message limit has been reached to maxUnackedMsgsPerDispatcher.
      Parameters:
      dispatcher -
      numberOfMessages -

    • checkUnAckMessageDispatching

      public void checkUnAckMessageDispatching()
      Adds given dispatcher's unackMessage count to broker-unack message count and if it reaches to the maxUnackedMessages then it blocks all the dispatchers which has unack-messages higher than maxUnackedMsgsPerDispatcher. It unblocks all dispatchers once broker-unack message counts decreased to (maxUnackedMessages/2)

    • isBrokerDispatchingBlocked

      public boolean isBrokerDispatchingBlocked()

    • unblockDispatchersOnUnAckMessages

      public void unblockDispatchersOnUnAckMessages(List<AbstractPersistentDispatcherMultipleConsumers> dispatcherList)
      Unblocks the dispatchers and removes it from the blockedDispatchers list.
      Parameters:
      dispatcherList -

    • initializeFallbackDelayedDeliveryTrackerFactory

      public void initializeFallbackDelayedDeliveryTrackerFactory()
      Initializes the in-memory delayed delivery tracker factory when BucketDelayedDeliveryTrackerFactory.newTracker failed.

    • extractTopic

      public static Optional<Topic> extractTopic(CompletableFuture<Optional<Topic>> topicFuture)
      Safely extract optional topic instance from a future, in a way to avoid unchecked exceptions and race conditions.

    • getListenPort

      public Optional<Integer> getListenPort()

    • getListenPortTls

      public Optional<Integer> getListenPortTls()

    • isAllowAutoTopicCreationAsync

      public CompletableFuture<Boolean> isAllowAutoTopicCreationAsync(String topic)

    • isAllowAutoTopicCreationAsync

      public CompletableFuture<Boolean> isAllowAutoTopicCreationAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • isDefaultTopicTypePartitioned

      public boolean isDefaultTopicTypePartitioned(org.apache.pulsar.common.naming.TopicName topicName, Optional<org.apache.pulsar.common.policies.data.Policies> policies)

    • getDefaultNumPartitions

      public int getDefaultNumPartitions(org.apache.pulsar.common.naming.TopicName topicName, Optional<org.apache.pulsar.common.policies.data.Policies> policies)

    • isAllowAutoSubscriptionCreationAsync

      @Nonnull public CompletableFuture<Boolean> isAllowAutoSubscriptionCreationAsync(@Nonnull org.apache.pulsar.common.naming.TopicName tpName)

    • isSystemTopic

      public boolean isSystemTopic(String topic)

    • isSystemTopic

      public boolean isSystemTopic(org.apache.pulsar.common.naming.TopicName topicName)

    • deleteSchema

      public CompletableFuture<org.apache.pulsar.common.protocol.schema.SchemaVersion> deleteSchema(org.apache.pulsar.common.naming.TopicName topicName)

    • setInterceptor

      public void setInterceptor(BrokerInterceptor interceptor)

    • getBrokerEntryMetadataInterceptors

      public Set<org.apache.pulsar.common.intercept.BrokerEntryMetadataInterceptor> getBrokerEntryMetadataInterceptors()

    • isBrokerEntryMetadataEnabled

      public boolean isBrokerEntryMetadataEnabled()

    • isBrokerPayloadProcessorEnabled

      public boolean isBrokerPayloadProcessorEnabled()

    • recordConnectionPaused

      public void recordConnectionPaused()

    • recordConnectionResumed

      public void recordConnectionResumed()

    • recordConnectionThrottled

      public void recordConnectionThrottled()

    • recordConnectionUnthrottled

      public void recordConnectionUnthrottled()

    • newTopic

      public <T extends Topic> T newTopic(String topic, ManagedLedger ledger, BrokerService brokerService, Class<T> topicClazz) throws PulsarServerException
      Throws:
      PulsarServerException

    • setPulsarChannelInitializerFactory

      public void setPulsarChannelInitializerFactory(PulsarChannelInitializer.Factory factory)