Package org.apache.pulsar.broker

Class PulsarService

java.lang.Object

org.apache.pulsar.broker.PulsarService

All Implemented Interfaces:

AutoCloseable, ShutdownService

public class PulsarService
extends Object
implements AutoCloseable, ShutdownService

Main class for Pulsar broker service.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	PulsarService.State

Field Summary

Fields

Modifier and Type	Field	Description
protected CompactionServiceFactory	compactionServiceFactory
protected final io.netty.channel.EventLoopGroup	ioEventLoopGroup

Constructor Summary

Constructors

Constructor	Description
PulsarService(ServiceConfiguration config)
PulsarService(ServiceConfiguration config, Optional<WorkerService> functionWorkerService, Consumer<Integer> processTerminator)
PulsarService(ServiceConfiguration config, WorkerConfig workerConfig, Optional<WorkerService> functionWorkerService, Consumer<Integer> processTerminator)
PulsarService(ServiceConfiguration config, WorkerConfig workerConfig, Optional<WorkerService> functionWorkerService, Consumer<Integer> processTerminator, Consumer<io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder> openTelemetrySdkBuilderCustomizer)

Method Summary

Modifier and Type	Method	Description
protected void	acquireSLANamespace()
void	addPrometheusRawMetricsProvider(PrometheusRawMetricsProvider metricsProvider)
static String	brokerUrl(String host, int port)
protected String	brokerUrl(ServiceConfiguration config)	Gets the broker service URL (non-TLS) associated with the internal listener.
static String	brokerUrlTls(String host, int port)
String	brokerUrlTls(ServiceConfiguration config)	Gets the broker service URL (TLS) associated with the internal listener.
void	close()
CompletableFuture<Void>	closeAsync()	Close the current pulsar service.
protected CompletableFuture<Void>	closeLocalMetadataStore()
void	closeMetadataServiceSession()	Close the session to the metadata service.
PulsarClientImpl	createClientImpl(ClientConfigurationData clientConf)
org.apache.pulsar.metadata.api.MetadataStore	createConfigurationMetadataStore(PulsarMetadataEventSynchronizer synchronizer, io.opentelemetry.api.OpenTelemetry openTelemetry)
org.apache.pulsar.metadata.api.extended.MetadataStoreExtended	createLocalMetadataStore(PulsarMetadataEventSynchronizer synchronizer, io.opentelemetry.api.OpenTelemetry openTelemetry)
LedgerOffloader	createManagedLedgerOffloader(org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)
PulsarAdmin	getAdminClient()
BookKeeper	getBookKeeperClient()
BookKeeperClientFactory	getBookKeeperClientFactory()
String	getBrokerId()	Return the broker id.
Optional<Integer>	getBrokerListenPort()
Optional<Integer>	getBrokerListenPortTls()
org.apache.pulsar.client.api.PulsarClient	getClient()
ScheduledExecutorService	getCompactorExecutor()
ServiceConfiguration	getConfiguration()	Get the current service configuration.
ManagedLedgerFactory	getDefaultManagedLedgerFactory()
org.apache.pulsar.common.conf.InternalConfigurationData	getInternalConfigurationData()
LeaderElectionService	getLeaderElectionService()	Get a reference of the current LeaderElectionService instance associated with the current PulsarService instance.
Optional<Integer>	getListenPortHTTP()
Optional<Integer>	getListenPortHTTPS()
LedgerOffloader	getManagedLedgerOffloader(org.apache.pulsar.common.naming.NamespaceName namespaceName, org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)	First, get LedgerOffloader from local map cache, create new LedgerOffloader if not in cache or the OffloadPolicies changed, return the LedgerOffloader directly if exist in cache and the OffloadPolicies not changed.
MonotonicSnapshotClock	getMonotonicSnapshotClock()
NamespaceService	getNamespaceService()	Get a reference of the current namespace service instance.
Supplier<NamespaceService>	getNamespaceServiceProvider()
Compactor	getNullableCompactor()
protected org.apache.bookkeeper.common.util.OrderedScheduler	getOffloaderScheduler(org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)
PackagesManagement	getPackagesManagement()
Map<String,String>	getProtocolDataToAdvertise()
String	getSafeBrokerServiceUrl()	Deprecated.
String	getSafeWebServiceAddress()
String	getStatusFilePath()
StrategicTwoPhaseCompactor	getStrategicCompactor()
Optional<WorkerConfig>	getWorkerConfig()	Get the current function worker service configuration.
WorkerService	getWorkerService()
Optional<WorkerService>	getWorkerServiceOpt()
void	initConfigMetadataSynchronizerIfNeeded()
static WorkerConfig	initializeWorkerConfigFromBrokerConfig(ServiceConfiguration brokerConfig, String workerConfigFile)
boolean	isRunning()	check the current pulsar service is running, including Started and Init state.
void	loadNamespaceTopics(NamespaceBundle bundle)	Load all the topics contained in a namespace.
BookKeeperClientFactory	newBookKeeperClientFactory()
protected BrokerInterceptor	newBrokerInterceptor()
protected BrokerService	newBrokerService(PulsarService pulsar)
protected ManagedLedgerStorage	newManagedLedgerStorage()
protected org.apache.bookkeeper.common.util.OrderedExecutor	newOrderedExecutor()
protected PulsarResources	newPulsarResources()
StrategicTwoPhaseCompactor	newStrategicCompactor()
CompletableFuture<TopicCompactionService>	newTopicCompactionService(String topic)
void	runWhenReadyForIncomingRequests(Runnable runnable)
void	setTransactionBufferProvider(TransactionBufferProvider transactionBufferProvider)
void	shutdownNow()	Shutdown the broker immediately, without waiting for all resources to be released.
void	start()	Start the pulsar service instance.
protected void	startLeaderElectionService()
protected void	startLoadManagementService()
protected void	startNamespaceService()
void	waitUntilClosed()	Block until the service is finally closed.
void	waitUntilReadyForIncomingRequests()
static String	webAddress(String host, int port)
String	webAddress(ServiceConfiguration config)
static String	webAddressTls(String host, int port)
String	webAddressTls(ServiceConfiguration config)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

compactionServiceFactory

protected CompactionServiceFactory compactionServiceFactory

ioEventLoopGroup

protected final io.netty.channel.EventLoopGroup ioEventLoopGroup

Constructor Details

PulsarService

public PulsarService(ServiceConfiguration config)

PulsarService

public PulsarService(ServiceConfiguration config,
 Optional<WorkerService> functionWorkerService,
 Consumer<Integer> processTerminator)

PulsarService

public PulsarService(ServiceConfiguration config,
 WorkerConfig workerConfig,
 Optional<WorkerService> functionWorkerService,
 Consumer<Integer> processTerminator)

PulsarService

public PulsarService(ServiceConfiguration config,
 WorkerConfig workerConfig,
 Optional<WorkerService> functionWorkerService,
 Consumer<Integer> processTerminator,
 Consumer<io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdkBuilder> openTelemetrySdkBuilderCustomizer)

Method Details

createConfigurationMetadataStore

public org.apache.pulsar.metadata.api.MetadataStore createConfigurationMetadataStore(PulsarMetadataEventSynchronizer synchronizer,
 io.opentelemetry.api.OpenTelemetry openTelemetry)
                                                                              throws org.apache.pulsar.metadata.api.MetadataStoreException

Throws:

org.apache.pulsar.metadata.api.MetadataStoreException

closeMetadataServiceSession

public void closeMetadataServiceSession()
                                 throws Exception

Close the session to the metadata service.

This will immediately release all the resource locks held by this broker on the coordination service.

Throws:

Exception - if the close operation fails

close

public void close()
           throws PulsarServerException

Specified by:

close in interface AutoCloseable

Throws:

PulsarServerException

closeAsync

public CompletableFuture<Void> closeAsync()

Close the current pulsar service. All resources are released.

getConfiguration

public ServiceConfiguration getConfiguration()

Get the current service configuration.

Returns:

the current service configuration

getWorkerConfig

public Optional<WorkerConfig> getWorkerConfig()

Get the current function worker service configuration.

Returns:

the current function worker service configuration.

getProtocolDataToAdvertise

public Map<String,String> getProtocolDataToAdvertise()

start

public void start()
           throws PulsarServerException

Start the pulsar service instance.

Throws:

PulsarServerException

runWhenReadyForIncomingRequests

public void runWhenReadyForIncomingRequests(Runnable runnable)

waitUntilReadyForIncomingRequests

public void waitUntilReadyForIncomingRequests()
                                       throws ExecutionException,
InterruptedException

Throws:

ExecutionException

InterruptedException

newBrokerInterceptor

protected BrokerInterceptor newBrokerInterceptor()
                                          throws IOException

Throws:

IOException

newOrderedExecutor

protected org.apache.bookkeeper.common.util.OrderedExecutor newOrderedExecutor()

newManagedLedgerStorage

protected ManagedLedgerStorage newManagedLedgerStorage()
                                                throws Exception

Throws:

Exception

newPulsarResources

protected PulsarResources newPulsarResources()

createLocalMetadataStore

public org.apache.pulsar.metadata.api.extended.MetadataStoreExtended createLocalMetadataStore(PulsarMetadataEventSynchronizer synchronizer,
 io.opentelemetry.api.OpenTelemetry openTelemetry)
                                                                                       throws org.apache.pulsar.metadata.api.MetadataStoreException,
PulsarServerException

Throws:

org.apache.pulsar.metadata.api.MetadataStoreException

PulsarServerException

closeLocalMetadataStore

protected CompletableFuture<Void> closeLocalMetadataStore()
                                                   throws Exception

Throws:

Exception

startLeaderElectionService

protected void startLeaderElectionService()

acquireSLANamespace

protected void acquireSLANamespace()

waitUntilClosed

public void waitUntilClosed()
                     throws InterruptedException

Block until the service is finally closed.

Throws:

InterruptedException

startNamespaceService

protected void startNamespaceService()
                              throws PulsarServerException

Throws:

PulsarServerException

getNamespaceServiceProvider

public Supplier<NamespaceService> getNamespaceServiceProvider()
                                                       throws PulsarServerException

Throws:

PulsarServerException

startLoadManagementService

protected void startLoadManagementService()
                                   throws PulsarServerException

Throws:

PulsarServerException

loadNamespaceTopics

public void loadNamespaceTopics(NamespaceBundle bundle)

Load all the topics contained in a namespace.

Parameters:

bundle - NamespaceBundle to identify the service unit

getStatusFilePath

public String getStatusFilePath()

getInternalConfigurationData

public org.apache.pulsar.common.conf.InternalConfigurationData getInternalConfigurationData()

isRunning

public boolean isRunning()

check the current pulsar service is running, including Started and Init state.

getLeaderElectionService

public LeaderElectionService getLeaderElectionService()

Get a reference of the current LeaderElectionService instance associated with the current PulsarService instance.

Returns:

a reference of the current LeaderElectionService instance.

getNamespaceService

public NamespaceService getNamespaceService()

Get a reference of the current namespace service instance.

Returns:

a reference of the current namespace service instance.

getWorkerServiceOpt

public Optional<WorkerService> getWorkerServiceOpt()

getWorkerService

public WorkerService getWorkerService()
                               throws UnsupportedOperationException

Throws:

UnsupportedOperationException

getBookKeeperClient

public BookKeeper getBookKeeperClient()

getDefaultManagedLedgerFactory

public ManagedLedgerFactory getDefaultManagedLedgerFactory()

getManagedLedgerOffloader

public LedgerOffloader getManagedLedgerOffloader(org.apache.pulsar.common.naming.NamespaceName namespaceName,
 org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)

First, get LedgerOffloader from local map cache, create new LedgerOffloader if not in cache or the OffloadPolicies changed, return the LedgerOffloader directly if exist in cache and the OffloadPolicies not changed.

Parameters:

namespaceName - NamespaceName

offloadPolicies - the OffloadPolicies

Returns:

LedgerOffloader

createManagedLedgerOffloader

public LedgerOffloader createManagedLedgerOffloader(org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)
                                             throws PulsarServerException

Throws:

PulsarServerException

newBookKeeperClientFactory

public BookKeeperClientFactory newBookKeeperClientFactory()

getBookKeeperClientFactory

public BookKeeperClientFactory getBookKeeperClientFactory()

getCompactorExecutor

public ScheduledExecutorService getCompactorExecutor()

getNullableCompactor

public Compactor getNullableCompactor()

newStrategicCompactor

public StrategicTwoPhaseCompactor newStrategicCompactor()
                                                 throws PulsarServerException

Throws:

PulsarServerException

getStrategicCompactor

public StrategicTwoPhaseCompactor getStrategicCompactor()
                                                 throws PulsarServerException

Throws:

PulsarServerException

getOffloaderScheduler

protected org.apache.bookkeeper.common.util.OrderedScheduler getOffloaderScheduler(org.apache.pulsar.common.policies.data.OffloadPoliciesImpl offloadPolicies)

createClientImpl

public PulsarClientImpl createClientImpl(ClientConfigurationData clientConf)
                                  throws org.apache.pulsar.client.api.PulsarClientException

Throws:

org.apache.pulsar.client.api.PulsarClientException

getClient

public org.apache.pulsar.client.api.PulsarClient getClient()
                                                    throws PulsarServerException

Throws:

PulsarServerException

getAdminClient

public PulsarAdmin getAdminClient()
                           throws PulsarServerException

Throws:

PulsarServerException

brokerUrl

protected String brokerUrl(ServiceConfiguration config)

Gets the broker service URL (non-TLS) associated with the internal listener.

brokerUrl

public static String brokerUrl(String host,
 int port)

brokerUrlTls

public String brokerUrlTls(ServiceConfiguration config)

Gets the broker service URL (TLS) associated with the internal listener.

brokerUrlTls

public static String brokerUrlTls(String host,
 int port)

webAddress

public String webAddress(ServiceConfiguration config)

webAddress

public static String webAddress(String host,
 int port)

webAddressTls

public String webAddressTls(ServiceConfiguration config)

webAddressTls

public static String webAddressTls(String host,
 int port)

getSafeWebServiceAddress

public String getSafeWebServiceAddress()

getSafeBrokerServiceUrl

@Deprecated
public String getSafeBrokerServiceUrl()

Deprecated.

getBrokerId

public String getBrokerId()

Return the broker id. The broker id is used in the load manager to uniquely identify the broker at runtime. It should not be used for making connections to the broker. The broker id is available after start() has been called.

Returns:

broker id

addPrometheusRawMetricsProvider

public void addPrometheusRawMetricsProvider(PrometheusRawMetricsProvider metricsProvider)

getPackagesManagement

public PackagesManagement getPackagesManagement()
                                         throws UnsupportedOperationException

Throws:

UnsupportedOperationException

getListenPortHTTP

public Optional<Integer> getListenPortHTTP()

getListenPortHTTPS

public Optional<Integer> getListenPortHTTPS()

getBrokerListenPort

public Optional<Integer> getBrokerListenPort()

getBrokerListenPortTls

public Optional<Integer> getBrokerListenPortTls()

getMonotonicSnapshotClock

public MonotonicSnapshotClock getMonotonicSnapshotClock()

initializeWorkerConfigFromBrokerConfig

public static WorkerConfig initializeWorkerConfigFromBrokerConfig(ServiceConfiguration brokerConfig,
 String workerConfigFile)
                                                           throws IOException

Throws:

IOException

shutdownNow

public void shutdownNow()

Shutdown the broker immediately, without waiting for all resources to be released. This possibly is causing the JVM process to exit, depending on how th PulsarService was constructed.

Specified by:

shutdownNow in interface ShutdownService

newBrokerService

protected BrokerService newBrokerService(PulsarService pulsar)
                                  throws Exception

Throws:

Exception

setTransactionBufferProvider

public void setTransactionBufferProvider(TransactionBufferProvider transactionBufferProvider)

newTopicCompactionService

public CompletableFuture<TopicCompactionService> newTopicCompactionService(String topic)

initConfigMetadataSynchronizerIfNeeded

public void initConfigMetadataSynchronizerIfNeeded()