Package org.apache.pulsar.broker.loadbalance.extensions

Class ExtensibleLoadManagerImpl

java.lang.Object

org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl

All Implemented Interfaces:

Closeable, AutoCloseable, ExtensibleLoadManager, BrokerSelectionStrategyFactory

public class ExtensibleLoadManagerImpl
extends Object
implements ExtensibleLoadManager, BrokerSelectionStrategyFactory

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	ExtensibleLoadManagerImpl.Role

Field Summary

Fields

Modifier and Type	Field	Description
static final String	BROKER_LOAD_DATA_STORE_TOPIC
static final long	COMPACTION_THRESHOLD
static final Set<String>	INTERNAL_TOPICS
protected PulsarService	pulsar
protected ServiceUnitStateTableViewSyncer	serviceUnitStateTableViewSyncer
static final String	TOP_BUNDLES_LOAD_DATA_STORE_TOPIC

Constructor Summary

Constructors

Constructor	Description
ExtensibleLoadManagerImpl()	Life cycle: Constructor -> initialize -> start -> close.

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Optional<BrokerLookupData>>	assign(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic, org.apache.pulsar.common.naming.ServiceUnitId serviceUnit, LookupOptions options)	The incoming service unit (e.g.
CompletableFuture<Boolean>	checkOwnershipAsync(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic, org.apache.pulsar.common.naming.ServiceUnitId bundleUnit)	Check the incoming service unit is owned by the current broker.
void	close()	Close the load manager.
static boolean	configureSystemTopics(PulsarService pulsar, long target)
protected BrokerRegistry	createBrokerRegistry(PulsarService pulsar)
BrokerSelectionStrategy	createBrokerSelectionStrategy()
protected ServiceUnitStateChannel	createServiceUnitStateChannel(PulsarService pulsar)
static void	createSystemTopic(PulsarService pulsar, String topic)
static boolean	debug(ServiceConfiguration config, org.slf4j.Logger log)
void	disableBroker()
static ExtensibleLoadManagerImpl	get(LoadManager loadManager)
static ExtensibleLoadManagerImpl	get(PulsarService pulsar)	A static util func to get the ExtensibleLoadManagerImpl instance.
static CompletableFuture<Optional<BrokerLookupData>>	getAssignedBrokerLookupData(PulsarService pulsar, String topic)	Gets the assigned broker for the given topic.
List<org.apache.pulsar.common.stats.Metrics>	getMetrics()
Set<NamespaceBundle>	getOwnedServiceUnits()
CompletableFuture<Set<NamespaceBundle>>	getOwnedServiceUnitsAsync()	Deprecated.
CompletableFuture<Optional<String>>	getOwnershipAsync(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic, org.apache.pulsar.common.naming.ServiceUnitId serviceUnit)
CompletableFuture<Optional<BrokerLookupData>>	getOwnershipWithLookupDataAsync(org.apache.pulsar.common.naming.ServiceUnitId bundleUnit)
void	initialize(PulsarService pulsar)	Initialize this load manager using the given pulsar service.
static boolean	isInternalTopic(String topic)
static boolean	isLoadManagerExtensionEnabled(PulsarService pulsar)
protected void	monitor()
CompletableFuture<Optional<String>>	selectAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle, Set<String> excludeBrokerSet, LookupOptions options)
CompletableFuture<Void>	splitNamespaceBundleAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle, NamespaceBundleSplitAlgorithm splitAlgorithm, List<Long> boundaries)
void	start()	Start the extensible load manager.
CompletableFuture<NamespaceEphemeralData>	tryAcquiringOwnership(NamespaceBundle namespaceBundle)	Method to get the current owner of the NamespaceBundle or set the local broker as the owner if absent.
CompletableFuture<Void>	unloadNamespaceBundleAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle, Optional<String> destinationBroker, boolean force, long timeout, TimeUnit timeoutUnit)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

BROKER_LOAD_DATA_STORE_TOPIC

public static final String BROKER_LOAD_DATA_STORE_TOPIC

TOP_BUNDLES_LOAD_DATA_STORE_TOPIC

public static final String TOP_BUNDLES_LOAD_DATA_STORE_TOPIC

COMPACTION_THRESHOLD

public static final long COMPACTION_THRESHOLD

See Also:

• Constant Field Values

INTERNAL_TOPICS

public static final Set<String> INTERNAL_TOPICS

pulsar

protected PulsarService pulsar

serviceUnitStateTableViewSyncer

protected ServiceUnitStateTableViewSyncer serviceUnitStateTableViewSyncer

Constructor Details

ExtensibleLoadManagerImpl

public ExtensibleLoadManagerImpl()

Life cycle: Constructor -> initialize -> start -> close.

Method Details

getOwnedServiceUnitsAsync

@Deprecated
public CompletableFuture<Set<NamespaceBundle>> getOwnedServiceUnitsAsync()

Deprecated.

Get all the bundles that are owned by this broker.

getOwnedServiceUnits

public Set<NamespaceBundle> getOwnedServiceUnits()

createBrokerSelectionStrategy

public BrokerSelectionStrategy createBrokerSelectionStrategy()

Specified by:

createBrokerSelectionStrategy in interface BrokerSelectionStrategyFactory

isLoadManagerExtensionEnabled

public static boolean isLoadManagerExtensionEnabled(PulsarService pulsar)

get

public static ExtensibleLoadManagerImpl get(LoadManager loadManager)

get

public static ExtensibleLoadManagerImpl get(PulsarService pulsar)

A static util func to get the ExtensibleLoadManagerImpl instance.

Parameters:

pulsar - PulsarService

Returns:

the ExtensibleLoadManagerImpl instance

debug

public static boolean debug(ServiceConfiguration config,
 org.slf4j.Logger log)

createSystemTopic

public static void createSystemTopic(PulsarService pulsar,
 String topic)
                              throws PulsarServerException

Throws:

PulsarServerException

configureSystemTopics

public static boolean configureSystemTopics(PulsarService pulsar,
 long target)

getAssignedBrokerLookupData

public static CompletableFuture<Optional<BrokerLookupData>> getAssignedBrokerLookupData(PulsarService pulsar,
 String topic)

Gets the assigned broker for the given topic.

Parameters:

pulsar - PulsarService instance

topic - Topic Name

Returns:

the assigned broker's BrokerLookupData instance. Empty, if not assigned by Extensible LoadManager or the optimized bundle unload process is disabled.

start

public void start()
           throws PulsarServerException

Description copied from interface: ExtensibleLoadManager

Start the extensible load manager. 1. Start the broker registry. 2. Register self to registry. 3. Start the load data store. 4. Init the load manager context. 5. Start load data reporter. 6. Start the namespace unload scheduler. 7. Start the namespace split scheduler. 8. Listen the broker up or down, so we can split immediately.

Specified by:

start in interface ExtensibleLoadManager

Throws:

PulsarServerException

initialize

public void initialize(PulsarService pulsar)

Description copied from interface: ExtensibleLoadManager

Initialize this load manager using the given pulsar service.

Specified by:

initialize in interface ExtensibleLoadManager

assign

public CompletableFuture<Optional<BrokerLookupData>> assign(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic,
 org.apache.pulsar.common.naming.ServiceUnitId serviceUnit,
 LookupOptions options)

Description copied from interface: ExtensibleLoadManager

The incoming service unit (e.g. bundle) selects the appropriate broker through strategies.

Specified by:

assign in interface ExtensibleLoadManager

Parameters:

topic - The optional topic, some method won't provide topic var in this param (e.g. NamespaceService#internalGetWebServiceUrl(NamespaceBundle, LookupOptions)), So the topic is optional.

serviceUnit - service unit (e.g. bundle).

options - The lookup options.

Returns:

The broker lookup data.

tryAcquiringOwnership

public CompletableFuture<NamespaceEphemeralData> tryAcquiringOwnership(NamespaceBundle namespaceBundle)

Method to get the current owner of the NamespaceBundle or set the local broker as the owner if absent.

Parameters:

namespaceBundle - the NamespaceBundle

Returns:

The ephemeral node data showing the current ownership info in ServiceUnitStateChannel

selectAsync

public CompletableFuture<Optional<String>> selectAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle,
 Set<String> excludeBrokerSet,
 LookupOptions options)

checkOwnershipAsync

public CompletableFuture<Boolean> checkOwnershipAsync(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic,
 org.apache.pulsar.common.naming.ServiceUnitId bundleUnit)

Description copied from interface: ExtensibleLoadManager

Check the incoming service unit is owned by the current broker.

Specified by:

checkOwnershipAsync in interface ExtensibleLoadManager

Parameters:

topic - The optional topic, some method won't provide topic var in this param.

bundleUnit - The service unit (e.g. bundle).

Returns:

The broker lookup data.

getOwnershipAsync

public CompletableFuture<Optional<String>> getOwnershipAsync(Optional<org.apache.pulsar.common.naming.ServiceUnitId> topic,
 org.apache.pulsar.common.naming.ServiceUnitId serviceUnit)

getOwnershipWithLookupDataAsync

public CompletableFuture<Optional<BrokerLookupData>> getOwnershipWithLookupDataAsync(org.apache.pulsar.common.naming.ServiceUnitId bundleUnit)

unloadNamespaceBundleAsync

public CompletableFuture<Void> unloadNamespaceBundleAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle,
 Optional<String> destinationBroker,
 boolean force,
 long timeout,
 TimeUnit timeoutUnit)

splitNamespaceBundleAsync

public CompletableFuture<Void> splitNamespaceBundleAsync(org.apache.pulsar.common.naming.ServiceUnitId bundle,
 NamespaceBundleSplitAlgorithm splitAlgorithm,
 List<Long> boundaries)

close

public void close()
           throws PulsarServerException

Description copied from interface: ExtensibleLoadManager

Close the load manager.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface ExtensibleLoadManager

Throws:

PulsarServerException - if it fails to stop the load manager.

isInternalTopic

public static boolean isInternalTopic(String topic)

getMetrics

public List<org.apache.pulsar.common.stats.Metrics> getMetrics()

monitor

protected void monitor()

disableBroker

public void disableBroker()
                   throws Exception

Throws:

Exception

createBrokerRegistry

protected BrokerRegistry createBrokerRegistry(PulsarService pulsar)

createServiceUnitStateChannel

protected ServiceUnitStateChannel createServiceUnitStateChannel(PulsarService pulsar)