Package org.apache.pulsar.compaction

Class StrategicTwoPhaseCompactor

java.lang.Object

org.apache.pulsar.compaction.Compactor

org.apache.pulsar.compaction.AbstractTwoPhaseCompactor<org.apache.pulsar.client.api.MessageId>

org.apache.pulsar.compaction.PublishingOrderCompactor

org.apache.pulsar.compaction.StrategicTwoPhaseCompactor

public class StrategicTwoPhaseCompactor
extends PublishingOrderCompactor

Compaction will go through the topic in two passes. The first pass selects valid message(defined in the TopicCompactionStrategy.isValid()) for each key in the topic. Then, the second pass writes these values to a ledger.

As the first pass caches the entire message(not just offset) for each key into a map, this compaction could be memory intensive if the message payload is large.

Field Summary

Fields inherited from class org.apache.pulsar.compaction.AbstractTwoPhaseCompactor

topicCompactionRetainNullKey

Fields inherited from class org.apache.pulsar.compaction.Compactor

bk, COMPACTED_TOPIC_LEDGER_PROPERTY, COMPACTION_SUBSCRIPTION, conf, mxBean, pulsar, scheduler

Constructor Summary

Constructors

Constructor	Description
StrategicTwoPhaseCompactor(ServiceConfiguration conf, org.apache.pulsar.client.api.PulsarClient pulsar, BookKeeper bk, ScheduledExecutorService scheduler)

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Long>	compact(String topic)
<T> CompletableFuture<Long>	compact(String topic, org.apache.pulsar.common.topics.TopicCompactionStrategy<T> strategy)
<T> CompletableFuture<Long>	compact(String topic, org.apache.pulsar.common.topics.TopicCompactionStrategy<T> strategy, org.apache.pulsar.client.api.CryptoKeyReader cryptoKeyReader)

Methods inherited from class org.apache.pulsar.compaction.PublishingOrderCompactor

compactBatchMessage, compactMessage, extractIdsAndKeysAndSizeFromBatch, toLatestMessageIdForKey

Methods inherited from class org.apache.pulsar.compaction.AbstractTwoPhaseCompactor

closeLedger, createLedger, deleteLedger, doCompaction, extractKeyAndSize, getPhaseOneLoopReadTimeoutInSeconds, rebatchMessage

Methods inherited from class org.apache.pulsar.compaction.Compactor

getStats

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

StrategicTwoPhaseCompactor

public StrategicTwoPhaseCompactor(ServiceConfiguration conf,
 org.apache.pulsar.client.api.PulsarClient pulsar,
 BookKeeper bk,
 ScheduledExecutorService scheduler)

Method Details

compact

public CompletableFuture<Long> compact(String topic)

Overrides:

compact in class Compactor

compact

public <T> CompletableFuture<Long> compact(String topic,
 org.apache.pulsar.common.topics.TopicCompactionStrategy<T> strategy)

compact

public <T> CompletableFuture<Long> compact(String topic,
 org.apache.pulsar.common.topics.TopicCompactionStrategy<T> strategy,
 org.apache.pulsar.client.api.CryptoKeyReader cryptoKeyReader)