Package io.confluent.parallelconsumer.offsets

Class OffsetMapCodecManager<K,V>

java.lang.Object

io.confluent.parallelconsumer.offsets.OffsetMapCodecManager<K,V>

public class OffsetMapCodecManager<K,V>
extends Object

Uses multiple encodings to compare, when decided, can refactor other options out for analysis only - encodeOffsetsCompressed(long, io.confluent.parallelconsumer.state.PartitionState<K, V>)

TODO: consider IO exception management - question sneaky throws usage?

TODO: enforce max uncommitted < encoding length (Short.MAX)

Bitset serialisation format:

• byte1: magic
• byte2-3: Short: bitset size
• byte4-n: serialised BitSet

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	OffsetMapCodecManager.HighestOffsetAndIncompletes	Decoding result for encoded offsets

Field Summary

Fields

Modifier and Type	Field	Description
static Charset	CHARSET_TO_USE
static int	DefaultMaxMetadataSize	Maximum size of the commit offset metadata
static Optional<OffsetEncoding>	forcedCodec	Forces the use of a specific codec, instead of choosing the most efficient one.
static String	METADATA_DATA_SIZE_RESOURCE_LOCK	Used to prevent tests running in parallel that depends on setting static state in this class.

Constructor Summary

Constructors

Constructor	Description
OffsetMapCodecManager(org.apache.kafka.clients.consumer.Consumer<K,V> consumer)

Method Summary

Modifier and Type	Method	Description
static OffsetMapCodecManager.HighestOffsetAndIncompletes	deserialiseIncompleteOffsetMapFromBase64(long committedOffsetForPartition, String base64EncodedOffsetPayload)
Map<org.apache.kafka.common.TopicPartition,PartitionState<K,V>>	loadPartitionStateForAssignment(Collection<org.apache.kafka.common.TopicPartition> assignment)	Load all the previously completed offsets that were not committed
String	makeOffsetMetadataPayload(long finalOffsetForPartition, PartitionState<K,V> state)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

METADATA_DATA_SIZE_RESOURCE_LOCK

public static final String METADATA_DATA_SIZE_RESOURCE_LOCK

Used to prevent tests running in parallel that depends on setting static state in this class. Manipulation of static state in tests needs to be removed to this isn't necessary.

todo remove static state manipulation from tests (make non static)

See Also:

Constant Field Values

DefaultMaxMetadataSize

public static int DefaultMaxMetadataSize

Maximum size of the commit offset metadata

See Also:

OffsetConfig#DefaultMaxMetadataSize, "kafka.coordinator.group.OffsetConfig#DefaultMaxMetadataSize"

CHARSET_TO_USE

public static final Charset CHARSET_TO_USE

forcedCodec

public static Optional<OffsetEncoding> forcedCodec

Forces the use of a specific codec, instead of choosing the most efficient one. Useful for testing.

Constructor Details

OffsetMapCodecManager

public OffsetMapCodecManager(org.apache.kafka.clients.consumer.Consumer<K,V> consumer)

Method Details

loadPartitionStateForAssignment

public Map<org.apache.kafka.common.TopicPartition,PartitionState<K,V>> loadPartitionStateForAssignment(Collection<org.apache.kafka.common.TopicPartition> assignment)

Load all the previously completed offsets that were not committed

deserialiseIncompleteOffsetMapFromBase64

public static OffsetMapCodecManager.HighestOffsetAndIncompletes deserialiseIncompleteOffsetMapFromBase64(long committedOffsetForPartition, String base64EncodedOffsetPayload) throws OffsetDecodingError

Throws:

OffsetDecodingError

makeOffsetMetadataPayload

public String makeOffsetMetadataPayload(long finalOffsetForPartition, PartitionState<K,V> state) throws NoEncodingPossibleException

Throws:

NoEncodingPossibleException