Package io.confluent.parallelconsumer.offsets

Class BitSetEncoder

java.lang.Object

io.confluent.parallelconsumer.offsets.OffsetEncoder

io.confluent.parallelconsumer.offsets.BitSetEncoder

public class BitSetEncoder
extends OffsetEncoder

Encodes a range of offsets, from an incompletes collection into a BitSet.

Highly efficient when the completion status is random.

Highly inefficient when the completion status is in large blocks (RunLengthEncoder is much better)

Because our system works on manipulating INCOMPLETE offsets, it doesn't matter if the offset range we're encoding is Sequential or not. Because as records are always in commit order, if we've seen a range of offsets, we know we've seen all that exist (within said range). So if offset 8 is missing from the partition, we will encode it as having been completed (when in fact it doesn't exist), because we only compare against known incompletes, and assume all others are complete.

So, when we deserialize, the INCOMPLETES collection is then restored, and that's what's used to compare to see if a record should be skipped or not. So if record 8 is recorded as completed, it will be absent from the restored INCOMPLETES list, and we are assured we will never see record 8.

See Also:

RunLengthEncoder, OffsetBitSet

Field Summary

Fields

Modifier and Type	Field	Description
static Integer	MAX_LENGTH_ENCODABLE

Constructor Summary

Constructors

Constructor	Description
BitSetEncoder(int length, OffsetSimultaneousEncoder offsetSimultaneousEncoder)
BitSetEncoder(int length, OffsetSimultaneousEncoder offsetSimultaneousEncoder, OffsetEncoding.Version newVersion)

Method Summary

Modifier and Type	Method	Description
void	encodeCompletedOffset(int index)
void	encodeIncompleteOffset(int index)
BitSet	getBitSet()
protected byte[]	getEncodedBytes()
int	getEncodedSize()
protected OffsetEncoding	getEncodingType()
protected OffsetEncoding	getEncodingTypeCompressed()
byte[]	serialise()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

MAX_LENGTH_ENCODABLE

public static final Integer MAX_LENGTH_ENCODABLE

Constructor Details

BitSetEncoder

public BitSetEncoder(int length, OffsetSimultaneousEncoder offsetSimultaneousEncoder) throws BitSetEncodingNotSupportedException

Throws:

BitSetEncodingNotSupportedException

BitSetEncoder

public BitSetEncoder(int length, OffsetSimultaneousEncoder offsetSimultaneousEncoder, OffsetEncoding.Version newVersion) throws BitSetEncodingNotSupportedException

Parameters:

length - the difference between the highest and lowest offset to be encoded

Throws:

BitSetEncodingNotSupportedException

Method Details

getEncodingType

protected OffsetEncoding getEncodingType()

Specified by:

getEncodingType in class OffsetEncoder

getEncodingTypeCompressed

protected OffsetEncoding getEncodingTypeCompressed()

Specified by:

getEncodingTypeCompressed in class OffsetEncoder

encodeIncompleteOffset

public void encodeIncompleteOffset(int index)

encodeCompletedOffset

public void encodeCompletedOffset(int index)

serialise

public byte[] serialise() throws BitSetEncodingNotSupportedException

Throws:

BitSetEncodingNotSupportedException

getEncodedSize

public int getEncodedSize()

getEncodedBytes

protected byte[] getEncodedBytes()

Specified by:

getEncodedBytes in class OffsetEncoder

getBitSet

public BitSet getBitSet()