Package com.networknt.kafka.common
Class TransactionalIdsGenerator
- java.lang.Object
-
- com.networknt.kafka.common.TransactionalIdsGenerator
-
public class TransactionalIdsGenerator extends Object
Class responsible for generating transactional ids to use when communicating with Kafka.It guarantees that:
- generated ids to use will never clash with ids to use from different subtasks
- generated ids to abort will never clash with ids to abort from different subtasks
- generated ids to use will never clash with ids to abort from different subtasks
-
-
Constructor Summary
Constructors Constructor Description TransactionalIdsGenerator(String prefix, int subtaskIndex, int totalNumberOfSubtasks, int poolSize, int safeScaleDownFactor)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description Set<String>generateIdsToAbort()If we have to abort previous transactional id in case of restart after a failure BEFORE first checkpoint completed, we don't know what was the parallelism used in previous attempt.Set<String>generateIdsToUse(long nextFreeTransactionalId)Range of available transactional ids to use is: [nextFreeTransactionalId, nextFreeTransactionalId + parallelism * kafkaProducersPoolSize) loop below picks in a deterministic way a subrange of those available transactional ids based on index of this subtask.
-
-
-
Constructor Detail
-
TransactionalIdsGenerator
public TransactionalIdsGenerator(String prefix, int subtaskIndex, int totalNumberOfSubtasks, int poolSize, int safeScaleDownFactor)
-
-
Method Detail
-
generateIdsToUse
public Set<String> generateIdsToUse(long nextFreeTransactionalId)
Range of available transactional ids to use is: [nextFreeTransactionalId, nextFreeTransactionalId + parallelism * kafkaProducersPoolSize) loop below picks in a deterministic way a subrange of those available transactional ids based on index of this subtask.- Parameters:
nextFreeTransactionalId- next free transactional id- Returns:
- Set
-
generateIdsToAbort
public Set<String> generateIdsToAbort()
If we have to abort previous transactional id in case of restart after a failure BEFORE first checkpoint completed, we don't know what was the parallelism used in previous attempt. In that case we must guess the ids range to abort based on current configured pool size, current parallelism and safeScaleDownFactor.- Returns:
- Set
-
-