Package org.apache.pulsar.broker.transaction.buffer

Interface TransactionBuffer

All Known Implementing Classes:
TopicTransactionBuffer, TransactionBufferDisable
@Beta public interface TransactionBuffer
A class represent a transaction buffer. The transaction buffer is per partition. All the messages published within transactions are appended to a transaction buffer. They are not visible to consumers or readers before the transaction is committed.

When committing transaction starts, the broker will append a `COMMITTED` marker to the data partition first to mark the transaction is committed. The broker knows the data ledger of the commit marker and calls commitTxn(TxnID, long) to commit and seal the buffer.

When the marker is appended to the data partition, all the entries are visible to the consumers. So a transaction reader TransactionBufferReader will be opened to read the entries when the broker sees the commit marker. There is a chance broker crashes after writing the marker to data partition but before committing the transaction in transaction buffer. That is fine. Because the transaction buffer will fail opening the transaction buffer reader since the transaction is still marked as open. The broker can keep retry until the TC (transaction coordinator) eventually commits the buffer again.

  • Method Details

    • getTransactionMeta

      CompletableFuture<TransactionMeta> getTransactionMeta(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Return the metadata of a transaction in the buffer.
      Parameters:
      txnID - the transaction id
      Returns:
      a future represents the result of the operation
      Throws:
      TransactionBufferException.TransactionNotFoundException - if the transaction is not in the buffer.

    • appendBufferToTxn

      CompletableFuture<Position> appendBufferToTxn(org.apache.pulsar.client.api.transaction.TxnID txnId, long sequenceId, io.netty.buffer.ByteBuf buffer)
      Append the buffer to the transaction buffer.

      The entry will be indexed by txnId and sequenceId.

      Parameters:
      txnId - the transaction id
      sequenceId - the sequence id of the entry in this transaction buffer.
      buffer - the entry buffer
      Returns:
      a future represents the result of the operation.
      Throws:
      TransactionException.TransactionSealedException - if the transaction has been sealed.

    • openTransactionBufferReader

      CompletableFuture<TransactionBufferReader> openTransactionBufferReader(org.apache.pulsar.client.api.transaction.TxnID txnID, long startSequenceId)
      Open a TransactionBufferReader to read entries of a given transaction starting from the provided sequenceId.
      Parameters:
      txnID - transaction id
      startSequenceId - the sequence id to start read
      Returns:
      a future represents the result of open operation.
      Throws:
      TransactionBufferException.TransactionNotFoundException - if the transaction is not in the buffer.

    • commitTxn

      CompletableFuture<Void> commitTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Commit the transaction and seal the buffer for this transaction.

      If a transaction is sealed, no more entries can be appendBufferToTxn(TxnID, long, ByteBuf).

      Parameters:
      txnID - the transaction id
      lowWaterMark - the low water mark of this transaction
      Returns:
      a future represents the result of commit operation.
      Throws:
      TransactionBufferException.TransactionNotFoundException - if the transaction is not in the buffer.

    • abortTxn

      CompletableFuture<Void> abortTxn(org.apache.pulsar.client.api.transaction.TxnID txnID, long lowWaterMark)
      Abort the transaction and all the entries of this transaction will be discarded.
      Parameters:
      txnID - the transaction id
      lowWaterMark - the low water mark of this transaction
      Returns:
      a future represents the result of abort operation.
      Throws:
      TransactionBufferException.TransactionNotFoundException - if the transaction is not in the buffer.

    • purgeTxns

      CompletableFuture<Void> purgeTxns(List<Long> dataLedgers)
      Purge all the data of the transactions who are committed and stored in the provided data ledgers.

      This method will be called by the broker before they delete the ledgers. It ensures that all the transactions committed in those ledgers can be purged.

      Parameters:
      dataLedgers - the list of data ledgers.
      Returns:
      a future represents the result of purge operations.

    • clearSnapshot

      CompletableFuture<Void> clearSnapshot()
      Clear up the snapshot of the TransactionBuffer.
      Returns:
      Clear up operation result.

    • closeAsync

      CompletableFuture<Void> closeAsync()
      Close the buffer asynchronously.
      Returns:

    • isTxnAborted

      boolean isTxnAborted(org.apache.pulsar.client.api.transaction.TxnID txnID, Position readPosition)
      Check if the txn is aborted. TODO: To avoid broker oom, we will load the aborted txn from snapshot on demand. So we need the readPosition to check if the txn is loaded.
      Parameters:
      txnID - TxnID txnId.
      readPosition - the persistent position of the txn message.
      Returns:
      whether the txn is aborted.

    • syncMaxReadPositionForNormalPublish

      void syncMaxReadPositionForNormalPublish(Position position, boolean isMarkerMessage)
      Sync max read position for normal publish.
      Parameters:
      position - Position the position to sync.
      isMarkerMessage - whether the message is marker message.

    • getMaxReadPosition

      Position getMaxReadPosition()
      Get the can read max position.
      Returns:
      the stable position.

    • getSnapshotType

      AbortedTxnProcessor.SnapshotType getSnapshotType()
      Get the snapshot type. The snapshot type can be either "Single" or "Segment". In "Single" mode, a single snapshot log is used to record the transaction buffer stats. In "Segment" mode, a snapshot segment topic is used to record the stats, and a separate snapshot segment index topic is used to index these stats.
      Returns:
      the snapshot type

    • getTransactionInBufferStats

      org.apache.pulsar.common.policies.data.TransactionInBufferStats getTransactionInBufferStats(org.apache.pulsar.client.api.transaction.TxnID txnID)
      Get transaction in buffer stats.
      Returns:
      the transaction in buffer stats.

    • getStats

      org.apache.pulsar.common.policies.data.TransactionBufferStats getStats(boolean lowWaterMarks, boolean segmentStats)
      Get transaction stats in buffer.
      Returns:
      the transaction stats in buffer.

    • getStats

      org.apache.pulsar.common.policies.data.TransactionBufferStats getStats(boolean lowWaterMarks)
      Get transaction stats in buffer.
      Returns:
      the transaction stats in buffer.

    • checkIfTBRecoverCompletely

      CompletableFuture<Void> checkIfTBRecoverCompletely()
      Wait TransactionBuffer recovers completely.
      Returns:
      a future that will be completed after the transaction buffer recover completely.

    • getOngoingTxnCount

      long getOngoingTxnCount()

    • getAbortedTxnCount

      long getAbortedTxnCount()

    • getCommittedTxnCount

      long getCommittedTxnCount()