io.zeebe.logstreams.impl.log.fs

Class FsLogStorage

java.lang.Object

io.zeebe.logstreams.impl.log.fs.FsLogStorage

All Implemented Interfaces:

LogStorage

public class FsLogStorage
extends Object
implements LogStorage

Field Summary

Fields

Modifier and Type	Field and Description
protected FsLogStorageConfiguration	config
protected FsLogSegment	currentSegment
protected ReadResultProcessor	defaultReadResultProcessor
protected int	dirtySegmentId
static org.slf4j.Logger	LOG
protected FsLogSegments	logSegments Readable log segments
protected int	state
protected static int	STATE_CLOSED
protected static int	STATE_CREATED
protected static int	STATE_OPENED

Fields inherited from interface io.zeebe.logstreams.spi.LogStorage

OP_RESULT_BLOCK_SIZE_TOO_BIG, OP_RESULT_INSUFFICIENT_BUFFER_CAPACITY, OP_RESULT_INVALID_ADDR, OP_RESULT_NO_DATA

Constructor Summary

Constructors

Constructor and Description
FsLogStorage(FsLogStorageConfiguration cfg, MetricsManager metricsManager, int partitionId)

Method Summary

Modifier and Type	Method and Description
protected void	addressCheck(int segmentId, int segmentOffset)
long	append(ByteBuffer buffer) Writes a block containing one or multiple log entries in the storage and returns the address at which the block has been written.
protected void	applyTruncatedFileIfExists(File logDir)
protected void	checkConsistency()
void	close() Close the storage.
protected void	deleteBackupFilesIfExist(File logDir)
protected void	ensureNotOpenedStorage()
protected void	ensureOpenedStorage()
void	flush() Flushes all appended blocks to ensure that all blocks are written completely.
FsLogStorageConfiguration	getConfig()
long	getFirstBlockAddress() Returns the address of the first block in the storage or -1 if the storage is currently empty.
protected int	getSegmentId(File file)
protected void	initLogSegments(File logDir)
boolean	isByteAddressable()
boolean	isClosed()
boolean	isOpen()
protected void	markSegmentAsDirty(FsLogSegment segment)
protected void	onSegmentFilled()
void	open() Open the storage.
long	read(ByteBuffer readBuffer, long addr) Naive implementation of the LogStorage.read(ByteBuffer, long, ReadResultProcessor) method.
long	read(ByteBuffer readBuffer, long addr, ReadResultProcessor processor) Reads bytes into the read buffer starting at addr and process the read bytes with the help of the processor.
protected boolean	shouldApplyTruncatedSegment(File logDir, File truncatedFile, int truncatedSegmentId)
void	truncate(long address) Truncates the log up to the given address.
protected void	truncateLogSegment(int segmentId, int size) Creates a truncated backup file of given segment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

STATE_CREATED

protected static final int STATE_CREATED

See Also:

Constant Field Values

STATE_OPENED

protected static final int STATE_OPENED

See Also:

Constant Field Values

STATE_CLOSED

protected static final int STATE_CLOSED

See Also:

Constant Field Values

config

protected final FsLogStorageConfiguration config

defaultReadResultProcessor

protected final ReadResultProcessor defaultReadResultProcessor

logSegments

protected FsLogSegments logSegments

Readable log segments

currentSegment

protected FsLogSegment currentSegment

dirtySegmentId

protected int dirtySegmentId

state

protected volatile int state

Constructor Detail

FsLogStorage

public FsLogStorage(FsLogStorageConfiguration cfg,
                    MetricsManager metricsManager,
                    int partitionId)

Method Detail

isByteAddressable

public boolean isByteAddressable()

Specified by:

isByteAddressable in interface LogStorage

Returns:

true if the storage is byte addressable (each byte managed in the underlying storage can be uniquely addressed using a long addr. False in case the storage is block addressable.

append

public long append(ByteBuffer buffer)

Description copied from interface: LogStorage

Writes a block containing one or multiple log entries in the storage and returns the address at which the block has been written.

Storage implementations must guarantee eventually atomicity. When this method returns, either all the bytes must be written or none at all.

The caller of this method must guarantee that the provided block contains unfragmented log entries.

Specified by:

append in interface LogStorage

Parameters:

buffer - the buffer containing a block of log entries to be written into storage

Returns:

the address at which the block has been written or error status code

onSegmentFilled

protected void onSegmentFilled()

truncate

public void truncate(long address)

Description copied from interface: LogStorage

Truncates the log up to the given address.

Specified by:

truncate in interface LogStorage

Parameters:

address - The address at which to truncate the log.

addressCheck

protected void addressCheck(int segmentId,
                            int segmentOffset)

truncateLogSegment

protected void truncateLogSegment(int segmentId,
                                  int size)

Creates a truncated backup file of given segment.

read

public long read(ByteBuffer readBuffer,
                 long addr)

Description copied from interface: LogStorage

Naive implementation of the LogStorage.read(ByteBuffer, long, ReadResultProcessor) method. Does not process the bytes which are read.

Returns an operation result status code which is either

• positive long representing the next address at which the next block of data can be read
• LogStorage.OP_RESULT_INVALID_ADDR: in case the provided address does not exist
• LogStorage.OP_RESULT_NO_DATA: in case no data is (yet) available at that address
• LogStorage.OP_RESULT_INSUFFICIENT_BUFFER_CAPACITY: in case the storage is block addressable and the provided buffer does not have sufficient capacity to read a whole block

If this method returns with a positive status code, bytes will be written between the given readbuffer's Buffer.position() and Buffer.limit().

This method is invoked concurrently by consumer threads of the log.

Specified by:

read in interface LogStorage

Parameters:

readBuffer - the buffer to read into

addr - the address in the underlying storage from which bytes should be read

Returns:

the next address from which bytes can be read or error status code.

read

public long read(ByteBuffer readBuffer,
                 long addr,
                 ReadResultProcessor processor)

Description copied from interface: LogStorage

Reads bytes into the read buffer starting at addr and process the read bytes with the help of the processor.

Returns an operation result status code which is either

• positive long representing the next address at which the next block of data can be read
• LogStorage.OP_RESULT_INVALID_ADDR: in case the provided address does not exist
• LogStorage.OP_RESULT_NO_DATA: in case no data is (yet) available at that address
• LogStorage.OP_RESULT_INSUFFICIENT_BUFFER_CAPACITY: in case the storage is block addressable and the provided buffer does not have sufficient capacity to read a whole block

If this method returns with a positive status code, bytes will be written between the given readbuffer's Buffer.position() and Buffer.limit().

This method is invoked concurrently by consumer threads of the log.

Specified by:

read in interface LogStorage

Parameters:

readBuffer - the buffer to read into

addr - the address in the underlying storage from which bytes should be read

processor - the processor to process the buffer and the read result

Returns:

the next address from which bytes can be read or error status code.

open

public void open()

Description copied from interface: LogStorage

Open the storage. Called in the log conductor thread.

Specified by:

open in interface LogStorage

initLogSegments

protected void initLogSegments(File logDir)

checkConsistency

protected void checkConsistency()

deleteBackupFilesIfExist

protected void deleteBackupFilesIfExist(File logDir)

applyTruncatedFileIfExists

protected void applyTruncatedFileIfExists(File logDir)

shouldApplyTruncatedSegment

protected boolean shouldApplyTruncatedSegment(File logDir,
                                              File truncatedFile,
                                              int truncatedSegmentId)

close

public void close()

Description copied from interface: LogStorage

Close the storage. Called in the log conductor thread.

Specified by:

close in interface LogStorage

flush

public void flush()
           throws Exception

Description copied from interface: LogStorage

Flushes all appended blocks to ensure that all blocks are written completely. Note that a storage implementation may do nothing if LogStorage.append(ByteBuffer) guarantees that all blocks are written immediately.

Specified by:

flush in interface LogStorage

Throws:

Exception - if fails to flush all blocks

markSegmentAsDirty

protected void markSegmentAsDirty(FsLogSegment segment)

getConfig

public FsLogStorageConfiguration getConfig()

getFirstBlockAddress

public long getFirstBlockAddress()

Description copied from interface: LogStorage

Returns the address of the first block in the storage or -1 if the storage is currently empty.

Specified by:

getFirstBlockAddress in interface LogStorage

ensureOpenedStorage

protected void ensureOpenedStorage()

ensureNotOpenedStorage

protected void ensureNotOpenedStorage()

isOpen

public boolean isOpen()

Specified by:

isOpen in interface LogStorage

Returns:

true, if the storage is open.

isClosed

public boolean isClosed()

Specified by:

isClosed in interface LogStorage

getSegmentId

protected int getSegmentId(File file)