@ThreadSafe public class AtomicArrayKVStore extends AbstractKVStore implements AtomicKVStore
AtomicKVStore based on ArrayKVStore with background compaction.
This implementation is designed to maximize the speed of reads and minimize the amount of memory overhead per key/value pair. It is optimized for relatively infrequent writes.
A (read-only) ArrayKVStore is the basis for the database; the array files are mapped into memory. As mutations are
applied, they are added to an in-memory change set, and appended to a mutation log file for persistence. On restart,
the mutation log file (if any) is read to reconstruct the in-memory change set.
Compaction
Instances periodically compact outstanding changes into new array files (and truncate the mutation log file) in a background thread. A compaction is scheduled whenever:
scheduleCompaction() method is invokedIn order to prevent compaction from getting hopelessly behind when there is high write volume, a compaction space high-water mark is also used. When the size of the mutation log file exceeds the half-way point between the low-water and high-water marks, new write attempts start being artificially delayed, and this delay amount increases to infinity as the high-water mark is approached, so that as long as the high-water mark is exceeded, new writes are blocked completely until the current compaction cycle completes.
The number of bytes occupied by the in-memory change set and the length of the outstanding mutations log file are related. Therefore, the compaction high-water mark loosely correlates to a maximum amount of memory required by the in-memory change set.
Hot Backups
"Hot" backups may be created in parallel with normal operation via hotCopy().
Hard links are used to make this operation fast; only the mutation log file (if any) is actually copied.
The database directory is a required configuration property.
Key and value data must not exceed 2GB (each separately).
Instances may be stopped and (re)started multiple times.
| Modifier and Type | Field and Description |
|---|---|
static int |
DEFAULT_COMPACTION_HIGH_WATER
Default compaction space high-water mark in bytes (1073741824 bytes).
|
static int |
DEFAULT_COMPACTION_LOW_WATER
Default compaction space low-water mark in bytes (65536 bytes).
|
static int |
DEFAULT_COMPACTION_MAX_DELAY
Default compaction maximum delay in seconds (90 seconds).
|
| Constructor and Description |
|---|
AtomicArrayKVStore() |
| Modifier and Type | Method and Description |
|---|---|
void |
adjustCounter(byte[] key,
long amount) |
protected long |
calculateCompactionPressureDelay(float windowRatio)
Calculate the maximum amount of time that a thread attempting to
mutate() must wait
for a background compaction to complete as we start nearing the high water mark. |
long |
decodeCounter(byte[] bytes) |
byte[] |
encodeCounter(long value) |
protected void |
finalize()
Finalize this instance.
|
byte[] |
get(byte[] key) |
KVPair |
getAtLeast(byte[] minKey) |
KVPair |
getAtMost(byte[] maxKey) |
File |
getDirectory()
Get the filesystem directory containing the database.
|
Iterator<KVPair> |
getRange(byte[] minKey,
byte[] maxKey,
boolean reverse) |
void |
hotCopy(File target)
Create a filesystem atomic snapshot, or "hot" copy", of this instance in the specified destination directory.
|
void |
mutate(Mutations mutations,
boolean sync) |
void |
put(byte[] key,
byte[] value) |
void |
remove(byte[] key) |
void |
removeRange(byte[] minKey,
byte[] maxKey) |
Future<?> |
scheduleCompaction()
Schedule a new compaction cycle, unless there is one already scheduled or running, or there are no
outstanding uncompacted modifications.
|
void |
setCompactHighWater(int compactHighWater)
Configure the compaction space high-water mark in bytes.
|
void |
setCompactLowWater(int compactLowWater)
Configure the compaction space low-water mark in bytes.
|
void |
setCompactMaxDelay(int compactMaxDelay)
Configure the compaction time maximum delay.
|
void |
setDirectory(File directory)
Configure the filesystem directory containing the database.
|
void |
setScheduledExecutorService(ScheduledExecutorService scheduledExecutorService)
Configure the
ScheduledExecutorService used to schedule background compaction. |
CloseableKVStore |
snapshot() |
void |
start() |
void |
stop() |
String |
toString() |
public static final int DEFAULT_COMPACTION_MAX_DELAY
public static final int DEFAULT_COMPACTION_LOW_WATER
public static final int DEFAULT_COMPACTION_HIGH_WATER
public File getDirectory()
public void setDirectory(File directory)
directory - database directory, or null for noneIllegalStateException - if this instance is already start()edpublic void setScheduledExecutorService(ScheduledExecutorService scheduledExecutorService)
ScheduledExecutorService used to schedule background compaction.
If not explicitly configured, a ScheduledExecutorService will be created automatically during start()
using Executors.newSingleThreadScheduledExecutor() and shutdown by stop() (if explicitly configured here,
the configured ScheduledExecutorService will not be shutdown by stop()).
scheduledExecutorService - schduled executor service, or null to have one created automaticallyIllegalStateException - if this instance is already start()edpublic void setCompactMaxDelay(int compactMaxDelay)
compactMaxDelay - compaction time maximum delay in secondsIllegalArgumentException - if compactMaxDelay is negativeIllegalStateException - if this instance is already start()edpublic void setCompactLowWater(int compactLowWater)
This value is applied to the size of the on-disk modifications file.
compactLowWater - compaction space low-water mark in bytesIllegalArgumentException - if compactLowWater is negativeIllegalStateException - if this instance is already start()edpublic void setCompactHighWater(int compactHighWater)
This value is applied to the size of the on-disk modifications file.
If the compaction space high water mark is set smaller than the the compaction space low water mark, then it's treated as if it were the same as the compaction space low water mark.
compactHighWater - compaction space high-water mark in bytesIllegalArgumentException - if compactHighWater is negativeIllegalStateException - if this instance is already start()ed@PostConstruct public void start()
start in interface AtomicKVStore@PreDestroy public void stop()
stop in interface AtomicKVStorepublic byte[] get(byte[] key)
get in interface KVStoreget in class AbstractKVStorepublic KVPair getAtLeast(byte[] minKey)
getAtLeast in interface KVStoregetAtLeast in class AbstractKVStorepublic KVPair getAtMost(byte[] maxKey)
getAtMost in interface KVStoregetAtMost in class AbstractKVStorepublic void put(byte[] key,
byte[] value)
put in interface KVStoreput in class AbstractKVStorepublic void remove(byte[] key)
remove in interface KVStoreremove in class AbstractKVStorepublic void removeRange(byte[] minKey,
byte[] maxKey)
removeRange in interface KVStoreremoveRange in class AbstractKVStorepublic void adjustCounter(byte[] key,
long amount)
adjustCounter in interface KVStoreadjustCounter in class AbstractKVStorepublic byte[] encodeCounter(long value)
encodeCounter in interface KVStoreencodeCounter in class AbstractKVStorepublic long decodeCounter(byte[] bytes)
decodeCounter in interface KVStoredecodeCounter in class AbstractKVStorepublic CloseableKVStore snapshot()
snapshot in interface AtomicKVStorepublic void mutate(Mutations mutations, boolean sync)
mutate in interface AtomicKVStorepublic void hotCopy(File target) throws IOException
The target directory will be created if it does not exist; otherwise, it must be empty.
All files except the uncompacted modifications file are copied into target
in constant time using hard links if possible.
The hot copy operation can proceed in parallel with normal database activity, with the exception that the compaction operation cannot start while there are concurrent hot copies in progress.
Therefore, for best performance, consider performing a compaction immediately prior to this operation.
Note: when this method returns, all copied files, and the target directory, have been fsync()'d
and therefore may be considered durable in case of a system crash.
target - destination directoryIOException - if an I/O error occursIllegalArgumentException - if target exists and is not a directory or is non-emptyIllegalArgumentException - if target is nullprotected long calculateCompactionPressureDelay(float windowRatio)
mutate() must wait
for a background compaction to complete as we start nearing the high water mark.
The implementation in AtomicArrayKVStore returns -1 for all values below 50%,
and then scales according to an inverse power function returning zero at 50%, 100ms at 75%,
and Long.MAX_VALUE at 100%.
windowRatio - the uncompacted mutation size as a fraction of the interval between low and high
water marks; always a value value between zero and one (exclusive)public Future<?> scheduleCompaction()
IllegalStateException - if this instance is not startedprotected void finalize()
throws Throwable
stop() to close any unclosed iterators.Copyright © 2016. All rights reserved.