org.aksw.jena_sparql_api.io.binseach

Class BufferFromInputStream

java.lang.Object

org.aksw.jena_sparql_api.io.binseach.BufferFromInputStream

All Implemented Interfaces:

AutoCloseable, ChannelFactory<Seekable>

@ThreadSafe
public class BufferFromInputStream
extends Object
implements ChannelFactory<Seekable>

Implementation of a byte array that caches data in buckets from an InputStream. Instances of these class are thread safe, but the obtained channels are not; each channel should only be operated on by one thread. Differences to BufferedInputStream - this class caches all data read from the inputstream hence there is no mark / reset mechanism - buffer is split into buckets (no data copying required when allocating more space) - data is loaded on demand based on (possibly concurrent) requests to the seekable channels obtained with newChannel() Closest known-to-me Hadoop counterpart is BufferedFSInputStream (which is based on BufferedInputStream)

Author:

raven

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BufferFromInputStream.BucketPointer
class	BufferFromInputStream.ByteArrayChannel

Field Summary

Fields

Modifier and Type	Field and Description
protected BufferFromInputStream.BucketPointer	activeEnd End marker with two components (idx, pos) it is wrapped in an object to enable atomic replacement of the reference The pointer is monotonous in the sense that the end marker's logical linear location is only increased Reading an old version while a new one has been set will only cause a read to return on the old boundary, but a subsequent synchronized check for whether loading of additional data is needed is then made anyway
protected byte[][]	buckets The buffered data
protected InputStream	dataSupplier Supplier for additional data
protected boolean	isDataSupplierConsumed Flag to indicate that the dataSupplier has been consumed This is the case when dataSupplier(buffer) returns -1
protected long	knownDataSize The number of cached bytes.
protected int	maxReadSize Maximum number to read from the dataSupplier in one request
protected int	minReadSize

Constructor Summary

Constructors

Constructor and Description
BufferFromInputStream(int initialBucketSize, InputStream dataSupplier)

Method Summary

Modifier and Type	Method and Description
void	close()
static BufferFromInputStream	create(InputStream in, int maxReadSize, int... preconfiguredBucketSizes)
int	doRead(BufferFromInputStream.ByteArrayChannel reader, ByteBuffer dst)
protected void	ensureCapacityInActiveBucket()
static BufferFromInputStream.BucketPointer	getPointer(byte[][] buckets, BufferFromInputStream.BucketPointer end, long pos)
static long	getPosition(byte[][] buckets, int idx, int pos)
protected void	loadData(int needed) fetch a chunk from the input stream
protected void	loadDataUpTo(long requestedPos)
static void	main(String[] args)
Seekable	newChannel()
protected int	nextBucketSize()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

buckets

protected byte[][] buckets

The buffered data

activeEnd

protected BufferFromInputStream.BucketPointer activeEnd

End marker with two components (idx, pos) it is wrapped in an object to enable atomic replacement of the reference The pointer is monotonous in the sense that the end marker's logical linear location is only increased Reading an old version while a new one has been set will only cause a read to return on the old boundary, but a subsequent synchronized check for whether loading of additional data is needed is then made anyway

knownDataSize

protected long knownDataSize

The number of cached bytes. Corresponds to the linear representation of activeEnd.

dataSupplier

protected InputStream dataSupplier

Supplier for additional data

minReadSize

protected int minReadSize

maxReadSize

protected int maxReadSize

Maximum number to read from the dataSupplier in one request

isDataSupplierConsumed

protected boolean isDataSupplierConsumed

Flag to indicate that the dataSupplier has been consumed This is the case when dataSupplier(buffer) returns -1

Constructor Detail

BufferFromInputStream

public BufferFromInputStream(int initialBucketSize,
                             InputStream dataSupplier)

Method Detail

create

public static BufferFromInputStream create(InputStream in,
                                           int maxReadSize,
                                           int... preconfiguredBucketSizes)

Parameters:

maxReadSize - Maximum number of bytes to request form the input stream at once

in -

maxReadSize -

preconfiguredBucketSizes -

Returns:

getPosition

public static long getPosition(byte[][] buckets,
                               int idx,
                               int pos)

getPointer

public static BufferFromInputStream.BucketPointer getPointer(byte[][] buckets,
                                                             BufferFromInputStream.BucketPointer end,
                                                             long pos)

Parameters:

buckets -

pos -

Returns:

Pointer to a valid location in the know data block or null

newChannel

public Seekable newChannel()

Specified by:

newChannel in interface ChannelFactory<Seekable>

nextBucketSize

protected int nextBucketSize()

doRead

public int doRead(BufferFromInputStream.ByteArrayChannel reader,
                  ByteBuffer dst)

loadDataUpTo

protected void loadDataUpTo(long requestedPos)

loadData

protected void loadData(int needed)

fetch a chunk from the input stream

ensureCapacityInActiveBucket

protected void ensureCapacityInActiveBucket()

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception