Package cz.o2.proxima.beam.direct.io

Class CommitLogRead.BoundedCommitLogReadFn

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<byte[],StreamElement>

cz.o2.proxima.beam.direct.io.CommitLogRead.AbstractCommitLogReadFn

cz.o2.proxima.beam.direct.io.CommitLogRead.BoundedCommitLogReadFn

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

CommitLogRead

@BoundedPerElement
private class CommitLogRead.BoundedCommitLogReadFn
extends CommitLogRead.AbstractCommitLogReadFn

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

org.apache.beam.sdk.transforms.DoFn.AlwaysFetched, org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.BundleFinalizer, org.apache.beam.sdk.transforms.DoFn.Element, org.apache.beam.sdk.transforms.DoFn.FieldAccess, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetInitialWatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.GetSize, org.apache.beam.sdk.transforms.DoFn.GetWatermarkEstimatorStateCoder, org.apache.beam.sdk.transforms.DoFn.Key, org.apache.beam.sdk.transforms.DoFn.MultiOutputReceiver, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.NewWatermarkEstimator, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OnTimerFamily, org.apache.beam.sdk.transforms.DoFn.OnWindowExpiration, org.apache.beam.sdk.transforms.DoFn.OnWindowExpirationContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T extends java.lang.Object>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.RequiresStableInput, org.apache.beam.sdk.transforms.DoFn.RequiresTimeSortedInput, org.apache.beam.sdk.transforms.DoFn.Restriction, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SideInput, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerFamily, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.Timestamp, org.apache.beam.sdk.transforms.DoFn.TruncateRestriction, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.WindowedContext

Field Summary

Fields inherited from class cz.o2.proxima.beam.direct.io.CommitLogRead.AbstractCommitLogReadFn

limit, name, observers, partitionToSeekedOffset, position, readerFactory, repositoryFactory, runningObserves

Constructor Summary

Constructors

Modifier	Constructor	Description
private	BoundedCommitLogReadFn(java.lang.String name, Position position, long limit, RepositoryFactory repositoryFactory, CommitLogReader.Factory<?> readerFactory)

Method Summary

Modifier and Type	Method	Description
org.joda.time.Instant	getInitialWatermarkEstimatorState()
org.apache.beam.sdk.coders.Coder<OffsetRestrictionTracker.OffsetRange>	getRestrictionCoder()
org.apache.beam.sdk.coders.Coder<org.joda.time.Instant>	getWatermarkEstimatorStateCoder()
OffsetRestrictionTracker.OffsetRange	initialRestriction()
org.apache.beam.sdk.transforms.splittabledofn.WatermarkEstimators.Manual	newWatermarkEstimator(org.joda.time.Instant initialWatemark)
(package private) ObserveHandle	observeBulkOffsets(OffsetRestrictionTracker.OffsetRange restriction, CommitLogReader reader, BlockingQueueLogObserver.CommitLogObserver observer)
(package private) ObserveHandle	observeBulkPartitions(java.lang.String name, OffsetRestrictionTracker.OffsetRange restriction, CommitLogReader reader, BlockingQueueLogObserver.CommitLogObserver observer)
void	processBounded(org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<OffsetRestrictionTracker.OffsetRange,Offset> tracker, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<StreamElement> output, org.apache.beam.sdk.transforms.splittabledofn.ManualWatermarkEstimator<?> watermarkEstimator, org.apache.beam.sdk.transforms.DoFn.BundleFinalizer finalizer)
void	setup()
void	splitRestriction(OffsetRestrictionTracker.OffsetRange restriction, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<OffsetRestrictionTracker.OffsetRange> splits)
void	tearDown()

Methods inherited from class cz.o2.proxima.beam.direct.io.CommitLogRead.AbstractCommitLogReadFn

closeHandle, process

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BoundedCommitLogReadFn

private BoundedCommitLogReadFn(@Nullable
                               java.lang.String name,
                               Position position,
                               long limit,
                               RepositoryFactory repositoryFactory,
                               CommitLogReader.Factory<?> readerFactory)

Method Detail

processBounded

@ProcessElement
public void processBounded(org.apache.beam.sdk.transforms.splittabledofn.RestrictionTracker<OffsetRestrictionTracker.OffsetRange,Offset> tracker,
                           org.apache.beam.sdk.transforms.DoFn.OutputReceiver<StreamElement> output,
                           org.apache.beam.sdk.transforms.splittabledofn.ManualWatermarkEstimator<?> watermarkEstimator,
                           org.apache.beam.sdk.transforms.DoFn.BundleFinalizer finalizer)

setup

@Setup
public void setup()

Overrides:

setup in class CommitLogRead.AbstractCommitLogReadFn

tearDown

@Teardown
public void tearDown()

Overrides:

tearDown in class CommitLogRead.AbstractCommitLogReadFn

initialRestriction

@GetInitialRestriction
public OffsetRestrictionTracker.OffsetRange initialRestriction()

splitRestriction

@SplitRestriction
public void splitRestriction(@Restriction
                             OffsetRestrictionTracker.OffsetRange restriction,
                             org.apache.beam.sdk.transforms.DoFn.OutputReceiver<OffsetRestrictionTracker.OffsetRange> splits)

Overrides:

splitRestriction in class CommitLogRead.AbstractCommitLogReadFn

getRestrictionCoder

@GetRestrictionCoder
public org.apache.beam.sdk.coders.Coder<OffsetRestrictionTracker.OffsetRange> getRestrictionCoder()

Overrides:

getRestrictionCoder in class CommitLogRead.AbstractCommitLogReadFn

newWatermarkEstimator

@NewWatermarkEstimator
public org.apache.beam.sdk.transforms.splittabledofn.WatermarkEstimators.Manual newWatermarkEstimator(@WatermarkEstimatorState
                                                                                                      org.joda.time.Instant initialWatemark)

Overrides:

newWatermarkEstimator in class CommitLogRead.AbstractCommitLogReadFn

getInitialWatermarkEstimatorState

@GetInitialWatermarkEstimatorState
public org.joda.time.Instant getInitialWatermarkEstimatorState()

Overrides:

getInitialWatermarkEstimatorState in class CommitLogRead.AbstractCommitLogReadFn

getWatermarkEstimatorStateCoder

@GetWatermarkEstimatorStateCoder
public org.apache.beam.sdk.coders.Coder<org.joda.time.Instant> getWatermarkEstimatorStateCoder()

Overrides:

getWatermarkEstimatorStateCoder in class CommitLogRead.AbstractCommitLogReadFn

observeBulkOffsets

ObserveHandle observeBulkOffsets(OffsetRestrictionTracker.OffsetRange restriction,
                                 CommitLogReader reader,
                                 BlockingQueueLogObserver.CommitLogObserver observer)

Specified by:

observeBulkOffsets in class CommitLogRead.AbstractCommitLogReadFn

observeBulkPartitions

ObserveHandle observeBulkPartitions(java.lang.String name,
                                    OffsetRestrictionTracker.OffsetRange restriction,
                                    CommitLogReader reader,
                                    BlockingQueueLogObserver.CommitLogObserver observer)

Specified by:

observeBulkPartitions in class CommitLogRead.AbstractCommitLogReadFn