cz.o2.proxima.direct.time

Class UnboundedOutOfOrdernessWatermarkEstimator

java.lang.Object

cz.o2.proxima.time.AbstractWatermarkEstimator

cz.o2.proxima.direct.time.UnboundedOutOfOrdernessWatermarkEstimator

All Implemented Interfaces:

TimeProvider, WatermarkEstimator, WatermarkSupplier, java.io.Serializable

@Internal
public class UnboundedOutOfOrdernessWatermarkEstimator
extends AbstractWatermarkEstimator

Estimator of watermark based on timestamps of flowing elements. The estimator tries to estimate when it reaches near-realtime consumption of events and holds the watermark until then.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	UnboundedOutOfOrdernessWatermarkEstimator.Builder Builder of the UnboundedOutOfOrdernessWatermarkEstimator.
static class	UnboundedOutOfOrdernessWatermarkEstimator.Factory

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ALLOWED_TIMESTAMP_SKEW
private long	allowedTimestampSkew
static long	DEFAULT_ALLOWED_TIMESTAMP_SKEW
static long	DEFAULT_ESTIMATE_DURATION_MS
(package private) static long	DEFAULT_MIN_WATERMARK
static long	DEFAULT_STEP_MS
static java.lang.String	ESTIMATE_DURATION_MS
private long	estimateDurationMs
private java.util.concurrent.atomic.AtomicLong	lastRotate
private java.util.concurrent.atomic.AtomicLong	lastStatLogged
static java.lang.String	MIN_WATERMARK
private java.util.concurrent.atomic.AtomicInteger	rotatesToInitialize
private static long	serialVersionUID
static java.lang.String	STEP_MS
private long[]	stepDiffs
private long	stepMs
private TimestampSupplier	timestampSupplier
private java.util.concurrent.atomic.AtomicLong	watermark

Constructor Summary

Constructors

Constructor and Description
UnboundedOutOfOrdernessWatermarkEstimator(long estimateDurationMs, long stepMs, long allowedTimestampSkew, long minWatermark, TimestampSupplier supplier, WatermarkIdlePolicy idlePolicy)

Method Summary

Modifier and Type	Method and Description
(package private) void	add(long stamp) Accumulate given timestamp at current processing time.
protected long	estimateWatermark()
static UnboundedOutOfOrdernessWatermarkEstimator.Builder	newBuilder()
private void	rotate(long now, int moveCount)
private void	rotateIfNeeded()
void	setMinWatermark(long minWatermark)
void	updateWatermark(StreamElement element)

Methods inherited from class cz.o2.proxima.time.AbstractWatermarkEstimator

getWatermark, idle, update

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface cz.o2.proxima.time.WatermarkSupplier

getCurrentTime

Methods inherited from interface cz.o2.proxima.functional.TimeProvider

processingTime

Field Detail

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

DEFAULT_MIN_WATERMARK

static final long DEFAULT_MIN_WATERMARK

See Also:

Constant Field Values

MIN_WATERMARK

public static final java.lang.String MIN_WATERMARK

See Also:

Constant Field Values

ESTIMATE_DURATION_MS

public static final java.lang.String ESTIMATE_DURATION_MS

See Also:

Constant Field Values

STEP_MS

public static final java.lang.String STEP_MS

See Also:

Constant Field Values

ALLOWED_TIMESTAMP_SKEW

public static final java.lang.String ALLOWED_TIMESTAMP_SKEW

See Also:

Constant Field Values

DEFAULT_ESTIMATE_DURATION_MS

public static final long DEFAULT_ESTIMATE_DURATION_MS

See Also:

Constant Field Values

DEFAULT_STEP_MS

public static final long DEFAULT_STEP_MS

See Also:

Constant Field Values

DEFAULT_ALLOWED_TIMESTAMP_SKEW

public static final long DEFAULT_ALLOWED_TIMESTAMP_SKEW

See Also:

Constant Field Values

stepMs

private final long stepMs

estimateDurationMs

private final long estimateDurationMs

allowedTimestampSkew

private final long allowedTimestampSkew

timestampSupplier

private final TimestampSupplier timestampSupplier

stepDiffs

private final long[] stepDiffs

lastRotate

private final java.util.concurrent.atomic.AtomicLong lastRotate

rotatesToInitialize

private final java.util.concurrent.atomic.AtomicInteger rotatesToInitialize

watermark

private final java.util.concurrent.atomic.AtomicLong watermark

lastStatLogged

private final java.util.concurrent.atomic.AtomicLong lastStatLogged

Constructor Detail

UnboundedOutOfOrdernessWatermarkEstimator

UnboundedOutOfOrdernessWatermarkEstimator(long estimateDurationMs,
                                          long stepMs,
                                          long allowedTimestampSkew,
                                          long minWatermark,
                                          TimestampSupplier supplier,
                                          WatermarkIdlePolicy idlePolicy)

Method Detail

newBuilder

public static UnboundedOutOfOrdernessWatermarkEstimator.Builder newBuilder()

estimateWatermark

protected long estimateWatermark()

Specified by:

estimateWatermark in class AbstractWatermarkEstimator

updateWatermark

public void updateWatermark(StreamElement element)

Specified by:

updateWatermark in class AbstractWatermarkEstimator

setMinWatermark

public void setMinWatermark(long minWatermark)

add

void add(long stamp)

Accumulate given timestamp at current processing time.

Parameters:

stamp - the stamp to accumulate

rotateIfNeeded

private void rotateIfNeeded()

rotate

private void rotate(long now,
                    int moveCount)