class WindowedStream[T, K, W <: Window] extends AnyRef
A WindowedStream represents a data stream where elements are grouped by key, and for each key, the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.
The windows are conceptually evaluated for each key individually, meaning windows can trigger at different points for each key.
If an org.apache.flink.streaming.api.windowing.evictors.Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.
Note that the WindowedStream is purely and API construct, during runtime the WindowedStream will be collapsed together with the KeyedStream and the operation over the window into one single operation.
- T
The type of elements in the stream.
- K
The type of the key by which elements are grouped.
- W
The type of Window that the org.apache.flink.streaming.api.windowing.assigners.WindowAssigner assigns the elements to.
- Annotations
- @Public()
- Alphabetic
- By Inheritance
- WindowedStream
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new WindowedStream(javaStream: flink.streaming.api.datastream.WindowedStream[T, K, W])
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def aggregate(aggregationType: AggregationType, position: Int): DataStream[T]
-
def
aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: ProcessWindowFunction[V, R, K, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given aggregation function.
- preAggregator
The aggregation function that is used for pre-aggregation
- windowFunction
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: (K, W, Iterable[V], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given aggregation function.
- preAggregator
The aggregation function that is used for pre-aggregation
- windowFunction
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
aggregate[ACC, V, R](preAggregator: AggregateFunction[T, ACC, V], windowFunction: WindowFunction[V, R, K, W])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[V], arg2: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given aggregation function.
- preAggregator
The aggregation function that is used for pre-aggregation
- windowFunction
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
aggregate[ACC, R](aggregateFunction: AggregateFunction[T, ACC, R])(implicit arg0: TypeInformation[ACC], arg1: TypeInformation[R]): DataStream[R]
Applies the given aggregation function to each window and key.
Applies the given aggregation function to each window and key. The aggregation function is called for each element, aggregating values incrementally and keeping the state to one accumulator per key and window.
- aggregateFunction
The aggregation function.
- returns
The data stream that is the result of applying the fold function to the window.
- Annotations
- @PublicEvolving()
-
def
allowedLateness(lateness: Time): WindowedStream[T, K, W]
Sets the allowed lateness to a user-specified value.
Sets the allowed lateness to a user-specified value. If not explicitly set, the allowed lateness is 0L. Setting the allowed lateness is only valid for event-time windows. If a value different than 0 is provided with a processing-time org.apache.flink.streaming.api.windowing.assigners.WindowAssigner, then an exception is thrown.
- Annotations
- @PublicEvolving()
-
def
apply[R](function: (K, W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.
- function
The window function.
- returns
The data stream that is the result of applying the window function to the window.
-
def
apply[R](function: WindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.
- function
The window function.
- returns
The data stream that is the result of applying the window function to the window.
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
evictor(evictor: Evictor[_ >: T, _ >: W]): WindowedStream[T, K, W]
Sets the Evictor that should be used to evict elements from a window before emission.
Sets the Evictor that should be used to evict elements from a window before emission.
Note: When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.
- Annotations
- @PublicEvolving()
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
max(field: String): DataStream[T]
Applies an aggregation that that gives the maximum of the elements in the window at the given field.
-
def
max(position: Int): DataStream[T]
Applies an aggregation that that gives the maximum of the elements in the window at the given position.
-
def
maxBy(field: String): DataStream[T]
Applies an aggregation that that gives the maximum element of the window by the given field.
Applies an aggregation that that gives the maximum element of the window by the given field. When equality, returns the first.
-
def
maxBy(position: Int): DataStream[T]
Applies an aggregation that that gives the maximum element of the window by the given position.
Applies an aggregation that that gives the maximum element of the window by the given position. When equality, returns the first.
-
def
min(field: String): DataStream[T]
Applies an aggregation that that gives the minimum of the elements in the window at the given field.
-
def
min(position: Int): DataStream[T]
Applies an aggregation that that gives the minimum of the elements in the window at the given position.
-
def
minBy(field: String): DataStream[T]
Applies an aggregation that that gives the minimum element of the window by the given field.
Applies an aggregation that that gives the minimum element of the window by the given field. When equality, returns the first.
-
def
minBy(position: Int): DataStream[T]
Applies an aggregation that that gives the minimum element of the window by the given position.
Applies an aggregation that that gives the minimum element of the window by the given position. When equality, returns the first.
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
process[R](function: ProcessWindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.
- function
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
reduce[R](preAggregator: ReduceFunction[T], function: ProcessWindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given reduce function to each window.
Applies the given reduce function to each window. The window reduced value is then passed as input of the window function. The output of the window function is interpreted as a regular non-windowed stream.
- preAggregator
The reduce function that is used for pre-aggregation
- function
The process window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
reduce[R](preAggregator: (T, T) ⇒ T, function: ProcessWindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given reduce function to each window.
Applies the given reduce function to each window. The window reduced value is then passed as input of the window function. The output of the window function is interpreted as a regular non-windowed stream.
- preAggregator
The reduce function that is used for pre-aggregation
- function
The process window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @PublicEvolving()
-
def
reduce[R](preAggregator: (T, T) ⇒ T, windowFunction: (K, W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given pre-aggregation reducer.
- preAggregator
The reduce function that is used for pre-aggregation
- windowFunction
The window function.
- returns
The data stream that is the result of applying the window function to the window.
-
def
reduce[R](preAggregator: ReduceFunction[T], function: WindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given pre-aggregation reducer.
- preAggregator
The reduce function that is used for pre-aggregation
- function
The window function.
- returns
The data stream that is the result of applying the window function to the window.
-
def
reduce(function: (T, T) ⇒ T): DataStream[T]
Applies a reduce function to the window.
Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.
This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.
- function
The reduce function.
- returns
The data stream that is the result of applying the reduce function to the window.
-
def
reduce(function: ReduceFunction[T]): DataStream[T]
Applies a reduce function to the window.
Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.
This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.
- function
The reduce function.
- returns
The data stream that is the result of applying the reduce function to the window.
-
def
sideOutputLateData(outputTag: OutputTag[T]): WindowedStream[T, K, W]
Send late arriving data to the side output identified by the given OutputTag.
Send late arriving data to the side output identified by the given OutputTag. Data is considered late after the watermark has passed the end of the window plus the allowed lateness set using allowedLateness(Time).
You can get the stream of late data using DataStream.getSideOutput() on the DataStream resulting from the windowed operation with the same OutputTag.
- Annotations
- @PublicEvolving()
-
def
sum(field: String): DataStream[T]
Applies an aggregation that sums the elements in the window at the given field.
-
def
sum(position: Int): DataStream[T]
Applies an aggregation that sums the elements in the window at the given position.
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
trigger(trigger: Trigger[_ >: T, _ >: W]): WindowedStream[T, K, W]
Sets the Trigger that should be used to trigger window emission.
Sets the Trigger that should be used to trigger window emission.
- Annotations
- @PublicEvolving()
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
Deprecated Value Members
-
def
apply[R](preAggregator: (T, T) ⇒ T, windowFunction: (K, W, Iterable[T], Collector[R]) ⇒ Unit)(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given pre-aggregation reducer.
- preAggregator
The reduce function that is used for pre-aggregation
- windowFunction
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @deprecated
- Deprecated
-
def
apply[R](preAggregator: ReduceFunction[T], function: WindowFunction[T, R, K, W])(implicit arg0: TypeInformation[R]): DataStream[R]
Applies the given window function to each window.
Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.
Arriving data is pre-aggregated using the given pre-aggregation reducer.
- preAggregator
The reduce function that is used for pre-aggregation
- function
The window function.
- returns
The data stream that is the result of applying the window function to the window.
- Annotations
- @deprecated
- Deprecated
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated
- Deprecated