-
- All Implemented Interfaces:
-
java.io.Serializable,org.apache.beam.sdk.transforms.display.HasDisplayData
public final class MapProcessContextFn<InputT extends Object, OutputT extends Object> extends BaseElementFn<InputT, OutputT>
This class allows to handle a generic and custom org.apache.beam.sdk.transforms.DoFn for map operation with error handling.
<br></br>
This class is based on an input class and output type descriptor and take a org.apache.beam.sdk.transforms.SerializableFunction to execute the mapping treatment lazily. These types allow to give type information and handle default coders. This function is from org.apache.beam.sdk.transforms.DoFn.ProcessContext object to the output type. In some case, developers need to access to ProcessContext, to get technical data (timestamp...) or handle side inputs. This function is mandatory and executed in the ProcessElement stage of Beam lifecycle.
<br></br>
This class can take actions SerializableAction, used in the DoFn Beam lifecycle.
withSetupAction : executed in the setup method
withStartBundleAction : executed in the start bundle method
withFinishBundleAction : executed in the finish bundle method
withTeardownAction : executed in the teardown method
These functions are not required and if they are given, they are executed lazily in the dedicated method.
<br></br>
Example usage:
// With serializable function but without lifecycle actions. MapProcessContextFn.from(String.class) .into(TypeDescriptors.integers()) .via((ProcessContext ctx) -> 1 / ctx.element().length) // Could throw ArithmeticException // With serializable function and some lifecycle actions. MapProcessContextFn.from(String.class) .into(TypeDescriptors.integers()) .via((String word) -> 1 / word.length) .withSetupAction(() -> System.out.println("Starting of mapping...") .withStartBundleAction(() -> System.out.println("Starting bundle of mapping...") .withFinishBundleAction(() -> System.out.println("Ending bundle of mapping...") .withTeardownAction(() -> System.out.println("Ending of mapping...")
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classMapProcessContextFn.Companion
-
Method Summary
Modifier and Type Method Description final <NewOutputT extends Any> MapProcessContextFn<InputT, NewOutputT>into(TypeDescriptor<NewOutputT> outputType)Add the output type descriptors, it's required because it allows to add default coder for Output. final MapProcessContextFn<InputT, OutputT>via(SerializableFunction<DoFn.ProcessContext<InputT, OutputT>, OutputT> processContextMapper)Method that takes the org.apache.beam.sdk.transforms.SerializableFunction that will be evaluated in the process element phase. final MapProcessContextFn<InputT, OutputT>withSetupAction(SerializableAction setupAction)Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the setup phase. final MapProcessContextFn<InputT, OutputT>withStartBundleAction(SerializableAction startBundleAction)Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the start bundle phase. final MapProcessContextFn<InputT, OutputT>withFinishBundleAction(SerializableAction finishBundleAction)Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the finish bundle phase. final MapProcessContextFn<InputT, OutputT>withTeardownAction(SerializableAction teardownAction)Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the teardown phase. final Unitsetup()Setup action in the DoFn worker lifecycle. final UnitstartBundle()Start bundle action in the DoFn worker lifecycle. final UnitfinishBundle()Finish bundle action in the DoFn worker lifecycle. final Unitteardown()Teardown action in the DoFn worker lifecycle. final UnitprocessElement(DoFn.ProcessContext<InputT, OutputT> ctx)ProcessElement. -
Methods inherited from class fr.groupbees.midgard.transforms.BaseElementFn
getInputTypeDescriptor, getOutputTypeDescriptor -
Methods inherited from class org.apache.beam.sdk.transforms.DoFn
getAllowedTimestampSkew, populateDisplayData, prepareForProcessing -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Method Detail
-
into
final <NewOutputT extends Any> MapProcessContextFn<InputT, NewOutputT> into(TypeDescriptor<NewOutputT> outputType)
Add the output type descriptors, it's required because it allows to add default coder for Output.
- Parameters:
outputType- a org.apache.beam.sdk.values.TypeDescriptor object
-
via
final MapProcessContextFn<InputT, OutputT> via(SerializableFunction<DoFn.ProcessContext<InputT, OutputT>, OutputT> processContextMapper)
Method that takes the org.apache.beam.sdk.transforms.SerializableFunction that will be evaluated in the process element phase. This function is based on a org.apache.beam.sdk.transforms.DoFn.ProcessContext as input and a generic ouput.
This function is mandatory in process element phase.
- Parameters:
processContextMapper- serializable function from process context and to output
-
withSetupAction
final MapProcessContextFn<InputT, OutputT> withSetupAction(SerializableAction setupAction)
Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the setup phase.
This function is not mandatory in the setup phase.
- Parameters:
setupAction- setup action
-
withStartBundleAction
final MapProcessContextFn<InputT, OutputT> withStartBundleAction(SerializableAction startBundleAction)
Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the start bundle phase.
This function is not mandatory in the start bundle phase.
- Parameters:
startBundleAction- start bundle action
-
withFinishBundleAction
final MapProcessContextFn<InputT, OutputT> withFinishBundleAction(SerializableAction finishBundleAction)
Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the finish bundle phase.
This function is not mandatory in the finish bundle phase.
- Parameters:
finishBundleAction- finish bundle action
-
withTeardownAction
final MapProcessContextFn<InputT, OutputT> withTeardownAction(SerializableAction teardownAction)
Method that takes the fr.groupbees.midgard.transforms.SerializableAction that will be evaluated in the teardown phase.
This function is not mandatory in the teardown phase.
- Parameters:
teardownAction- teardown action
-
startBundle
@DoFn.StartBundle() final Unit startBundle()
Start bundle action in the DoFn worker lifecycle.
-
finishBundle
@DoFn.FinishBundle() final Unit finishBundle()
Finish bundle action in the DoFn worker lifecycle.
-
processElement
@DoFn.ProcessElement() final Unit processElement(DoFn.ProcessContext<InputT, OutputT> ctx)
ProcessElement.
- Parameters:
ctx- a org.apache.beam.sdk.transforms.DoFn.ProcessContext object
-
-
-
-