com.tinkerpop.gremlin.process.computer.lambda

Class LambdaMapReduce<MK,MV,RK,RV,R>

java.lang.Object

com.tinkerpop.gremlin.process.computer.util.StaticMapReduce<MK,MV,RK,RV,R>

com.tinkerpop.gremlin.process.computer.lambda.LambdaMapReduce<MK,MV,RK,RV,R>

All Implemented Interfaces:

MapReduce<MK,MV,RK,RV,R>, java.lang.Cloneable

public class LambdaMapReduce<MK,MV,RK,RV,R>
extends StaticMapReduce<MK,MV,RK,RV,R>

Author:

Marko A. Rodriguez (http://markorodriguez.com)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LambdaMapReduce.Builder<MK,MV,RK,RV,R>

Nested classes/interfaces inherited from interface com.tinkerpop.gremlin.process.computer.MapReduce

MapReduce.MapEmitter<K,V>, MapReduce.NullObject, MapReduce.ReduceEmitter<OK,OV>, MapReduce.Stage

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	COMBINE_LAMBDA
static java.lang.String	MAP_KEY_SORT
static java.lang.String	MAP_LAMBDA
static java.lang.String	MEMORY_KEY
static java.lang.String	MEMORY_LAMBDA
static java.lang.String	REDUCE_KEY_SORT
static java.lang.String	REDUCE_LAMBDA

Fields inherited from interface com.tinkerpop.gremlin.process.computer.MapReduce

MAP_REDUCE

Method Summary

Modifier and Type	Method and Description
static <MK,MV,RK,RV,R> LambdaMapReduce.Builder<MK,MV,RK,RV,R>	build()
void	combine(MK key, java.util.Iterator<MV> values, MapReduce.ReduceEmitter<RK,RV> emitter) The combine() method is logically executed at all "machines" in parallel.
boolean	doStage(MapReduce.Stage stage) A MapReduce job can be map-only, map-reduce-only, or map-combine-reduce.
R	generateFinalResult(java.util.Iterator<KeyValue<RK,RV>> keyValues) The key/value pairs emitted by reduce() (or map() in a map-only job) can be iterated to generate a local JVM Java object.
java.util.Optional<java.util.Comparator<MK>>	getMapKeySort() If a Comparator is provided, then all pairs leaving the MapReduce.MapEmitter are sorted.
java.lang.String	getMemoryKey() The results of the MapReduce job are associated with a memory-key to ultimately be stored in Memory.
java.util.Optional<java.util.Comparator<RK>>	getReduceKeySort() If a Comparator is provided, then all pairs leaving the MapReduce.ReduceEmitter are sorted.
void	loadState(org.apache.commons.configuration.Configuration configuration) When it is necessary to load the state of a MapReduce job, this method is called.
void	map(Vertex vertex, MapReduce.MapEmitter<MK,MV> emitter) The map() method is logically executed at all vertices in the graph in parallel.
void	reduce(MK key, java.util.Iterator<MV> values, MapReduce.ReduceEmitter<RK,RV> emitter) The reduce() method is logically on the "machine" the respective key hashes to.
void	storeState(org.apache.commons.configuration.Configuration configuration) When it is necessary to store the state of a MapReduce job, this method is called.
java.lang.String	toString()

Methods inherited from class com.tinkerpop.gremlin.process.computer.util.StaticMapReduce

clone

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.tinkerpop.gremlin.process.computer.MapReduce

addResultToMemory, createMapReduce

Field Detail

MAP_LAMBDA

public static final java.lang.String MAP_LAMBDA

See Also:

Constant Field Values

MAP_KEY_SORT

public static final java.lang.String MAP_KEY_SORT

See Also:

Constant Field Values

COMBINE_LAMBDA

public static final java.lang.String COMBINE_LAMBDA

See Also:

Constant Field Values

REDUCE_LAMBDA

public static final java.lang.String REDUCE_LAMBDA

See Also:

Constant Field Values

REDUCE_KEY_SORT

public static final java.lang.String REDUCE_KEY_SORT

See Also:

Constant Field Values

MEMORY_LAMBDA

public static final java.lang.String MEMORY_LAMBDA

See Also:

Constant Field Values

MEMORY_KEY

public static final java.lang.String MEMORY_KEY

See Also:

Constant Field Values

Method Detail

loadState

public void loadState(org.apache.commons.configuration.Configuration configuration)

Description copied from interface: MapReduce

When it is necessary to load the state of a MapReduce job, this method is called. This is typically required when the MapReduce job needs to be serialized to another machine. Note that what is loaded is simply the instance state, not any processed data.

It is important that the state loaded from loadState() is identical to any state created from a constructor. For those GraphComputers that do not need to use Configurations to migrate state between JVMs, the constructor will only be used.

Parameters:

configuration - the configuration to load the state of the MapReduce job from.

storeState

public void storeState(org.apache.commons.configuration.Configuration configuration)

Description copied from interface: MapReduce

When it is necessary to store the state of a MapReduce job, this method is called. This is typically required when the MapReduce job needs to be serialized to another machine. Note that what is stored is simply the instance state, not any processed data.

Specified by:

storeState in interface MapReduce<MK,MV,RK,RV,R>

Overrides:

storeState in class StaticMapReduce<MK,MV,RK,RV,R>

Parameters:

configuration - the configuration to store the state of the MapReduce job in.

doStage

public boolean doStage(MapReduce.Stage stage)

Description copied from interface: MapReduce

A MapReduce job can be map-only, map-reduce-only, or map-combine-reduce. Before executing the particular stage, this method is called to determine if the respective stage is defined. This method should return true if the respective stage as a non-default method implementation.

Parameters:

stage - the stage to check for definition.

Returns:

whether that stage should be executed.

map

public void map(Vertex vertex,
                MapReduce.MapEmitter<MK,MV> emitter)

Description copied from interface: MapReduce

The map() method is logically executed at all vertices in the graph in parallel. The map() method emits key/value pairs given some analysis of the data in the vertices (and/or its incident edges).

Parameters:

vertex - the current vertex being map() processed.

emitter - the component that allows for key/value pairs to be emitted to the next stage.

combine

public void combine(MK key,
                    java.util.Iterator<MV> values,
                    MapReduce.ReduceEmitter<RK,RV> emitter)

Description copied from interface: MapReduce

The combine() method is logically executed at all "machines" in parallel. The combine() method pre-combines the values for a key prior to propagation over the wire. The combine() method must emit the same key/value pairs as the reduce() method. If there is a combine() implementation, there must be a reduce() implementation. If the MapReduce implementation is single machine, it can skip executing this method as reduce() is sufficient.

Parameters:

key - the key that has aggregated values

values - the aggregated values associated with the key

emitter - the component that allows for key/value pairs to be emitted to the reduce stage.

reduce

public void reduce(MK key,
                   java.util.Iterator<MV> values,
                   MapReduce.ReduceEmitter<RK,RV> emitter)

Description copied from interface: MapReduce

The reduce() method is logically on the "machine" the respective key hashes to. The reduce() method combines all the values associated with the key and emits key/value pairs.

Parameters:

key - the key that has aggregated values

values - the aggregated values associated with the key

emitter - the component that allows for key/value pairs to be emitted as the final result.

getMapKeySort

public java.util.Optional<java.util.Comparator<MK>> getMapKeySort()

Description copied from interface: MapReduce

If a Comparator is provided, then all pairs leaving the MapReduce.MapEmitter are sorted. The sorted results are either fed sorted to the combine/reduce-stage or as the final output. If sorting is not required, then Optional.empty() should be returned as sorting is computationally expensive. The default implementation returns Optional.empty().

Returns:

an Optional of a comparator for sorting the map output.

getReduceKeySort

public java.util.Optional<java.util.Comparator<RK>> getReduceKeySort()

Description copied from interface: MapReduce

If a Comparator is provided, then all pairs leaving the MapReduce.ReduceEmitter are sorted. If sorting is not required, then Optional.empty() should be returned as sorting is computationally expensive. The default implementation returns Optional.empty().

Returns:

an Optional of a comparator for sorting the reduce output.

generateFinalResult

public R generateFinalResult(java.util.Iterator<KeyValue<RK,RV>> keyValues)

Description copied from interface: MapReduce

The key/value pairs emitted by reduce() (or map() in a map-only job) can be iterated to generate a local JVM Java object.

Parameters:

keyValues - the key/value pairs that were emitted from reduce() (or map() in a map-only job)

Returns:

the resultant object formed from the emitted key/values.

getMemoryKey

public java.lang.String getMemoryKey()

Description copied from interface: MapReduce

The results of the MapReduce job are associated with a memory-key to ultimately be stored in Memory.

Returns:

the memory key of the generated result object.

toString

public java.lang.String toString()

Overrides:

toString in class StaticMapReduce<MK,MV,RK,RV,R>

build

public static <MK,MV,RK,RV,R> LambdaMapReduce.Builder<MK,MV,RK,RV,R> build()