org.deeplearning4j.spark.canova

Class CanovaSequenceDataSetFunction

java.lang.Object

org.deeplearning4j.spark.canova.CanovaSequenceDataSetFunction

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.Function<Collection<Collection<org.canova.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>

public class CanovaSequenceDataSetFunction
extends Object
implements org.apache.spark.api.java.function.Function<Collection<Collection<org.canova.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>, Serializable

Map Collection<Collection<Writable>> objects (out of a canova-spark sequence record reader function) to DataSet objects for Spark training. Analogous to SequenceRecordReaderDataSetIterator, but in the context of Spark. Supports loading data from a single source only (hence no masknig arrays, many-to-one etc here) see CanovaSequencePairDataSetFunction for the separate collections for input and labels version

Author:

Alex Black

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CanovaSequenceDataSetFunction(int labelIndex, int numPossibleLabels, boolean regression)
CanovaSequenceDataSetFunction(int labelIndex, int numPossibleLabels, boolean regression, org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor, org.canova.api.io.WritableConverter converter)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.dataset.DataSet	call(Collection<Collection<org.canova.api.writable.Writable>> input)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CanovaSequenceDataSetFunction

public CanovaSequenceDataSetFunction(int labelIndex,
                                     int numPossibleLabels,
                                     boolean regression)

Parameters:

labelIndex - Index of the label column

numPossibleLabels - Number of classes for classification (not used if regression = true)

regression - False for classification, true for regression

CanovaSequenceDataSetFunction

public CanovaSequenceDataSetFunction(int labelIndex,
                                     int numPossibleLabels,
                                     boolean regression,
                                     org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor,
                                     org.canova.api.io.WritableConverter converter)

Parameters:

labelIndex - Index of the label column

numPossibleLabels - Number of classes for classification (not used if regression = true)

regression - False for classification, true for regression

preProcessor - DataSetPreprocessor (may be null)

converter - WritableConverter (may be null)

Method Detail

call

public org.nd4j.linalg.dataset.DataSet call(Collection<Collection<org.canova.api.writable.Writable>> input)
                                     throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function<Collection<Collection<org.canova.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>

Throws:

Exception