org.deeplearning4j.spark.canova

Class CanovaByteDataSetFunction

java.lang.Object

org.deeplearning4j.spark.canova.CanovaByteDataSetFunction

All Implemented Interfaces:

Serializable, org.apache.spark.api.java.function.PairFunction<scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>,Double,org.nd4j.linalg.dataset.DataSet>

public class CanovaByteDataSetFunction
extends Object
implements org.apache.spark.api.java.function.PairFunction<scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>,Double,org.nd4j.linalg.dataset.DataSet>

Created by nyghtowl on 1/16/16.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CanovaByteDataSetFunction(int labelIndex, int numPossibleLabels, int batchSize, int byteFileLen)
CanovaByteDataSetFunction(int labelIndex, int numPossibleLabels, int batchSize, int byteFileLen, boolean regression)
CanovaByteDataSetFunction(int labelIndex, int numPossibleLabels, int batchSize, int byteFileLen, boolean regression, org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)

Method Summary

Modifier and Type	Method and Description
scala.Tuple2<Double,org.nd4j.linalg.dataset.DataSet>	call(scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable> inputTuple)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CanovaByteDataSetFunction

public CanovaByteDataSetFunction(int labelIndex,
                                 int numPossibleLabels,
                                 int batchSize,
                                 int byteFileLen)

CanovaByteDataSetFunction

public CanovaByteDataSetFunction(int labelIndex,
                                 int numPossibleLabels,
                                 int batchSize,
                                 int byteFileLen,
                                 boolean regression)

CanovaByteDataSetFunction

public CanovaByteDataSetFunction(int labelIndex,
                                 int numPossibleLabels,
                                 int batchSize,
                                 int byteFileLen,
                                 boolean regression,
                                 org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)

Parameters:

labelIndex - Index of the label column

numPossibleLabels - Number of classes for classification (not used if regression = true)

batchSize - size of examples in DataSet. Pass in total examples if including all

byteFileLen - number of bytes per individual file

regression - False for classification, true for regression

preProcessor - DataSetPreprocessor (may be null)

Method Detail

call

public scala.Tuple2<Double,org.nd4j.linalg.dataset.DataSet> call(scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable> inputTuple)
                                                          throws Exception

Specified by:

call in interface org.apache.spark.api.java.function.PairFunction<scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>,Double,org.nd4j.linalg.dataset.DataSet>

Throws:

Exception