|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.conf.Configured
org.htuple.examples.SecondarySort
public final class SecondarySort
An example MapReduce job showing how the Tuple
and ShuffleUtils
can be used in conjunction with
each other to secondary sort people's names.
Nested Class Summary | |
---|---|
static class |
SecondarySort.Map
This map class simply tokenizes each input line, and emits a (tuple, line) pair, where the tuple contains the last and first name. |
static class |
SecondarySort.Reduce
The reducer just emits the map output values, allowing us to examine the resulting output and determine the results of the secondary sort. |
Field Summary | |
---|---|
static String[] |
EXAMPLE_NAMES
Sample input used by this example job. |
Constructor Summary | |
---|---|
SecondarySort()
|
Method Summary | |
---|---|
static void |
main(String[] args)
Main entry point for the example. |
int |
run(String[] args)
The MapReduce driver - setup and launch the job. |
static void |
setupSecondarySort(org.apache.hadoop.conf.Configuration conf)
Partition and group on just the last name; sort on both last and first name. |
static Tuple |
stringToTuple(String line)
Split the input line and return a Tuple representation of the last and first names. |
static void |
writeInput(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path inputDir)
Writes the contents of EXAMPLE_NAMES into a file in the job input directory in HDFS. |
Methods inherited from class org.apache.hadoop.conf.Configured |
---|
getConf, setConf |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
---|
getConf, setConf |
Field Detail |
---|
public static final String[] EXAMPLE_NAMES
Constructor Detail |
---|
public SecondarySort()
Method Detail |
---|
public static void main(String[] args) throws Exception
args
- arguments
Exception
- when something goes wrongpublic static void writeInput(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path inputDir) throws IOException
EXAMPLE_NAMES
into a file in the job input directory in HDFS.
conf
- the Hadoop configinputDir
- the HDFS input directory where we'll write a file
IOException
- if something goes wrongpublic int run(String[] args) throws Exception
run
in interface org.apache.hadoop.util.Tool
args
- the command-line arguments
Exception
- if something goes wrongpublic static void setupSecondarySort(org.apache.hadoop.conf.Configuration conf)
conf
- the Hadoop configpublic static Tuple stringToTuple(String line)
Tuple
representation of the last and first names.
line
- a line containing a tab-delimited last and first name.
Tuple
representation of the line
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |