org.cogroo.analyzer
Class Tokenizer

java.lang.Object
  extended by org.cogroo.analyzer.Tokenizer
All Implemented Interfaces:
Analyzer

public class Tokenizer
extends Object
implements Analyzer

The Tokenizer class separates every word in a given sentence and allocates them in a list of tokens.


Constructor Summary
Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)
           
 
Method Summary
 void analyze(Document document)
          Analyzes part of a text or a word.
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

Tokenizer

public Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)
Method Detail

analyze

public void analyze(Document document)
Description copied from interface: Analyzer
Analyzes part of a text or a word.

For example, it can search all the sentences in the text or tag every word in a sentence.

Specified by:
analyze in interface Analyzer
Parameters:
document - contains the whole text given by the user. After an analysis it can store the text's sentences, words or its tags.


Copyright © 2012-2013 CoGrOO. All Rights Reserved.