org.cogroo.analyzer
Class Tokenizer
java.lang.Object
org.cogroo.analyzer.Tokenizer
- All Implemented Interfaces:
- Analyzer
public class Tokenizer
- extends Object
- implements Analyzer
The Tokenizer
class separates every word in a given sentence and allocates them in a
list of tokens.
Constructor Summary |
Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)
|
Method Summary |
void |
analyze(Document document)
Analyzes part of a text or a word. |
Tokenizer
public Tokenizer(opennlp.tools.tokenize.TokenizerME tokenizer)
analyze
public void analyze(Document document)
- Description copied from interface:
Analyzer
- Analyzes part of a text or a word.
For example, it can search all the
sentences in the text or tag every word in a sentence.
- Specified by:
analyze
in interface Analyzer
- Parameters:
document
- contains the whole text given by the user. After an analysis it can store the text's sentences, words or its tags.
Copyright © 2012-2013 CoGrOO. All Rights Reserved.