org.dkpro.tc.features.pair.core.ngram

Class LuceneNGramCPFE

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.fit.component.Resource_ImplBase

org.dkpro.tc.api.features.FeatureExtractorResource_ImplBase

org.dkpro.tc.features.ngram.base.NGramFeatureExtractorBase

org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

org.dkpro.tc.features.pair.core.ngram.LuceneNGramPFE

org.dkpro.tc.features.pair.core.ngram.LuceneNGramCPFE

All Implemented Interfaces:

org.apache.uima.fit.component.ExternalResourceAware, org.apache.uima.resource.Resource, MetaDependent, PairFeatureExtractor

public class LuceneNGramCPFE
extends LuceneNGramPFE
implements PairFeatureExtractor

Combination pair ngram feature extractor. Creates features that are combinations of ngrams from both documents in the pair. Returns all combinations of qualified ngrams from each of the two documents. A feature value is 1 if each of the ngrams appeared in their respective text, and 0 otherwise.
For example, given two documents:
Document 1: "Cats eat mice"
Document 2: "Birds chase cats"

some combination ngrams are:
comboNG_cats_birds:1, comboNG_cats_eat_birds:1, comboNG_cats_birds_chase:1, etc.
Note: To extract ngrams from a pair of documents that are not combinations of ngrams across the documents, please use LuceneNGramPFE.

Field Summary

Fields

Modifier and Type	Field and Description
static String	LUCENE_NGRAM_FIELDCOMBO
protected int	ngramMaxNCombo
protected int	ngramMinNCombo
protected boolean	ngramUseSymmetricalCombos
protected int	ngramUseTopKCombo
static String	PARAM_NGRAM_MAX_N_COMBO Maximum token length of the combination
static String	PARAM_NGRAM_MIN_N_COMBO Minimum token length of the combination.
static String	PARAM_NGRAM_SYMMETRY_COMBO If true, both orderings of ngram combinations will be used.
static String	PARAM_NGRAM_USE_TOP_K_COMBO Use this number of most frequent combinations
protected de.tudarmstadt.ukp.dkpro.core.api.frequency.util.FrequencyDistribution<String>	topKSetCombo

Fields inherited from class org.dkpro.tc.features.pair.core.ngram.LuceneNGramPFE

LUCENE_NGRAM_FIELD1, LUCENE_NGRAM_FIELD2, markViewBlindNgramsWithLocalView, ngramBinaryFeatureValuesCombos, ngramMaxN1, ngramMaxN2, ngramMinN1, ngramMinN2, ngramUseTopK1, ngramUseTopK2, PARAM_MARK_VIEWBLIND_NGRAMS_WITH_LOCAL_VIEW, PARAM_NGRAM_BINARY_FEATURE_VALUES_COMBO, PARAM_NGRAM_MAX_N_VIEW1, PARAM_NGRAM_MAX_N_VIEW2, PARAM_NGRAM_MIN_N_VIEW1, PARAM_NGRAM_MIN_N_VIEW2, PARAM_NGRAM_USE_TOP_K_VIEW1, PARAM_NGRAM_USE_TOP_K_VIEW2, PARAM_USE_VIEW1_NGRAMS_AS_FEATURES, PARAM_USE_VIEW2_NGRAMS_AS_FEATURES, PARAM_USE_VIEWBLIND_NGRAMS_AS_FEATURES, topKSetView1, topKSetView2, useView1NgramsAsFeatures, useView2NgramsAsFeatures, useViewBlindNgramsAsFeatures

Fields inherited from class org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

forceRereadFromIndex, LUCENE_NGRAM_FIELD, luceneDir, PARAM_SOURCE_LOCATION

Fields inherited from class org.dkpro.tc.features.ngram.base.NGramFeatureExtractorBase

dfStore, filterPartialStopwordMatches, ngramFreqThreshold, ngramLowerCase, ngramMaxN, ngramMinN, ngramStopwordsFile, ngramUseTopK, PARAM_FILTER_PARTIAL_STOPWORD_MATCHES, PARAM_NGRAM_FREQ_THRESHOLD, PARAM_NGRAM_LOWER_CASE, PARAM_NGRAM_MAX_N, PARAM_NGRAM_MIN_N, PARAM_NGRAM_STOPWORDS_FILE, PARAM_NGRAM_USE_TOP_K, PARAM_TF_IDF_CALCULATION, prefix, stopwords, tfIdfCalculation, topKSet

Fields inherited from class org.dkpro.tc.api.features.FeatureExtractorResource_ImplBase

featureExtractorName, PARAM_UNIQUE_EXTRACTOR_NAME

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
LuceneNGramCPFE()

Method Summary

Modifier and Type	Method and Description
Set<Feature>	extract(org.apache.uima.jcas.JCas view1, org.apache.uima.jcas.JCas view2)
List<MetaCollectorConfiguration>	getMetaCollectorClasses(Map<String,Object> parameterSettings)
boolean	initialize(org.apache.uima.resource.ResourceSpecifier aSpecifier, Map<String,Object> aAdditionalParams)

Methods inherited from class org.dkpro.tc.features.pair.core.ngram.LuceneNGramPFE

addToFeatureArray, getFeaturePrefix, getFieldName, getTopN, getTopNgrams, getTopNgramsView1, getTopNgramsView2, getViewNgrams

Methods inherited from class org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

logSelectionProcess, passesScreening

Methods inherited from class org.apache.uima.fit.component.Resource_ImplBase

afterResourcesInitialized, getLogger, getResourceName

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

destroy, getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_NGRAM_MIN_N_COMBO

public static final String PARAM_NGRAM_MIN_N_COMBO

Minimum token length of the combination. If neither ngram is empty, this value must be at least 2.

See Also:

Constant Field Values

ngramMinNCombo

protected int ngramMinNCombo

PARAM_NGRAM_MAX_N_COMBO

public static final String PARAM_NGRAM_MAX_N_COMBO

Maximum token length of the combination

See Also:

Constant Field Values

ngramMaxNCombo

protected int ngramMaxNCombo

PARAM_NGRAM_USE_TOP_K_COMBO

public static final String PARAM_NGRAM_USE_TOP_K_COMBO

Use this number of most frequent combinations

See Also:

Constant Field Values

ngramUseTopKCombo

protected int ngramUseTopKCombo

PARAM_NGRAM_SYMMETRY_COMBO

public static final String PARAM_NGRAM_SYMMETRY_COMBO

If true, both orderings of ngram combinations will be used.
Example: If ngram 'cat' comes from Document 1, and ngram 'dog' comes from Document 2, then when this param = true, the features comboNG_cat_dog AND comboNG_dog_cat are produced.

See Also:

Constant Field Values

ngramUseSymmetricalCombos

protected boolean ngramUseSymmetricalCombos

LUCENE_NGRAM_FIELDCOMBO

public static final String LUCENE_NGRAM_FIELDCOMBO

See Also:

Constant Field Values

topKSetCombo

protected de.tudarmstadt.ukp.dkpro.core.api.frequency.util.FrequencyDistribution<String> topKSetCombo

Constructor Detail

LuceneNGramCPFE

public LuceneNGramCPFE()

Method Detail

initialize

public boolean initialize(org.apache.uima.resource.ResourceSpecifier aSpecifier,
                          Map<String,Object> aAdditionalParams)
                   throws org.apache.uima.resource.ResourceInitializationException

Specified by:

initialize in interface org.apache.uima.resource.Resource

Overrides:

initialize in class LuceneNGramPFE

Throws:

org.apache.uima.resource.ResourceInitializationException

getMetaCollectorClasses

public List<MetaCollectorConfiguration> getMetaCollectorClasses(Map<String,Object> parameterSettings)
                                                         throws org.apache.uima.resource.ResourceInitializationException

Specified by:

getMetaCollectorClasses in interface MetaDependent

Overrides:

getMetaCollectorClasses in class LuceneNGramPFE

Throws:

org.apache.uima.resource.ResourceInitializationException

extract

public Set<Feature> extract(org.apache.uima.jcas.JCas view1,
                            org.apache.uima.jcas.JCas view2)
                     throws org.dkpro.tc.api.exception.TextClassificationException

Specified by:

extract in interface PairFeatureExtractor

Overrides:

extract in class LuceneNGramPFE

Throws:

org.dkpro.tc.api.exception.TextClassificationException