org.dkpro.tc.features.pair.core.ngram

Class LuceneKeywordCPFE

java.lang.Object

org.apache.uima.resource.Resource_ImplBase

org.apache.uima.fit.component.Resource_ImplBase

org.dkpro.tc.api.features.FeatureExtractorResource_ImplBase

org.dkpro.tc.features.ngram.base.NGramFeatureExtractorBase

org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

org.dkpro.tc.features.pair.core.ngram.meta.LucenePFEBase

org.dkpro.tc.features.pair.core.ngram.LuceneKeywordPFE

org.dkpro.tc.features.pair.core.ngram.LuceneKeywordCPFE

All Implemented Interfaces:

org.apache.uima.fit.component.ExternalResourceAware, org.apache.uima.resource.Resource, MetaDependent, PairFeatureExtractor

public class LuceneKeywordCPFE
extends LuceneKeywordPFE
implements PairFeatureExtractor

Combination pair ngram feature extractor. Creates features that are combinations of ngrams from both documents in the pair. Returns all combinations of qualified ngrams from each of the two documents. A feature value is 1 if each of the ngrams appeared in their respective text, and 0 otherwise.
For example, given two documents:
Document 1: "Cats eat mice"
Document 2: "Birds chase cats"

some combination ngrams are:
comboNG_cats_birds:1, comboNG_cats_eat_birds:1, comboNG_cats_birds_chase:1, etc.
Note: To extract ngrams from a pair of documents that are not combinations of ngrams across the documents, please use LuceneNGramPFE.

Field Summary

Fields

Modifier and Type	Field and Description
static String	KEYWORD_NGRAM_FIELD_COMBO
protected int	ngramMaxNCombo
protected int	ngramMinNCombo
protected boolean	ngramUseSymmetricalCombos
protected int	ngramUseTopKCombo
static String	PARAM_KEYWORD_NGRAM_MAX_N_COMBO Maximum token length of the combination
static String	PARAM_KEYWORD_NGRAM_MIN_N_COMBO Minimum token length of the combination.
static String	PARAM_KEYWORD_NGRAM_SYMMETRY_COMBO If true, both orderings of ngram combinations will be used.
static String	PARAM_KEYWORD_NGRAM_USE_TOP_K_COMBO Use this number of most frequent combinations
protected de.tudarmstadt.ukp.dkpro.core.api.frequency.util.FrequencyDistribution<String>	topKSetCombo

Fields inherited from class org.dkpro.tc.features.pair.core.ngram.LuceneKeywordPFE

includeCommas, KEYWORD_NGRAM_FIELD, KEYWORD_NGRAM_FIELD1, KEYWORD_NGRAM_FIELD2, keywordMaxN, keywordMinN, keywordNgramUseTopK, keywords, keywordsFile, markSentenceBoundary, markSentenceLocation, markViewBlindNgramsWithLocalView, ngramMaxN1, ngramMaxN2, ngramMinN1, ngramMinN2, PARAM_KEYWORD_NGRAM_INCLUDE_COMMAS, PARAM_KEYWORD_NGRAM_MARK_SENTENCE_BOUNDARY, PARAM_KEYWORD_NGRAM_MARK_SENTENCE_LOCATION, PARAM_KEYWORD_NGRAM_MAX_N, PARAM_KEYWORD_NGRAM_MAX_N_VIEW1, PARAM_KEYWORD_NGRAM_MAX_N_VIEW2, PARAM_KEYWORD_NGRAM_MIN_N, PARAM_KEYWORD_NGRAM_MIN_N_VIEW1, PARAM_KEYWORD_NGRAM_MIN_N_VIEW2, PARAM_KEYWORD_NGRAM_USE_TOP_K, PARAM_MARK_VIEWBLIND_KEYWORD_NGRAMS_WITH_LOCAL_VIEW, PARAM_NGRAM_KEYWORDS_FILE, PARAM_USE_VIEW1_KEYWORD_NGRAMS_AS_FEATURES, PARAM_USE_VIEW2_KEYWORD_NGRAMS_AS_FEATURES, PARAM_USE_VIEWBLIND_KEYWORD_NGRAMS_AS_FEATURES, useView1NgramsAsFeatures, useView2NgramsAsFeatures, useViewBlindNgramsAsFeatures

Fields inherited from class org.dkpro.tc.features.pair.core.ngram.meta.LucenePFEBase

fieldOfTheMoment, kngramUseTopK, ngramBinaryFeatureValuesCombos, ngramUseTopK1, ngramUseTopK2, PARAM_NGRAM_BINARY_FEATURE_VALUES_COMBO, PARAM_NGRAM_USE_TOP_K_VIEW1, PARAM_NGRAM_USE_TOP_K_VIEW2, topKSetView1, topKSetView2, topNOfTheMoment

Fields inherited from class org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

forceRereadFromIndex, LUCENE_NGRAM_FIELD, luceneDir, PARAM_SOURCE_LOCATION

Fields inherited from class org.dkpro.tc.features.ngram.base.NGramFeatureExtractorBase

dfStore, filterPartialStopwordMatches, ngramFreqThreshold, ngramLowerCase, ngramStopwordsFile, ngramUseTopK, PARAM_FILTER_PARTIAL_STOPWORD_MATCHES, PARAM_NGRAM_FREQ_THRESHOLD, PARAM_NGRAM_LOWER_CASE, PARAM_NGRAM_MAX_N, PARAM_NGRAM_MIN_N, PARAM_NGRAM_STOPWORDS_FILE, PARAM_NGRAM_USE_TOP_K, PARAM_TF_IDF_CALCULATION, prefix, stopwords, tfIdfCalculation, topKSet

Fields inherited from class org.dkpro.tc.api.features.FeatureExtractorResource_ImplBase

featureExtractorName, PARAM_UNIQUE_EXTRACTOR_NAME

Fields inherited from interface org.apache.uima.resource.Resource

PARAM_AGGREGATE_SOFA_MAPPINGS, PARAM_CONFIG_MANAGER, PARAM_CONFIG_PARAM_SETTINGS, PARAM_EXTERNAL_OVERRIDE_SETTINGS, PARAM_PERFORMANCE_TUNING_SETTINGS, PARAM_RESOURCE_MANAGER, PARAM_UIMA_CONTEXT

Constructor Summary

Constructors

Constructor and Description
LuceneKeywordCPFE()

Method Summary

Modifier and Type	Method and Description
Set<Feature>	extract(org.apache.uima.jcas.JCas view1, org.apache.uima.jcas.JCas view2)
List<MetaCollectorConfiguration>	getMetaCollectorClasses(Map<String,Object> parameterSettings)
boolean	initialize(org.apache.uima.resource.ResourceSpecifier aSpecifier, Map<String,Object> aAdditionalParams)
protected boolean	passesScreening(String term)

Methods inherited from class org.dkpro.tc.features.pair.core.ngram.LuceneKeywordPFE

getFeaturePrefix, getFieldName, getTopN, getViewNgrams

Methods inherited from class org.dkpro.tc.features.pair.core.ngram.meta.LucenePFEBase

addToFeatureArray

Methods inherited from class org.dkpro.tc.features.ngram.base.LuceneFeatureExtractorBase

getTopNgrams, logSelectionProcess

Methods inherited from class org.apache.uima.fit.component.Resource_ImplBase

afterResourcesInitialized, getLogger, getResourceName

Methods inherited from class org.apache.uima.resource.Resource_ImplBase

destroy, getCasManager, getMetaData, getRelativePathResolver, getResourceManager, getUimaContext, getUimaContextAdmin, setLogger, setMetaData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PARAM_KEYWORD_NGRAM_MIN_N_COMBO

public static final String PARAM_KEYWORD_NGRAM_MIN_N_COMBO

Minimum token length of the combination. If neither ngram is empty, this value must be at least 2.

See Also:

Constant Field Values

ngramMinNCombo

protected int ngramMinNCombo

PARAM_KEYWORD_NGRAM_MAX_N_COMBO

public static final String PARAM_KEYWORD_NGRAM_MAX_N_COMBO

Maximum token length of the combination

See Also:

Constant Field Values

ngramMaxNCombo

protected int ngramMaxNCombo

PARAM_KEYWORD_NGRAM_USE_TOP_K_COMBO

public static final String PARAM_KEYWORD_NGRAM_USE_TOP_K_COMBO

Use this number of most frequent combinations

See Also:

Constant Field Values

ngramUseTopKCombo

protected int ngramUseTopKCombo

PARAM_KEYWORD_NGRAM_SYMMETRY_COMBO

public static final String PARAM_KEYWORD_NGRAM_SYMMETRY_COMBO

If true, both orderings of ngram combinations will be used.
Example: If ngram 'cat' comes from Document 1, and ngram 'dog' comes from Document 2, then when this param = true, the features comboNG_cat_dog AND comboNG_dog_cat are produced.

See Also:

Constant Field Values

ngramUseSymmetricalCombos

protected boolean ngramUseSymmetricalCombos

KEYWORD_NGRAM_FIELD_COMBO

public static final String KEYWORD_NGRAM_FIELD_COMBO

See Also:

Constant Field Values

topKSetCombo

protected de.tudarmstadt.ukp.dkpro.core.api.frequency.util.FrequencyDistribution<String> topKSetCombo

Constructor Detail

LuceneKeywordCPFE

public LuceneKeywordCPFE()

Method Detail

getMetaCollectorClasses

public List<MetaCollectorConfiguration> getMetaCollectorClasses(Map<String,Object> parameterSettings)
                                                         throws org.apache.uima.resource.ResourceInitializationException

Specified by:

getMetaCollectorClasses in interface MetaDependent

Overrides:

getMetaCollectorClasses in class LuceneKeywordPFE

Throws:

org.apache.uima.resource.ResourceInitializationException

initialize

public boolean initialize(org.apache.uima.resource.ResourceSpecifier aSpecifier,
                          Map<String,Object> aAdditionalParams)
                   throws org.apache.uima.resource.ResourceInitializationException

Specified by:

initialize in interface org.apache.uima.resource.Resource

Overrides:

initialize in class LuceneKeywordPFE

Throws:

org.apache.uima.resource.ResourceInitializationException

extract

public Set<Feature> extract(org.apache.uima.jcas.JCas view1,
                            org.apache.uima.jcas.JCas view2)
                     throws org.dkpro.tc.api.exception.TextClassificationException

Specified by:

extract in interface PairFeatureExtractor

Overrides:

extract in class LuceneKeywordPFE

Throws:

org.dkpro.tc.api.exception.TextClassificationException

passesScreening

protected boolean passesScreening(String term)

Overrides:

passesScreening in class LuceneFeatureExtractorBase