org.allenai.nlpstack.parse.poly.polyparser
GoldParseTrainingVectorSource
Related Doc:
package polyparser
org.allenai.nlpstack.parse.poly.polyparser
the data source for the parse trees
the transition system factory to use (for generating states)
a trained cost function factory to adapt (optional)
a trained cost function factory to adapt (optional)
This generates a list of labeled feature vectors from a gold parse tree (for training).
This generates a list of labeled feature vectors from a gold parse tree (for training). The gold parse tree is reduced to its representation as a list of 2*n transitions, then a TrainingVector is produced for each transition (in order).
Note that this function is implemented using tail-recursion.
the sculpture to generate feature vectors from
a list of training vectors
the data source for the parse trees
Returns the set of tasks associated with the training vectors in this source.
Returns the set of tasks associated with the training vectors in this source.
In a perhaps over-careful attempt to avoid having all the non-uniqued tasks being stored in memory simultaneously, this was originally implemented as:
format: OFF lazy val tasks: Iterable[ClassificationTask] = taskHelper(Set(), getVectorIterator) tailrec private def taskHelper( resultSoFar: Set[ClassificationTask], vectorIter: Iterator[FSMTrainingVector] ): Set[ClassificationTask] = {
if (!vectorIter.hasNext) { resultSoFar } else { taskHelper(resultSoFar + vectorIter.next().task, vectorIter) } } format: ON
the transition system factory to use (for generating states)
A GoldParseTrainingVectorSource reduces a gold parse tree to a set of feature vectors for classifier training.
Essentially, we derive the 2*n parser states that lead to the gold parse. Each of these states becomes a feature vector (using the apply method of the provided TransitionParserFeature), labeled with the transition executed from that state in the gold parse.
One of the constructor arguments is a TaskIdentifer. This will dispatch the feature vectors to train different classifiers. For instance, if taskIdentifier(state) != taskIdentifier(state2), then their respective feature vectors (i.e. feature(state) and feature(state2)) will be used to train different classifiers.
the data source for the parse trees
the transition system factory to use (for generating states)
a trained cost function factory to adapt (optional)