A class representing a stochastic context free grammar (SCFG).
This class includes the representation of the grammar itself and
methods for training and testing it against some corpus.
At presnet of grammars in Chomsky Normal Form are supported. That
is rules may be binary or unary. If binary the mother an two
daughters are nonterminals, if unary the mother must be nonterminal
and daughter a terminal symbol.
The terminals and nonterminals symbol sets are derived automatically
from the LISP representation of the rules at initialization time
and are represented as \Ref{EST_Discrete}s. The distinguished
symbol is assumed to be the first mother of the first rule in
the given grammar.
Initialize from a set of rules
| void | set_rules | ( | LISP rules) |
Set (or reset) rules from external source after construction
Return rules as LISP list.
| void | find_terms_nonterms | ( | EST_StrList &nt, EST_StrList &t, LISP rules) |
Find the terminals and nonterminals in the given grammar, adding
them to the appropriate given string lists
Convert nonterminal index to string form
Convert terminal index to string form
Convert nonterminal string to index
Convert terminal string to index
| int | num_nonterminals | ( | ) const |
Number of nonterminals
| int | num_terminals | ( | ) const |
Number of terminals
| double | prob_B | ( | int p, int q, int r) const |
The rule probability of given binary rule
| double | prob_U | ( | int p, int m) const |
The rule probability of given unary rule
| void | set_rule_prob_cache | ( | ) |
(re-)set rule probability caches
Load grammar from named file
Save current grammar to named file
A stochastic context free grammar rule.
At present only two types of rule are supported:
{\tt est\_scfg\_binary\_rule} and {\tt est\_scfg\_unary\_rule}.
This is sufficient for the representation of grammars in
Chomsky Normal Form. Each rule also has a probability associated
with it. Terminals and noterminals are represented as ints using
the \Ref{EST_Discrete}s in \Ref{EST_SCFG} to reference the actual
alphabets.
Although this class includes a ``probability'' nothing in the rule
itself enforces it to be a true probability. It is responsibility
of the classes that use this rule to enforce that condition if
desired.
| est_scfg_rtype | type | ( | ) const |
rule type
In a unary rule this is a terminal, in a binary rule it
is a nonterminal
| void | set_rule | ( | double prob, int p, int m) |
| void | set_rule | ( | double prob, int p, int q, int r) |
A class used to train (and test) SCFGs is an extention of
\Ref{EST_SCFG}.
This offers an implementation of Pereira and Schabes ``Inside-Outside
reestimation from partially bracket corpora.'' ACL 1992.
A SCFG maybe trained from a corpus (optionally) containing brackets
over a series of passes reestimating the grammar probabilities
after each pass. This basically extends the \Ref{EST_SCFG} class
adding support for a bracket corpus and various indexes for efficient
use of the grammar.
Test the current grammar against the current corpus print summary.
Cross entropy measure only is given.
| void | test_crossbrackets | ( | ) |
Test the current grammar against the current corpus.
Sumamry includes percentage of cross bracketing accuracy
and percentage of fully correct parses.
Load a corpus from the given file.
Each setence in the corpus should be contained in parentheses.
Additional paranethesis may be used to denote phrasing within
a sentence. The corpus is read using the LISP reader so LISP
conventions shold apply, notable single quotes should appear
within double quotes.
| void | train_inout | ( | int passes, int startpass, int checkpoint, int spread, const EST_String &outfile) |
Train a grammar using the loaded corpus.
| Parameters |
|---|
| passes | the number of training passes desired. |
|---|
| startpass | from which pass to start from |
|---|
| checkpoint | save the grammar every n passes |
|---|
| spread | Percentage of corpus to use on each pass,
this cycles through the corpus on each pass. |
|---|
