Package edu.berkeley.nlp.lm.io

Interface Summary

Interface	Description
ArpaLmReaderCallback<V>	Callback that is called for each n-gram in the collection
LmReader<V,C extends LmReaderCallback<V>>
LmReaderCallback<V>	Callback that is called for each n-gram in the collection
NgramOrderedLmReaderCallback<V>	Callback that is called for each n-gram in the collection

Class Summary

Class	Description
ArpaLmReader<W>	A parser for ARPA LM files.
ComputeLogProbabilityOfTextStream	Computes the log probability of a list of files.
FirstPassCallback<V extends LongRepresentable<V>>	Reader callback which adds n-grams to an NgramMap
GoogleLmReader<W>	Reads in n-gram count collections in the format that the Google n-grams Web1T corpus comes in.
IOUtils	Some IO utility functions.
KneserNeyFileWritingLmReaderCallback<W>	Class for producing a Kneser-Ney language model in ARPA format from raw text.
KneserNeyLmReaderCallback<W>	Class for producing a Kneser-Ney language model in ARPA format from raw text.
LmReaders	This class contains a number of static methods for reading/writing/estimating n-gram language models.
MakeKneserNeyArpaFromText	Estimates a Kneser-Ney language model from raw text, and writes the language model out in ARPA-format.
MakeLmBinaryFromArpa	Given a language model in ARPA format, builds a binary representation of the language model and writes it to disk.
MakeLmBinaryFromGoogle	Given a directory in Google n-grams format, builds a binary representation of a stupid-backoff language model language model and writes it to disk.
MakeNgramMapBinaryFromGoogle	Like MakeLmBinaryFromGoogle, except it only writes the NgramMap portion of the LM, meaning the binary does not contain the vocabulary.
NgramMapAddingCallback<V>	Reader callback which adds n-grams to an NgramMap
TextReader<W>	Class for reading raw text files.