Package org.languagetool.rules.ngrams

Class NgramProbabilityRule

java.lang.Object

org.languagetool.rules.Rule

org.languagetool.rules.ngrams.NgramProbabilityRule

@Experimental
public class NgramProbabilityRule
extends Rule

LanguageTool's probability check that uses ngram lookups to decide if an ngram of the input text is so rare in our ngram index that it should be considered an error. Also see http://wiki.languagetool.org/finding-errors-using-n-gram-data.

Since:

3.2

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
(package private) static class	NgramProbabilityRule.AdvancedReplacement
(package private) class	NgramProbabilityRule.Alternative
(package private) class	NgramProbabilityRule.Alternatives
(package private) static class	NgramProbabilityRule.Replacement

Field Summary

Fields

Modifier and Type	Field	Description
private static final List<NgramProbabilityRule.AdvancedReplacement>	ADV_REPLACEMENTS
private static final boolean	DEBUG
private final Language	language
private final LanguageModel	lm
private double	minProbability
private static final List<NgramProbabilityRule.Replacement>	REPLACEMENTS
static final String	RULE_ID

Fields inherited from class org.languagetool.rules.Rule

messages

Constructor Summary

Constructors

Constructor	Description
NgramProbabilityRule(ResourceBundle messages, LanguageModel languageModel, Language language)

Method Summary

Modifier and Type	Method	Description
protected boolean	acceptMatch(RuleMatch match, Probability p, AnalyzedSentence sentence)	Overwrite this method to discard matches by returning false.
private void	debug(String message, Object... vars)
private NgramProbabilityRule.Alternatives	getBetterAlternatives(GoogleToken prevToken, String token, GoogleToken next, GoogleToken googleToken, Probability p, AnalyzedSentence sentence)
private Optional<List<NgramProbabilityRule.Alternative>>	getBetterAlternatives(NgramProbabilityRule.Replacement replacement, GoogleToken prevToken, GoogleToken token, GoogleToken next, Probability p)
private Optional<AnalyzedToken>	getByPosTag(Set<AnalyzedToken> tokens, String wantedPosTagRegex)
String	getDescription()	A short description of the error this rule can detect, usually in the language of the text that is checked.
protected Tokenizer	getGoogleStyleWordTokenizer()
String	getId()	A string used to identify the rule in e.g.
RuleMatch[]	match(AnalyzedSentence sentence)	Check whether the given sentence matches this error rule, i.e.
void	setMinProbability(double minProbability)

Methods inherited from class org.languagetool.rules.Rule

addExamplePair, estimateContextForSureMatch, getAntiPatterns, getCategory, getConfigureText, getCorrectExamples, getDefaultValue, getErrorTriggeringExamples, getIncorrectExamples, getLocQualityIssueType, getMaxConfigurableValue, getMinConfigurableValue, getSentenceWithImmunization, getUrl, hasConfigurableValue, isDefaultOff, isDefaultTempOff, isDictionaryBasedSpellingRule, isOfficeDefaultOff, isOfficeDefaultOn, makeAntiPatterns, setCategory, setCorrectExamples, setDefaultOff, setDefaultOn, setDefaultTempOff, setErrorTriggeringExamples, setIncorrectExamples, setLocQualityIssueType, setOfficeDefaultOff, setOfficeDefaultOn, setUrl, supportsLanguage, toRuleMatchArray, useInOffice

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

RULE_ID

public static final String RULE_ID

Since:

3.2

See Also:

• Constant Field Values

DEBUG

private static final boolean DEBUG

See Also:

• Constant Field Values

REPLACEMENTS

private static final List<NgramProbabilityRule.Replacement> REPLACEMENTS

ADV_REPLACEMENTS

private static final List<NgramProbabilityRule.AdvancedReplacement> ADV_REPLACEMENTS

lm

private final LanguageModel lm

language

private final Language language

minProbability

private double minProbability

Constructor Details

NgramProbabilityRule

public NgramProbabilityRule(ResourceBundle messages,
 LanguageModel languageModel,
 Language language)

Method Details

getId

public String getId()

Description copied from class: Rule

A string used to identify the rule in e.g. configuration files. This string is supposed to be unique and to stay the same in all upcoming versions of LanguageTool. It's supposed to contain only the characters A-Z and the underscore.

Specified by:

getId in class Rule

setMinProbability

@Experimental
public void setMinProbability(double minProbability)

match

public RuleMatch[] match(AnalyzedSentence sentence)
                  throws IOException

Description copied from class: Rule

Check whether the given sentence matches this error rule, i.e. whether it contains the error detected by this rule. Note that the order in which this method is called is not always guaranteed, i.e. the sentence order in the text may be different than the order in which you get the sentences (this may be the case when LanguageTool is used as a LibreOffice/OpenOffice add-on, for example).

Specified by:

match in class Rule

Parameters:

sentence - a pre-analyzed sentence

Returns:

an array of RuleMatch objects

Throws:

IOException

acceptMatch

protected boolean acceptMatch(RuleMatch match,
 Probability p,
 AnalyzedSentence sentence)

Overwrite this method to discard matches by returning false.

Since:

3.3

getBetterAlternatives

private NgramProbabilityRule.Alternatives getBetterAlternatives(GoogleToken prevToken,
 String token,
 GoogleToken next,
 GoogleToken googleToken,
 Probability p,
 AnalyzedSentence sentence)
                                                         throws IOException

Throws:

IOException

getBetterAlternatives

private Optional<List<NgramProbabilityRule.Alternative>> getBetterAlternatives(NgramProbabilityRule.Replacement replacement,
 GoogleToken prevToken,
 GoogleToken token,
 GoogleToken next,
 Probability p)
                                                                        throws IOException

Throws:

IOException

getByPosTag

private Optional<AnalyzedToken> getByPosTag(Set<AnalyzedToken> tokens,
 String wantedPosTagRegex)

getDescription

public String getDescription()

Description copied from class: Rule

A short description of the error this rule can detect, usually in the language of the text that is checked.

Specified by:

getDescription in class Rule

getGoogleStyleWordTokenizer

protected Tokenizer getGoogleStyleWordTokenizer()

debug

private void debug(String message,
 Object... vars)