Package org.languagetool

Class AnalyzedSentence

java.lang.Object

org.languagetool.AnalyzedSentence

public final class AnalyzedSentence
extends Object

A sentence that has been tokenized and analyzed.

Field Summary

Fields

Modifier and Type	Field	Description
private final Set<String>	lemmaSet
private final AnalyzedTokenReadings[]	nonBlankPreDisambigTokens
private final AnalyzedTokenReadings[]	nonBlankTokens
private final AnalyzedTokenReadings[]	preDisambigTokens
private final AnalyzedTokenReadings[]	tokens
private final Set<String>	tokenSet
private final int[]	whPositions

Constructor Summary

Constructors

Modifier	Constructor	Description
	AnalyzedSentence(AnalyzedTokenReadings[] tokens)	Creates an AnalyzedSentence from the given AnalyzedTokenReadings.
private	AnalyzedSentence(AnalyzedTokenReadings[] tokens, int[] mapping, AnalyzedTokenReadings[] nonBlankTokens, AnalyzedTokenReadings[] nonBlankPreDisambigTokens)
	AnalyzedSentence(AnalyzedTokenReadings[] tokens, AnalyzedTokenReadings[] preDisambigTokens)

Method Summary

Modifier and Type	Method	Description
AnalyzedSentence	copy(AnalyzedSentence sentence)	The method copies AnalyzedSentence and returns the copy.
boolean	equals(Object o)
String	getAnnotations()	Get disambiguator actions log.
Set<String>	getLemmaSet()	Get the lowercase lemmas of this sentence in a set.
private Set<String>	getLemmaSet(AnalyzedTokenReadings[] tokens)
private @NotNull List<AnalyzedTokenReadings>	getNonBlankReadings(AnalyzedTokenReadings[] tokens, int whCounter, int nonWhCounter, int[] mapping)
int	getOriginalPosition(int nonWhPosition)	Get a position of a non-whitespace token in the original sentence with whitespace.
AnalyzedTokenReadings[]	getPreDisambigTokens()
AnalyzedTokenReadings[]	getPreDisambigTokensWithoutWhitespace()
String	getText()	Return the original text.
AnalyzedTokenReadings[]	getTokens()	Returns the AnalyzedTokenReadings of the analyzed text.
Set<String>	getTokenSet()	Get the lowercase tokens of this sentence in a set.
private Set<String>	getTokenSet(AnalyzedTokenReadings[] tokens)
AnalyzedTokenReadings[]	getTokensWithoutWhitespace()	Returns the AnalyzedTokenReadings of the analyzed text, with whitespace tokens removed but with the artificial SENT_START token included.
int	hashCode()
boolean	hasParagraphEndMark(Language lang)	Returns true if sentences ends with a paragraph break.
String	toShortString(String readingDelimiter)	Return string representation without chunk information.
String	toString()
String	toString(String readingDelimiter)	Return string representation with chunk information.
private String	toString(String readingDelimiter, boolean includeChunks)
(package private) String	toTextString()	Return string representation without any analysis information, just the original text.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

tokens

private final AnalyzedTokenReadings[] tokens

preDisambigTokens

private final AnalyzedTokenReadings[] preDisambigTokens

nonBlankTokens

private final AnalyzedTokenReadings[] nonBlankTokens

nonBlankPreDisambigTokens

private final AnalyzedTokenReadings[] nonBlankPreDisambigTokens

whPositions

private final int[] whPositions

tokenSet

private final Set<String> tokenSet

lemmaSet

private final Set<String> lemmaSet

Constructor Details

AnalyzedSentence

public AnalyzedSentence(AnalyzedTokenReadings[] tokens)

Creates an AnalyzedSentence from the given AnalyzedTokenReadings. Whitespace is also a token.

AnalyzedSentence

public AnalyzedSentence(AnalyzedTokenReadings[] tokens,
 AnalyzedTokenReadings[] preDisambigTokens)

AnalyzedSentence

private AnalyzedSentence(AnalyzedTokenReadings[] tokens,
 int[] mapping,
 AnalyzedTokenReadings[] nonBlankTokens,
 AnalyzedTokenReadings[] nonBlankPreDisambigTokens)

Method Details

getNonBlankReadings

@NotNull
private @NotNull List<AnalyzedTokenReadings> getNonBlankReadings(AnalyzedTokenReadings[] tokens,
 int whCounter,
 int nonWhCounter,
 int[] mapping)

getTokenSet

private Set<String> getTokenSet(AnalyzedTokenReadings[] tokens)

getLemmaSet

private Set<String> getLemmaSet(AnalyzedTokenReadings[] tokens)

copy

public AnalyzedSentence copy(AnalyzedSentence sentence)

The method copies AnalyzedSentence and returns the copy. Useful for performing local immunization (for example).

Parameters:

sentence - AnalyzedSentence to be copied

Returns:

a new object which is a copy

Since:

2.5

getTokens

public AnalyzedTokenReadings[] getTokens()

Returns the AnalyzedTokenReadings of the analyzed text. Whitespace is also a token.

getPreDisambigTokens

@Experimental
public AnalyzedTokenReadings[] getPreDisambigTokens()

Since:

4.5

getTokensWithoutWhitespace

public AnalyzedTokenReadings[] getTokensWithoutWhitespace()

Returns the AnalyzedTokenReadings of the analyzed text, with whitespace tokens removed but with the artificial SENT_START token included.

getPreDisambigTokensWithoutWhitespace

@Experimental
public AnalyzedTokenReadings[] getPreDisambigTokensWithoutWhitespace()

Since:

4.5

getOriginalPosition

public int getOriginalPosition(int nonWhPosition)

Get a position of a non-whitespace token in the original sentence with whitespace.

Parameters:

nonWhPosition - position of a non-whitespace token

Returns:

position in the original sentence.

toString

public String toString()

Overrides:

toString in class Object

toShortString

public String toShortString(String readingDelimiter)

Return string representation without chunk information.

Since:

2.3

getText

public String getText()

Return the original text.

Since:

2.7

toTextString

String toTextString()

Return string representation without any analysis information, just the original text.

Since:

2.6

toString

public String toString(String readingDelimiter)

Return string representation with chunk information.

toString

private String toString(String readingDelimiter,
 boolean includeChunks)

getAnnotations

public String getAnnotations()

Get disambiguator actions log.

getTokenSet

public Set<String> getTokenSet()

Get the lowercase tokens of this sentence in a set. Used internally for performance optimization.

Since:

2.4

getLemmaSet

public Set<String> getLemmaSet()

Get the lowercase lemmas of this sentence in a set. Used internally for performance optimization.

Since:

2.5

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

hasParagraphEndMark

public boolean hasParagraphEndMark(Language lang)

Returns true if sentences ends with a paragraph break.

Since:

4.3