Package org.languagetool

Class AnalyzedTokenReadings

java.lang.Object
org.languagetool.AnalyzedTokenReadings
All Implemented Interfaces:
Iterable<AnalyzedToken>
public final class AnalyzedTokenReadings extends Object implements Iterable<AnalyzedToken>
An array of AnalyzedTokens used to store multiple POS tags and lemmas for a given single token.

  • Field Details

    • NON_WORD_REGEX

      private static final Pattern NON_WORD_REGEX

    • isWhitespace

      private final boolean isWhitespace

    • isLinebreak

      private final boolean isLinebreak

    • isSentStart

      private final boolean isSentStart

    • anTokReadings

      private AnalyzedToken[] anTokReadings

    • startPos

      private int startPos

    • token

      private String token

    • chunkTags

      private List<ChunkTag> chunkTags

    • isSentEnd

      private boolean isSentEnd

    • isParaEnd

      private boolean isParaEnd

    • isWhitespaceBefore

      private boolean isWhitespaceBefore

    • isPosTagUnknown

      private boolean isPosTagUnknown

    • whitespaceBeforeChar

      private String whitespaceBeforeChar

    • isImmunized

      private boolean isImmunized

    • isIgnoredBySpeller

      private boolean isIgnoredBySpeller

    • historicalAnnotations

      private String historicalAnnotations

    • hasSameLemmas

      private boolean hasSameLemmas

  • Constructor Details

  • Method Details

    • getReadings

      public List<AnalyzedToken> getReadings()

    • getAnalyzedToken

      public AnalyzedToken getAnalyzedToken(int idx)
      Get a token reading.
      See Also:

    • hasPosTag

      public boolean hasPosTag(String posTag)
      Checks if the token has a particular POS tag.
      Parameters:
      posTag - POS tag to look for

    • hasPosTagAndLemma

      public boolean hasPosTagAndLemma(String posTag, String lemma)
      Checks if the token has a particular POS tag and lemma.
      Parameters:
      posTag - POS tag and lemma to look for

    • hasReading

      public boolean hasReading()
      Checks if there is at least one POS tag
      Since:
      4.7

    • hasLemma

      public boolean hasLemma(String lemma)
      Checks if one of the token's readings has a particular lemma.
      Parameters:
      lemma - lemma POS tag to look for

    • hasAnyLemma

      public boolean hasAnyLemma(String... lemmas)
      Checks if one of the token's readings has one of the given lemmas
      Parameters:
      lemmas - to look for

    • hasPartialPosTag

      public boolean hasPartialPosTag(String posTag)
      Checks if the token has a particular POS tag, where only a part of the given POS tag needs to match.
      Parameters:
      posTag - POS tag substring to look for
      Since:
      1.8

    • hasAnyPartialPosTag

      public boolean hasAnyPartialPosTag(String... posTags)
      Checks if the token has any of the given particular POS tags (only a part of the given POS tag needs to match)
      Parameters:
      posTags - POS tag substring to look for
      Since:
      4.0

    • hasPosTagStartingWith

      public boolean hasPosTagStartingWith(String posTag)
      Checks if the token has a POS tag starting with the given string.
      Parameters:
      posTag - POS tag substring to look for
      Since:
      4.0

    • matchesPosTagRegex

      public boolean matchesPosTagRegex(String posTagRegex)
      Checks if at least one of the readings matches a given POS tag regex.
      Parameters:
      posTagRegex - POS tag regular expression to look for
      Since:
      2.9

    • addReading

      public void addReading(AnalyzedToken token, String ruleApplied)
      Add a new reading.
      Parameters:
      token - new reading, given as AnalyzedToken

    • removeReading

      public void removeReading(AnalyzedToken token, String ruleApplied)
      Removes a reading from the list of readings. Note: if the token has only one reading, then a new reading with an empty POS tag and an empty lemma is created.
      Parameters:
      token - reading to be removed

    • leaveReading

      public void leaveReading(AnalyzedToken token)
      Removes all readings but the one that matches the token given.
      Parameters:
      token - Token to be matched
      Since:
      1.5

    • getReadingsLength

      public int getReadingsLength()
      Number of readings.

    • isWhitespace

      public boolean isWhitespace()

    • isLinebreak

      public boolean isLinebreak()
      Returns true if the token equals \n, \r, \n\r, or \r\n.

    • isSentenceStart

      public boolean isSentenceStart()
      Since:
      2.3

    • isParagraphEnd

      public boolean isParagraphEnd()
      Returns:
      true when the token is a last token in a paragraph.
      Since:
      2.3

    • setParagraphEnd

      public void setParagraphEnd()
      Add a reading with a paragraph end token unless this is already a paragraph end.
      Since:
      2.3

    • isSentenceEnd

      public boolean isSentenceEnd()
      Returns:
      true when the token is a last token in a sentence.
      Since:
      2.3

    • isFieldCode

      public boolean isFieldCode()
      Returns:
      true if the token is LibreOffice/OpenOffice field code.
      Since:
      0.9.9

    • setSentEnd

      public void setSentEnd()
      Add a SENT_END tag.

    • getStartPos

      public int getStartPos()

    • getEndPos

      public int getEndPos()
      Since:
      2.9

    • setStartPos

      public void setStartPos(int position)

    • getToken

      public String getToken()

    • setWhitespaceBefore

      public void setWhitespaceBefore(String prevToken)

    • getWhitespaceBefore

      public String getWhitespaceBefore()

    • isWhitespaceBefore

      public boolean isWhitespaceBefore()

    • immunize

      public void immunize()

    • isImmunized

      public boolean isImmunized()

    • ignoreSpelling

      public void ignoreSpelling()
      Make the token ignored by all spelling rules.
      Since:
      2.5

    • isIgnoredBySpeller

      public boolean isIgnoredBySpeller()
      Test if the token can be ignored by spelling rules.
      Returns:
      true if the token should be ignored.
      Since:
      2.5

    • isPosTagUnknown

      public boolean isPosTagUnknown()
      Test if the token's POStag equals null.
      Returns:
      true if the token does not have a POStag
      Since:
      3.9

    • setNoRealPOStag

      private void setNoRealPOStag()
      Sets the flag on AnalyzedTokens to make matching on UNKNOWN POS tag correct in the Element class.

    • getHistoricalAnnotations

      public String getHistoricalAnnotations()
      Used to track disambiguator actions.
      Returns:
      the historicalAnnotations

    • setHistoricalAnnotations

      private void setHistoricalAnnotations(String historicalAnnotations)
      Used to track disambiguator actions.
      Parameters:
      historicalAnnotations - the historicalAnnotations to set

    • addHistoricalAnnotations

      private void addHistoricalAnnotations(String oldValue, String ruleApplied)

    • setChunkTags

      public void setChunkTags(List<ChunkTag> chunkTags)
      Since:
      2.3

    • getChunkTags

      public List<ChunkTag> getChunkTags()
      Since:
      2.3

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • isTagged

      public boolean isTagged()
      Returns:
      true if AnalyzedTokenReadings has some real POS tag (= not null or a special tag)
      Since:
      2.3

    • areLemmasSame

      private boolean areLemmasSame()
      Used to configure the internal variable for lemma equality.
      Returns:
      true if all AnalyzedToken lemmas are the same.
      Since:
      2.5

    • hasSameLemmas

      public boolean hasSameLemmas()
      Used to optimize pattern matching.
      Returns:
      true if all AnalyzedToken lemmas are the same.

    • isNonWord

      public boolean isNonWord()
      Returns:
      true if AnalyzedTokenReadings is a punctuation mark, bracket, etc
      Since:
      4.4

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • iterator

      public Iterator<AnalyzedToken> iterator()
      Specified by:
      iterator in interface Iterable<AnalyzedToken>
      Since:
      2.3