Package org.fife.ui.rsyntaxtextarea

Class TokenImpl

java.lang.Object
org.fife.ui.rsyntaxtextarea.TokenImpl
All Implemented Interfaces:
Token, TokenTypes
public class TokenImpl extends Object implements Token
The default implementation of Token.

Note: The instances of Token returned by RSyntaxDocuments are pooled and should always be treated as immutable. They should not be cast to TokenImpl and modified. Modifying tokens you did not create yourself can and will result in rendering issues and/or runtime exceptions. You have been warned!

Version:
0.3

  • Field Details

    • text

      public char[] text
      The text this token represents. This is implemented as a segment so we can point directly to the text in the document without having to make a copy of it.

    • textOffset

      public int textOffset

    • textCount

      public int textCount

    • offset

      private int offset
      The offset into the document at which this token resides.

    • type

      private int type
      The type of token this is; for example, TokenTypes.FUNCTION.

    • nextToken

      private Token nextToken
      The next token in this linked list.

    • languageIndex

      private int languageIndex
      The language this token is in, >= 0.

  • Constructor Details

    • TokenImpl

      public TokenImpl()
      Creates a "null" token. The token itself is not null; rather, it signifies that it is the last token in a linked list of tokens and that it is not part of a "multi-line token."

    • TokenImpl

      public TokenImpl(Segment line, int beg, int end, int startOffset, int type, int languageIndex)
      Constructor.
      Parameters:
      line - The segment from which to get the token.
      beg - The first character's position in line.
      end - The last character's position in line.
      startOffset - The offset into the document at which this token begins.
      type - A token type listed as "generic" above.
      languageIndex - The language index for this token.

    • TokenImpl

      public TokenImpl(char[] line, int beg, int end, int startOffset, int type, int languageIndex)
      Constructor.
      Parameters:
      line - The segment from which to get the token.
      beg - The first character's position in line.
      end - The last character's position in line.
      startOffset - The offset into the document at which this token begins.
      type - A token type listed as "generic" above.
      languageIndex - The language index for this token.

    • TokenImpl

      public TokenImpl(Token t2)
      Creates this token as a copy of the passed-in token.
      Parameters:
      t2 - The token from which to make a copy.

  • Method Details

    • appendHTMLRepresentation

      public StringBuilder appendHTMLRepresentation(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily)
      Description copied from interface: Token
      Appends HTML code for painting this token, using the given text area's color scheme.
      Specified by:
      appendHTMLRepresentation in interface Token
      Parameters:
      sb - The buffer to append to.
      textArea - The text area whose color scheme to use.
      fontFamily - Whether to include the font family in the HTML for this token. You can pass false for this parameter if, for example, you are making all your HTML be monospaced, and don't want any crazy fonts being used in the editor to be reflected in your HTML.
      Returns:
      The buffer appended to.
      See Also:

    • appendHTMLRepresentation

      public StringBuilder appendHTMLRepresentation(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily, boolean tabsToSpaces)
      Description copied from interface: Token
      Appends HTML code for painting this token, using the given text area's color scheme.
      Specified by:
      appendHTMLRepresentation in interface Token
      Parameters:
      sb - The buffer to append to.
      textArea - The text area whose color scheme to use.
      fontFamily - Whether to include the font family in the HTML for this token. You can pass false for this parameter if, for example, you are making all your HTML be monospaced, and don't want any crazy fonts being used in the editor to be reflected in your HTML.
      tabsToSpaces - Whether to convert tabs into spaces.
      Returns:
      The buffer appended to.
      See Also:

    • appendHtmlLexeme

      private StringBuilder appendHtmlLexeme(RSyntaxTextArea textArea, StringBuilder sb, boolean tabsToSpaces)
      Appends an HTML version of the lexeme of this token (i.e. no style HTML, but replacing chars such as \t, < and > with their escapes).
      Parameters:
      textArea - The text area.
      sb - The buffer to append to.
      tabsToSpaces - Whether to convert tabs into spaces.
      Returns:
      The same buffer.

    • charAt

      public char charAt(int index)
      Description copied from interface: Token
      Returns the character at the specified offset in the token.
      Specified by:
      charAt in interface Token
      Parameters:
      index - The index. This should be in the range 0-(Token.length()-1).
      Returns:
      The character.
      See Also:

    • containsPosition

      public boolean containsPosition(int pos)
      Description copied from interface: Token
      Returns whether the token straddles the specified position in the document.
      Specified by:
      containsPosition in interface Token
      Parameters:
      pos - The position in the document to check.
      Returns:
      Whether the specified position is straddled by this token.

    • copyFrom

      public void copyFrom(Token t2)
      Makes one token point to the same text segment, and have the same value as another token.
      Parameters:
      t2 - The token from which to copy.

    • documentToToken

      public int documentToToken(int pos)
      Description copied from interface: Token
      Returns the position in the token's internal char array corresponding to the specified document position.

      Note that this method does NOT do any bounds checking; you can pass in a document position that does not correspond to a position in this token, and you will not receive an Exception or any other notification; it is up to the caller to ensure valid input.

      Specified by:
      documentToToken in interface Token
      Parameters:
      pos - A position in the document that is represented by this token.
      Returns:
      The corresponding token position >= textOffset and < textOffset+textCount.
      See Also:

    • endsWith

      public boolean endsWith(char[] ch)
      Description copied from interface: Token
      Returns whether this token's lexeme ends with the specified characters.
      Specified by:
      endsWith in interface Token
      Parameters:
      ch - The characters.
      Returns:
      Whether this token's lexeme ends with the specified characters.
      See Also:

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • getEndOffset

      public int getEndOffset()
      Description copied from interface: Token
      Returns the end offset of this token in the document (exclusive). In other words, the token ranges from [getOffset(), getEndOffset()).
      Specified by:
      getEndOffset in interface Token
      Returns:
      The end offset of this token.
      See Also:

    • getHTMLFormatForColor

      private static String getHTMLFormatForColor(Color color)
      Returns a String of the form "#xxxxxx" good for use in HTML, representing the given color.
      Parameters:
      color - The color to get a string for.
      Returns:
      The HTML form of the color. If color is null, #000000 is returned.

    • getHTMLRepresentation

      public String getHTMLRepresentation(RSyntaxTextArea textArea)
      Description copied from interface: Token
      Returns a String containing HTML code for painting this token, using the given text area's color scheme.
      Specified by:
      getHTMLRepresentation in interface Token
      Parameters:
      textArea - The text area whose color scheme to use.
      Returns:
      The HTML representation of the token.
      See Also:

    • getLanguageIndex

      public int getLanguageIndex()
      Description copied from interface: Token
      Returns the language index of this token.
      Specified by:
      getLanguageIndex in interface Token
      Returns:
      The language index. A value of 0 denotes the "main" language, any positive value denotes a specific secondary language.
      See Also:

    • getLastNonCommentNonWhitespaceToken

      public Token getLastNonCommentNonWhitespaceToken()
      Description copied from interface: Token
      Returns the last token in this list that is not whitespace or a comment.
      Specified by:
      getLastNonCommentNonWhitespaceToken in interface Token
      Returns:
      The last non-comment, non-whitespace token, or null if there isn't one.

    • getLastPaintableToken

      public Token getLastPaintableToken()
      Description copied from interface: Token
      Returns the last paintable token in this token list, or null if there is no paintable token.
      Specified by:
      getLastPaintableToken in interface Token
      Returns:
      The last paintable token in this token list.

    • getLexeme

      public String getLexeme()
      Description copied from interface: Token
      Returns the text of this token, as a string.

      Note that this method isn't used much by the ryntaxtextarea package internally, as it tries to limit memory allocation.

      Specified by:
      getLexeme in interface Token
      Returns:
      The text of this token.

    • getListOffset

      public int getListOffset(RSyntaxTextArea textArea, TabExpander e, float x0, float x)
      Description copied from interface: Token
      Determines the offset into this token list (i.e., into the document) that covers pixel location x if the token list starts at pixel location x0

      . This method will return the document position "closest" to the x-coordinate (i.e., if they click on the "right-half" of the w in awe, the caret will be placed in between the w and e; similarly, clicking on the left-half places the caret between the a and w). This makes it useful for methods such as viewToModel found in javax.swing.text.View subclasses.

      Specified by:
      getListOffset in interface Token
      Parameters:
      textArea - The text area from which the token list was derived.
      e - How to expand tabs.
      x0 - The pixel x-location that is the beginning of tokenList.
      x - The pixel-position for which you want to get the corresponding offset.
      Returns:
      The position (in the document, NOT into the token list!) that covers the pixel location. If tokenList is null or has type Token.NULL, then -1 is returned; the caller should recognize this and return the actual end position of the (empty) line.

    • getNextToken

      public Token getNextToken()
      Description copied from interface: Token
      Returns the token after this one in the linked list.
      Specified by:
      getNextToken in interface Token
      Returns:
      The next token.

    • getOffset

      public int getOffset()
      Description copied from interface: Token
      Returns the offset into the document at which this token resides.
      Specified by:
      getOffset in interface Token
      Returns:
      The offset into the document.
      See Also:

    • getOffsetBeforeX

      public int getOffsetBeforeX(RSyntaxTextArea textArea, TabExpander e, float startX, float endBeforeX)
      Description copied from interface: Token
      Returns the position in the document that represents the last character in the token that will fit into endBeforeX-startX pixels. For example, if you're using a monospaced 8-pixel-per-character font, have the token "while" and startX is 0 and endBeforeX is 30, this method will return the document position of the "i" in "while", because the "i" ends at pixel 24, while the "l" ends at 32. If not even the first character fits in endBeforeX-startX, the first character's position is still returned so calling methods don't go into infinite loops.
      Specified by:
      getOffsetBeforeX in interface Token
      Parameters:
      textArea - The text area in which this token is being painted.
      e - How to expand tabs.
      startX - The x-coordinate at which the token will be painted. This is needed because of tabs.
      endBeforeX - The x-coordinate for which you want to find the last character of t which comes before it.
      Returns:
      The last document position that will fit in the specified amount of pixels.

    • getTextArray

      public char[] getTextArray()
      Description copied from interface: Token
      Returns the character array backing the lexeme of this token. This value should be treated as read-only.
      Specified by:
      getTextArray in interface Token
      Returns:
      A character array containing the lexeme of this token.
      See Also:

    • getTextOffset

      public int getTextOffset()
      Description copied from interface: Token
      Returns the offset into the character array of the lexeme in Token.getTextArray().
      Specified by:
      getTextOffset in interface Token
      Returns:
      The offset of the lexeme in the character array.
      See Also:

    • getType

      public int getType()
      Description copied from interface: Token
      Returns the type of this token.
      Specified by:
      getType in interface Token
      Returns:
      The type of this token.
      See Also:

    • getWidth

      public float getWidth(RSyntaxTextArea textArea, TabExpander e, float x0)
      Description copied from interface: Token
      Returns the width of this token given the specified parameters.
      Specified by:
      getWidth in interface Token
      Parameters:
      textArea - The text area in which the token is being painted.
      e - Describes how to expand tabs. This parameter cannot be null.
      x0 - The pixel-location at which the token begins. This is needed because of tabs.
      Returns:
      The width of the token, in pixels.
      See Also:

    • getWidthUpTo

      public float getWidthUpTo(int numChars, RSyntaxTextArea textArea, TabExpander e, float x0)
      Description copied from interface: Token
      Returns the width of a specified number of characters in this token. For example, for the token "while", specifying a value of 3 here returns the width of the "whi" portion of the token.
      Specified by:
      getWidthUpTo in interface Token
      Parameters:
      numChars - The number of characters for which to get the width.
      textArea - The text area in which the token is being painted.
      e - How to expand tabs. This value cannot be null.
      x0 - The pixel-location at which this token begins. This is needed because of tabs.
      Returns:
      The width of the specified number of characters in this token.
      See Also:

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • is

      public boolean is(char[] lexeme)
      Description copied from interface: Token
      Returns whether this token's lexeme matches a specific character array.
      Specified by:
      is in interface Token
      Parameters:
      lexeme - The lexeme to check for.
      Returns:
      Whether this token has that lexeme.
      See Also:

    • is

      public boolean is(int type, char[] lexeme)
      Description copied from interface: Token
      Returns whether this token is of the specified type, with the specified lexeme.

      This method is preferred over the other overload in performance-critical code where this operation may be called frequently, since it does not involve any String allocations.

      Specified by:
      is in interface Token
      Parameters:
      type - The type to check for.
      lexeme - The lexeme to check for.
      Returns:
      Whether this token has that type and lexeme.
      See Also:

    • is

      public boolean is(int type, String lexeme)
      Description copied from interface: Token
      Returns whether this token is of the specified type, with the specified lexeme.

      The other overload of this method is preferred over this one in performance-critical code, as this one involves a String allocation while the other does not.

      Specified by:
      is in interface Token
      Parameters:
      type - The type to check for.
      lexeme - The lexeme to check for.
      Returns:
      Whether this token has that type and lexeme.
      See Also:

    • isComment

      public boolean isComment()
      Description copied from interface: Token
      Returns whether this token is a comment.
      Specified by:
      isComment in interface Token
      Returns:
      Whether this token is a comment.
      See Also:

    • isCommentOrWhitespace

      public boolean isCommentOrWhitespace()
      Description copied from interface: Token
      Returns whether this token is a comment or whitespace.
      Specified by:
      isCommentOrWhitespace in interface Token
      Returns:
      Whether this token is a comment or whitespace.
      See Also:

    • isHyperlink

      public boolean isHyperlink()
      Description copied from interface: Token
      Returns whether this token is a hyperlink.
      Specified by:
      isHyperlink in interface Token
      Returns:
      Whether this token is a hyperlink.
      See Also:

    • isIdentifier

      public boolean isIdentifier()
      Description copied from interface: Token
      Returns whether this token is an identifier.
      Specified by:
      isIdentifier in interface Token
      Returns:
      Whether this token is an identifier.

    • isLeftCurly

      public boolean isLeftCurly()
      Description copied from interface: Token
      Returns whether this token is a TokenTypes.SEPARATOR representing a single left curly brace.
      Specified by:
      isLeftCurly in interface Token
      Returns:
      Whether this token is a left curly brace.
      See Also:

    • isRightCurly

      public boolean isRightCurly()
      Description copied from interface: Token
      Returns whether this token is a TokenTypes.SEPARATOR representing a single right curly brace.
      Specified by:
      isRightCurly in interface Token
      Returns:
      Whether this token is a right curly brace.
      See Also:

    • isPaintable

      public boolean isPaintable()
      Description copied from interface: Token
      Returns whether this token is "paintable;" i.e., whether the type of this token is one such that it has an associated syntax style. What this boils down to is whether the token type is greater than Token.NULL.
      Specified by:
      isPaintable in interface Token
      Returns:
      Whether this token is paintable.

    • isSingleChar

      public boolean isSingleChar(char ch)
      Description copied from interface: Token
      Returns whether this token is the specified single character.
      Specified by:
      isSingleChar in interface Token
      Parameters:
      ch - The character to check for.
      Returns:
      Whether this token's lexeme is the single character specified.
      See Also:

    • isSingleChar

      public boolean isSingleChar(int type, char ch)
      Description copied from interface: Token
      Returns whether this token is the specified single character, and of a specific type.
      Specified by:
      isSingleChar in interface Token
      Parameters:
      type - The token type.
      ch - The character to check for.
      Returns:
      Whether this token is of the specified type, and with a lexeme Equaling the single character specified.
      See Also:

    • isWhitespace

      public boolean isWhitespace()
      Description copied from interface: Token
      Returns whether this token is whitespace.
      Specified by:
      isWhitespace in interface Token
      Returns:
      true iff this token is whitespace.
      See Also:

    • length

      public int length()
      Description copied from interface: Token
      Returns the length of this token.
      Specified by:
      length in interface Token
      Returns:
      The length of this token.
      See Also:

    • listOffsetToView

      public Rectangle listOffsetToView(RSyntaxTextArea textArea, TabExpander e, int pos, int x0, Rectangle rect)
      Description copied from interface: Token
      Returns the bounding box for the specified document location. The location must be in the specified token list; if it isn't, null is returned.
      Specified by:
      listOffsetToView in interface Token
      Parameters:
      textArea - The text area from which the token list was derived.
      e - How to expand tabs.
      pos - The position in the document for which to get the bounding box in the view.
      x0 - The pixel x-location that is the beginning of tokenList.
      rect - The rectangle in which we'll be returning the results. This object is reused to keep from frequent memory allocations.
      Returns:
      The bounding box for the specified position in the model.

    • makeStartAt

      public void makeStartAt(int pos)
      Makes this token start at the specified offset into the document.

      Note: You should not modify Token instances you did not create yourself (e.g., came from an RSyntaxDocument). If you do, rendering issues and/or runtime exceptions will likely occur. You have been warned!

      Parameters:
      pos - The offset into the document this token should start at. Note that this token must already contain this position; if it doesn't, an exception is thrown.
      Throws:
      IllegalArgumentException - If pos is not already contained by this token.
      See Also:

    • moveOffset

      public void moveOffset(int amt)
      Moves the starting offset of this token.

      Note: You should not modify Token instances you did not create yourself (e.g., came from an RSyntaxDocument). If you do, rendering issues and/or runtime exceptions will likely occur. You have been warned!

      Parameters:
      amt - The amount to move the starting offset. This should be between 0 and textCount, inclusive.
      Throws:
      IllegalArgumentException - If amt is an invalid value.
      See Also:

    • set

      public void set(char[] line, int beg, int end, int offset, int type)
      Sets the value of this token to a particular segment of a document. The "next token" value is cleared.
      Parameters:
      line - The segment from which to get the token.
      beg - The first character's position in line.
      end - The last character's position in line.
      offset - The offset into the document at which this token begins.
      type - A token type listed as "generic" above.

    • setHyperlink

      public void setHyperlink(boolean hyperlink)
      Sets whether this token is a hyperlink.
      Specified by:
      setHyperlink in interface Token
      Parameters:
      hyperlink - Whether this token is a hyperlink.
      See Also:

    • setLanguageIndex

      public void setLanguageIndex(int languageIndex)
      Sets the language index for this token. If this value is positive, it denotes a specific "secondary" language this token represents (such as JavaScript code or CSS embedded in an HTML file). If this value is 0, this token is in the "main" language being edited. Negative values are invalid and treated as 0.
      Specified by:
      setLanguageIndex in interface Token
      Parameters:
      languageIndex - The new language index. A value of 0 denotes the "main" language, any positive value denotes a specific secondary language. Negative values will be treated as 0.
      See Also:

    • setNextToken

      public void setNextToken(Token nextToken)
      Sets the "next token" pointer of this token to point to the specified token.
      Parameters:
      nextToken - The new next token.
      See Also:

    • setOffset

      public void setOffset(int offset)
      Sets the offset into the document at which this token resides.
      Parameters:
      offset - The new offset into the document.
      See Also:

    • setType

      public void setType(int type)
      Description copied from interface: Token
      Sets the type of this token.
      Specified by:
      setType in interface Token
      Parameters:
      type - The new token type.
      See Also:

    • startsWith

      public boolean startsWith(char[] chars)
      Description copied from interface: Token
      Returns whether this token starts with the specified characters.
      Specified by:
      startsWith in interface Token
      Parameters:
      chars - The characters.
      Returns:
      Whether this token starts with those characters.
      See Also:

    • tokenToDocument

      public int tokenToDocument(int pos)
      Description copied from interface: Token
      Returns the position in the document corresponding to the specified position in this token's internal char array (textOffset - textOffset+textCount-1).

      Note that this method does NOT do any bounds checking; you can pass in an invalid token position, and you will not receive an Exception or any other indication that the returned document position is invalid. It is up to the user to ensure valid input.

      Specified by:
      tokenToDocument in interface Token
      Parameters:
      pos - A position in the token's internal char array (textOffset - textOffset+textCount).
      Returns:
      The corresponding position in the document.
      See Also:

    • toString

      public String toString()
      Returns this token as a String, which is useful for debugging.
      Overrides:
      toString in class Object
      Returns:
      A string describing this token.