Package com.javacc.parser

Class Token

java.lang.Object
com.javacc.parser.Token
All Implemented Interfaces:
JavaCCConstants, Node, freemarker.template.TemplateModel, freemarker.template.TemplateNodeModel, freemarker.template.TemplateScalarModel, Comparable<Node>
Direct Known Subclasses:
ANY_CHAR, Backslash, Comment, Delimiter, HASH, Identifier, InvalidToken, JavaCCKeyWord, KeyWord, Literal, MULTI_LINE_COMMENT_START, Operator, RightArrow, SINGLE_LINE_COMMENT_START, UP_TO_HERE, Whitespace
public class Token extends Object implements JavaCCConstants, Node

  • Constructor Details

  • Method Details

    • getGrammar

      public Grammar getGrammar()
      Specified by:
      getGrammar in interface Node

    • setGrammar

      public void setGrammar(Grammar grammar)
      Specified by:
      setGrammar in interface Node

    • getNodeName

      public String getNodeName()
      Specified by:
      getNodeName in interface freemarker.template.TemplateNodeModel

    • getLeadingComments

      public String getLeadingComments()

    • getNormalizedText

      public String getNormalizedText()

    • setImage

      public void setImage(String image)

    • isInserted

      public boolean isInserted()

    • preInsert

      public void preInsert(Token prependedToken)

    • newToken

      public static Token newToken(JavaCCConstants.TokenType type, String image, JavaCCLexer tokenSource)

    • setBeginOffset

      public void setBeginOffset(int beginOffset)
      It would be extremely rare that an application programmer would use this method. It needs to be public because it is part of the com.javacc.parser.Node interface.
      Specified by:
      setBeginOffset in interface Node

    • setEndOffset

      public void setEndOffset(int endOffset)
      It would be extremely rare that an application programmer would use this method. It needs to be public because it is part of the com.javacc.parser.Node interface.
      Specified by:
      setEndOffset in interface Node

    • getTokenSource

      public JavaCCLexer getTokenSource()
      Specified by:
      getTokenSource in interface Node
      Returns:
      the JavaCCLexer object that handles location info for the tokens.

    • setTokenSource

      public void setTokenSource(JavaCCLexer tokenSource)
      It should be exceedingly rare that an application programmer needs to use this method.
      Specified by:
      setTokenSource in interface Node

    • getType

      public JavaCCConstants.TokenType getType()
      Return the TokenType of this Token object

    • setType

      protected void setType(JavaCCConstants.TokenType type)

    • isVirtual

      public boolean isVirtual()
      Returns:
      whether this Token represent actual input or was it inserted somehow?

    • isSkipped

      public boolean isSkipped()
      Returns:
      Did we skip this token in parsing?

    • getBeginOffset

      public int getBeginOffset()
      Specified by:
      getBeginOffset in interface Node
      Returns:
      the offset in the input source where the token begins, expressed in code units.

    • getEndOffset

      public int getEndOffset()
      Specified by:
      getEndOffset in interface Node
      Returns:
      the offset in the input source where the token ends, expressed in code units. This is actually the offset where the very next token would begin.

    • getImage

      public String getImage()
      Returns:
      the string image of the token.

    • getNext

      public final Token getNext()
      Returns:
      the next _cached_ regular (i.e. parsed) token or null

    • getPrevious

      public final Token getPrevious()
      Returns:
      the previous regular (i.e. parsed) token or null

    • nextCachedToken

      public Token nextCachedToken()
      Returns:
      the next token of any sort (parsed or unparsed or invalid)

    • previousCachedToken

      public Token previousCachedToken()

    • replaceType

      public Token replaceType(JavaCCConstants.TokenType type)

    • getSource

      public String getSource()
      Specified by:
      getSource in interface Node
      Returns:
      the original source content this Node came from a reference to the #JavaCCLexer that stores the source code and the start/end location info stored in the Node object itself. This method could throw a NullPointerException if #getTokenSource returns null. Also, the return value could be spurious if the content of the source file was changed meanwhile. But this is just the default implementation of an API and it does not address this problem!

    • isUnparsed

      public boolean isUnparsed()
      Specified by:
      isUnparsed in interface Node
      Returns:
      whether this Node was created by regular operations of the parsing machinery.

    • setUnparsed

      public void setUnparsed(boolean unparsed)
      Description copied from interface: Node
      Mark whether this Node is unparsed, i.e. not the result of normal parsing
      Specified by:
      setUnparsed in interface Node
      Parameters:
      unparsed - whether to set the Node as unparsed or parsed.

    • clearChildren

      public void clearChildren()
      Description copied from interface: Node
      Remove all the child nodes
      Specified by:
      clearChildren in interface Node

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • precedingTokens

      public Iterator<Token> precedingTokens()
      Returns:
      An iterator of the tokens preceding this one.

    • precedingUnparsedTokens

      public List<Token> precedingUnparsedTokens()
      Returns:
      a list of the unparsed tokens preceding this one in the order they appear in the input

    • followingTokens

      public Iterator<Token> followingTokens()
      Returns:
      An iterator of the (cached) tokens that follow this one.

    • copyLocationInfo

      public void copyLocationInfo(Node from)
      Copy the location info from a Node
      Specified by:
      copyLocationInfo in interface Node
      Parameters:
      from - the Node to copy the info from

    • copyLocationInfo

      public void copyLocationInfo(Node start, Node end)
      Description copied from interface: Node
      Copy the location info given a start and end Node
      Specified by:
      copyLocationInfo in interface Node
      Parameters:
      start - the start node
      end - the end node

    • newToken

      public static Token newToken(JavaCCConstants.TokenType type, JavaCCLexer tokenSource, int beginOffset, int endOffset)

    • getLocation

      public String getLocation()
      Specified by:
      getLocation in interface Node
      Returns:
      a String that gives the starting location of this Node. This is a default implementation that could be overridden

    • setChild

      public void setChild(int i, Node n)
      Description copied from interface: Node
      Replace the node at index i
      Specified by:
      setChild in interface Node
      Parameters:
      i - the index
      n - the node

    • addChild

      public void addChild(Node n)
      Description copied from interface: Node
      appends a child node to this Node
      Specified by:
      addChild in interface Node
      Parameters:
      n - the Node to append

    • addChild

      public void addChild(int i, Node n)
      Description copied from interface: Node
      inserts a child Node at a specific index, displacing the nodes after the index by 1.
      Specified by:
      addChild in interface Node
      Parameters:
      i - the (zero-based) index at which to insert the node
      n - the Node to insert

    • removeChild

      public Node removeChild(int i)
      Description copied from interface: Node
      Remove the node at index i. Any Nodes after i are shifted to the left.
      Specified by:
      removeChild in interface Node
      Parameters:
      i - the index at which to remove
      Returns:
      the removed Node

    • indexOf

      public final int indexOf(Node n)
      Specified by:
      indexOf in interface Node
      Parameters:
      n - the Node to get the index of
      Returns:
      the index of the child Node. Or -1 if it is not a child Node.

    • getParent

      public Node getParent()
      Specified by:
      getParent in interface Node
      Returns:
      this node's parent Node

    • setParent

      public void setParent(Node parent)
      Specified by:
      setParent in interface Node
      Parameters:
      parent - The Node to set as the parent. Mostly used internally. The various addChild or appendChild sorts of methods should use this to set the node's parent.

    • getChildCount

      public final int getChildCount()
      Specified by:
      getChildCount in interface Node
      Returns:
      the number of child nodes

    • getChild

      public final Node getChild(int i)
      Specified by:
      getChild in interface Node
      Parameters:
      i - the index of the Node to return
      Returns:
      the Node at the specific offset

    • children

      public final List<Node> children()
      Specified by:
      children in interface Node

    • getParentNode

      public freemarker.template.TemplateNodeModel getParentNode()
      Specified by:
      getParentNode in interface freemarker.template.TemplateNodeModel

    • getChildNodes

      public freemarker.template.TemplateSequenceModel getChildNodes()
      Specified by:
      getChildNodes in interface freemarker.template.TemplateNodeModel

    • getNodeType

      public String getNodeType()
      Specified by:
      getNodeType in interface freemarker.template.TemplateNodeModel

    • getNodeNamespace

      public String getNodeNamespace()
      Specified by:
      getNodeNamespace in interface freemarker.template.TemplateNodeModel

    • getAsString

      public String getAsString()
      Specified by:
      getAsString in interface freemarker.template.TemplateScalarModel