Class TokenParser

java.lang.Object
org.projog.core.parser.TokenParser
class TokenParser extends Object
Parses an input stream into discrete 'tokens' that are used to represent Prolog queries and rules.
See Also:

  • Field Details

    • parser

      private final CharacterParser parser

    • operands

      private final Operands operands

    • lastParsedToken

      private Token lastParsedToken

    • rewound

      private boolean rewound

  • Constructor Details

  • Method Details

    • hasNext

      boolean hasNext()
      Returns:
      true if there are more tokens to be parsed, else false

    • next

      Token next()
      Parse and return the next Token.
      Returns:
      the token that was parsed as a result of this call
      Throws:
      ParserException - if there are no more tokens to parse (i.e. parser has reached the end of the underlying input stream)

    • parseToken

      private Token parseToken()

    • rewind

      void rewind(Token value)
      Rewinds the parser (i.e. "pushes-back" the last parsed token).

      The last parsed value will remain after the next call to next()

      Parameters:
      value - the value to rewind
      Throws:
      IllegalArgumentException - if already in a rewound state (i.e. have already called
      invalid reference 
      TokenParser#rewind(String)
      since the last call to next()), or value is not equal to
      invalid reference 
      #getValue()

    • isImmediatelyFollowedByBracket

      boolean isImmediatelyFollowedByBracket(boolean rewind)

    • isFollowedByTerm

      boolean isFollowedByTerm()
      Does the next value to be parsed represent a term (rather than a delimiter)

    • newParserException

      ParserException newParserException(String message)
      Returns a new ParserException with the specified message.

    • skipWhitespaceAndComments

      private void skipWhitespaceAndComments()

    • parseText

      private Token parseText(int c, TokenType t)
      Parameters:
      c - the first, already parsed, character of the token.

    • parseQuotedText

      private Token parseQuotedText()
      Reads a String consisting of all characters read from the parser up to the next '.

      If an atom's name is enclosed in quotes (i.e. ') then it may contain any character.

    • parseLeadingZero

      private Token parseLeadingZero(int zero)
      Parses a character code and represents it as an integer.

      e.g. the text 0'a results in a token with the value 97 (the ascii value for a) being returned.

    • parseEscapeSequence

      private int parseEscapeSequence()

    • parseUnicode

      private int parseUnicode()

    • parseHex

      private char parseHex()

    • parseNumber

      private Token parseNumber(int startChar)
      Parses a number, starting with the specified character, read from the parser.

      Deals with numbers of the form 3.4028235E38.

    • escape

      private int escape(int escape)

    • parseSymbol

      private Token parseSymbol(int c)

    • skipWhitespace

      private void skipWhitespace()

    • skipMultiLineComment

      private void skipMultiLineComment()

    • isValidParseableElement

      private boolean isValidParseableElement(String commandName)

    • isEndOfStream

      private static boolean isEndOfStream(int c)

    • isSingleLineComment

      private static boolean isSingleLineComment(int c)

    • isMultiLineCommentStart

      private static boolean isMultiLineCommentStart(int c1, int c2)

    • isMultiLineCommentEnd

      private static boolean isMultiLineCommentEnd(int c1, int c2)

    • isValidForAtom

      private static boolean isValidForAtom(int c)

    • isVariable

      private static boolean isVariable(int c)

    • isUnderscore

      private static boolean isUnderscore(int c)

    • isQuote

      private static boolean isQuote(int c)

    • isZero

      private static boolean isZero(int c)

    • isEscapeSequencePrefix

      private static boolean isEscapeSequencePrefix(int c)

    • createToken

      private static Token createToken(StringBuilder value, TokenType type)

    • createToken

      private static Token createToken(String value, TokenType type)