Package org.projog.core.parser

Class TokenParser

  • class TokenParser
extends java.lang.Object
    Parses an input stream into discrete 'tokens' that are used to represent Prolog queries and rules.
    See Also:
    SentenceParser

    • Field Detail

      • operands

        private final Operands operands

      • lastParsedToken

        private Token lastParsedToken

      • rewound

        private boolean rewound

    • Constructor Detail

      • TokenParser

        TokenParser​(java.io.Reader reader,
            Operands operands)

    • Method Detail

      • hasNext

        boolean hasNext()
        Returns:
        true if there are more tokens to be parsed, else false

      • next

        Token next()
        Parse and return the next Token.
        Returns:
        the token that was parsed as a result of this call
        Throws:
        ParserException - if there are no more tokens to parse (i.e. parser has reached the end of the underlying input stream)

      • parseToken

        private Token parseToken()

      • rewind

        void rewind​(Token value)
        Rewinds the parser (i.e. "pushes-back" the last parsed token).

        The last parsed value will remain after the next call to next()

        Parameters:
        value - the value to rewind
        Throws:
        java.lang.IllegalArgumentException - if already in a rewound state (i.e. have already called TokenParser#rewind(String) since the last call to next()), or value is not equal to #getValue()

      • isImmediatelyFollowedByBracket

        boolean isImmediatelyFollowedByBracket​(boolean rewind)

      • isFollowedByTerm

        boolean isFollowedByTerm()
        Does the next value to be parsed represent a term (rather than a delimiter)

      • skipWhitespaceAndComments

        private void skipWhitespaceAndComments()

      • parseText

        private Token parseText​(int c,
                        TokenType t)
        Parameters:
        c - the first, already parsed, character of the token.

      • parseQuotedText

        private Token parseQuotedText()
        Reads a String consisting of all characters read from the parser up to the next '.

        If an atom's name is enclosed in quotes (i.e. ') then it may contain any character.

      • parseLeadingZero

        private Token parseLeadingZero​(int zero)
        Parses a character code and represents it as an integer.

        e.g. the text 0'a results in a token with the value 97 (the ascii value for a) being returned.

      • parseEscapeSequence

        private int parseEscapeSequence()

      • parseUnicode

        private int parseUnicode()

      • parseHex

        private char parseHex()

      • parseNumber

        private Token parseNumber​(int startChar)
        Parses a number, starting with the specified character, read from the parser.

        Deals with numbers of the form 3.4028235E38.

      • escape

        private int escape​(int escape)

      • parseSymbol

        private Token parseSymbol​(int c)

      • skipWhitespace

        private void skipWhitespace()

      • skipMultiLineComment

        private void skipMultiLineComment()

      • isValidParseableElement

        private boolean isValidParseableElement​(java.lang.String commandName)

      • isEndOfStream

        private static boolean isEndOfStream​(int c)

      • isSingleLineComment

        private static boolean isSingleLineComment​(int c)

      • isMultiLineCommentStart

        private static boolean isMultiLineCommentStart​(int c1,
                                               int c2)

      • isMultiLineCommentEnd

        private static boolean isMultiLineCommentEnd​(int c1,
                                             int c2)

      • isValidForAtom

        private static boolean isValidForAtom​(int c)

      • isVariable

        private static boolean isVariable​(int c)

      • isUnderscore

        private static boolean isUnderscore​(int c)

      • isQuote

        private static boolean isQuote​(int c)

      • isZero

        private static boolean isZero​(int c)

      • isEscapeSequencePrefix

        private static boolean isEscapeSequencePrefix​(int c)

      • createToken

        private static Token createToken​(java.lang.StringBuilder value,
                                 TokenType type)

      • createToken

        private static Token createToken​(java.lang.String value,
                                 TokenType type)