Package org.codehaus.janino

Class Scanner

  • public class Scanner
extends java.lang.Object
    Splits up a character stream into tokens and returns them as String objects.

    • Constructor Summary

      Constructors 
      Constructor Description
      Scanner​(java.io.File file)
      Deprecated.
      This method is deprecated because it leaves the input file open
      Scanner​(java.io.File file, java.lang.String encoding)
      Deprecated.
      This method is deprecated because it leaves the input file open
      Scanner​(java.lang.String fileName)
      Deprecated.
      This method is deprecated because it leaves the input file open
      Scanner​(java.lang.String fileName, java.io.InputStream is)
      Sets up a scanner that reads tokens from the given InputStream in the platform default encoding.
      Scanner​(java.lang.String fileName, java.io.InputStream is, java.lang.String encoding)
      Sets up a scanner that reads tokens from the given InputStream with the given encoding (null means platform default encoding).
      Scanner​(java.lang.String fileName, java.io.Reader in)
      Sets up a scanner that reads tokens from the given Reader.
      Scanner​(java.lang.String fileName, java.io.Reader in, int initialLineNumber, int initialColumnNumber)
      Creates a Scanner that counts lines and columns from non-default initial values.
      Scanner​(java.lang.String fileName, java.lang.String encoding)
      Deprecated.
      This method is deprecated because it leaves the input file open

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      void close()
      Deprecated.
      This method is deprecated, because the concept described above is confusing.
      java.lang.String getFileName()  
      private int internalRead()  
      private static boolean isBinaryDigit​(int c)  
      private static boolean isDecimalDigit​(int c)
      To comply with the JLS, this method does not allow for non-latin digit (like Character.isDigit(char) does).
      private static boolean isHexDigit​(int c)
      To comply with the JLS, this method does not allow for non-latin digit (like Character.isDigit(char) does).
      private static boolean isOctalDigit​(int c)  
      Location location()  
      private int peek()
      Returns the next character, but does not consume it.
      private boolean peek​(java.lang.String expectedCharacters)
      Checks whether the next character is one of the expectedCharacters.
      private int peekButOne()
      Returns the next-but-one character, but does not consume any characters.
      private boolean peekRead​(int expected)
      Consumes the next character iff it equals the expected character.
      private boolean peekRead​(java.lang.String expectedCharacters)
      Consumes the next character iff it is one of the expectedCharacters
      Token produce()
      Produces and returns the next token.
      private char read()
      Consumes and returns the next character.
      private TokenType scan()  
      private void scanLiteralCharacter()
      Scans the next literal character into a StringBuilder.
      private TokenType scanNumericLiteral()  
      void setIgnoreWhiteSpace​(boolean value)
      If value is true, then white space in the input stream is ignored, rather than scanned as a TokenType.WHITE_SPACE token.
      private Token token​(TokenType type, java.lang.String value)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • SYSTEM_PROPERTY_SOURCE_DEBUGGING_ENABLE

        public static final java.lang.String SYSTEM_PROPERTY_SOURCE_DEBUGGING_ENABLE
        Setting this system property to 'true' enables source-level debugging. Typically, this means that compilation is executed with "-g:all" instead of "-g:none".
        See Also:
        Constant Field Values

      • SYSTEM_PROPERTY_SOURCE_DEBUGGING_DIR

        public static final java.lang.String SYSTEM_PROPERTY_SOURCE_DEBUGGING_DIR
        If the source code is not read from a file, debuggers have a hard time locating the source file for source-level debugging. As a workaround, a copy of the source code is written to a temporary file, which must be included in the debugger's source path. If this system property is set, the temporary source file is created in that directory, otherwise in the default temporary-file directory.
        See Also:
        File.createTempFile(String, String, File), Constant Field Values

      • SYSTEM_PROPERTY_SOURCE_DEBUGGING_KEEP

        public static final java.lang.String SYSTEM_PROPERTY_SOURCE_DEBUGGING_KEEP
        If set to "true", then the temporary source code files are not deleted on exit. That may be useful to interpret stack traces offline.
        See Also:
        Constant Field Values

      • sb

        private final java.lang.StringBuilder sb
        Holds the characters of the currently scanned token.

      • fileName

        @Nullable
private final java.lang.String fileName

      • in

        private final java.io.Reader in

      • ignoreWhiteSpace

        private boolean ignoreWhiteSpace

      • nextChar

        private int nextChar

      • nextButOneChar

        private int nextButOneChar

      • crLfPending

        private boolean crLfPending

      • nextCharLineNumber

        private int nextCharLineNumber

      • nextCharColumnNumber

        private int nextCharColumnNumber

      • tokenLineNumber

        private int tokenLineNumber
        Line number of the previously produced token (typically starting at one).

      • tokenColumnNumber

        private int tokenColumnNumber
        Column number of the first character of the previously produced token (1 if token is immediately preceded by a line break).

      • JAVA_KEYWORDS

        private static final java.util.Set<java.lang.String> JAVA_KEYWORDS

      • JAVA_OPERATORS

        private static final java.util.Set<java.lang.String> JAVA_OPERATORS

    • Constructor Detail

      • Scanner

        @Deprecated
public Scanner​(java.lang.String fileName)
        throws java.io.IOException
        Deprecated.
        This method is deprecated because it leaves the input file open
        Throws:
        java.io.IOException

      • Scanner

        @Deprecated
public Scanner​(java.lang.String fileName,
               java.lang.String encoding)
        throws java.io.IOException
        Deprecated.
        This method is deprecated because it leaves the input file open
        Throws:
        java.io.IOException

      • Scanner

        @Deprecated
public Scanner​(java.io.File file)
        throws java.io.IOException
        Deprecated.
        This method is deprecated because it leaves the input file open
        Throws:
        java.io.IOException

      • Scanner

        @Deprecated
public Scanner​(java.io.File file,
               @Nullable
               java.lang.String encoding)
        throws java.io.IOException
        Deprecated.
        This method is deprecated because it leaves the input file open
        Throws:
        java.io.IOException

      • Scanner

        public Scanner​(@Nullable
               java.lang.String fileName,
               java.io.InputStream is)
        throws java.io.IOException
        Sets up a scanner that reads tokens from the given InputStream in the platform default encoding.

        The fileName is solely used for reporting in thrown exceptions.

        Throws:
        java.io.IOException

      • Scanner

        public Scanner​(@Nullable
               java.lang.String fileName,
               java.io.InputStream is,
               @Nullable
               java.lang.String encoding)
        throws java.io.IOException
        Sets up a scanner that reads tokens from the given InputStream with the given encoding (null means platform default encoding).

        The fileName is used for reporting errors during compilation and for source level debugging, and should name an existing file. If null is passed, and the system property org.codehaus.janino.source_debugging.enable is set to "true", then a temporary file in org.codehaus.janino.source_debugging.dir or the system's default temp dir is created in order to make the source code available to a debugger.

        Throws:
        java.io.IOException

      • Scanner

        public Scanner​(@Nullable
               java.lang.String fileName,
               java.io.Reader in)
        throws java.io.IOException
        Sets up a scanner that reads tokens from the given Reader.

        The fileName is used for reporting errors during compilation and for source level debugging, and should name an existing file. If null is passed, and the system property org.codehaus.janino.source_debugging.enable is set to "true", then a temporary file in org.codehaus.janino.source_debugging.dir or the system's default temp dir is created in order to make the source code available to a debugger.

        Throws:
        java.io.IOException

      • Scanner

        public Scanner​(@Nullable
               java.lang.String fileName,
               java.io.Reader in,
               int initialLineNumber,
               int initialColumnNumber)
        throws java.io.IOException
        Creates a Scanner that counts lines and columns from non-default initial values.
        Throws:
        java.io.IOException

    • Method Detail

      • setIgnoreWhiteSpace

        public void setIgnoreWhiteSpace​(boolean value)
        If value is true, then white space in the input stream is ignored, rather than scanned as a TokenType.WHITE_SPACE token. Since white space is typically quite numerous, this optimization may save considerable overhead.

      • getFileName

        @Nullable
public java.lang.String getFileName()
        Returns:
        The file name optionally passed to the constructor

      • close

        @Deprecated
public void close()
           throws java.io.IOException
        Deprecated.
        This method is deprecated, because the concept described above is confusing. An application should close the underlying InputStream or Reader itself
        Closes the character source (file, InputStream, Reader) associated with this object. The results of future calls to produce() are undefined.
        Throws:
        java.io.IOException

      • location

        public Location location()
        Returns:
        The Location of the previously read (or peeked) token.

      • token

        private Token token​(TokenType type,
                    java.lang.String value)

      • isDecimalDigit

        private static boolean isDecimalDigit​(int c)
        To comply with the JLS, this method does not allow for non-latin digit (like Character.isDigit(char) does).

      • isHexDigit

        private static boolean isHexDigit​(int c)
        To comply with the JLS, this method does not allow for non-latin digit (like Character.isDigit(char) does).

      • isOctalDigit

        private static boolean isOctalDigit​(int c)

      • isBinaryDigit

        private static boolean isBinaryDigit​(int c)

      • scanLiteralCharacter

        private void scanLiteralCharacter()
                           throws CompileException,
                                  java.io.IOException
        Scans the next literal character into a StringBuilder.
        Throws:
        CompileException
        java.io.IOException

      • peek

        private int peek()
          throws CompileException,
                 java.io.IOException
        Returns the next character, but does not consume it.
        Returns:
        -1 iff there is no next character, i.e the input stream is at end-of-input
        Throws:
        CompileException
        java.io.IOException

      • peek

        private boolean peek​(java.lang.String expectedCharacters)
              throws CompileException,
                     java.io.IOException
        Checks whether the next character is one of the expectedCharacters.

        It is not possible to peek exactly for "end-of-input"; use peekRead(-1) instead.

        Throws:
        CompileException
        java.io.IOException

      • peekButOne

        private int peekButOne()
                throws CompileException,
                       java.io.IOException
        Returns the next-but-one character, but does not consume any characters.
        Throws:
        CompileException
        java.io.IOException

      • read

        private char read()
           throws CompileException,
                  java.io.IOException
        Consumes and returns the next character.
        Throws:
        CompileException - There is no next charcter, i.e. the input stream is at end-of-input
        java.io.IOException

      • peekRead

        private boolean peekRead​(int expected)
                  throws CompileException,
                         java.io.IOException
        Consumes the next character iff it equals the expected character.
        Returns:
        Whether the next character equalled the expected character
        Throws:
        CompileException
        java.io.IOException

      • peekRead

        private boolean peekRead​(java.lang.String expectedCharacters)
                  throws CompileException,
                         java.io.IOException
        Consumes the next character iff it is one of the expectedCharacters
        Returns:
        Whether the next character was one of the expectedCharacters
        Throws:
        CompileException
        java.io.IOException

      • internalRead

        private int internalRead()
                  throws java.io.IOException,
                         CompileException
        Returns:
        -1 iff there is no next character, i.e the input stream is at end-of-input
        Throws:
        java.io.IOException
        CompileException