Class STLexer

java.lang.Object
org.stringtemplate.v4.compiler.STLexer
All Implemented Interfaces:
org.antlr.runtime.TokenSource
public class STLexer extends Object implements org.antlr.runtime.TokenSource
This class represents the tokenizer for templates. It operates in two modes: inside and outside of expressions. It implements the TokenSource interface so it can be used with ANTLR parsers. Outside of expressions, we can return these token types: TEXT, INDENT, LDELIM (start of expression), RCURLY (end of subtemplate), and NEWLINE. Inside of an expression, this lexer returns all of the tokens needed by STParser. From the parser's point of view, it can treat a template as a simple stream of elements.

This class defines the token types and communicates these values to STParser.g via STLexer.tokens file (which must remain consistent).

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    STLexer.STToken
    We build STToken tokens instead of relying on CommonToken so we can override STLexer.STToken.toString().

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    AND
     
    static final int
    AT
     
    static final int
    BANG
     
    static final int
    COLON
     
    static final int
    COMMA
     
    static final int
    COMMENT
     
    static final int
    DOT
     
    static final int
    ELLIPSIS
     
    static final int
    ELSE
     
    static final int
    ELSEIF
     
    static final int
    ENDIF
     
    static final char
    EOF
     
    static final int
    EOF_TYPE
     
    static final int
    EQUALS
     
    static final int
    FALSE
     
    static final int
    ID
     
    static final int
    IF
     
    static final int
    INDENT
     
    static final int
    LBRACK
     
    static final int
    LCURLY
     
    static final int
    LDELIM
     
    static final int
    LPAREN
     
    static final int
    NEWLINE
     
    static final int
    OR
     
    static final int
    PIPE
     
    static final int
    RBRACK
     
    static final int
    RCURLY
     
    static final int
    RDELIM
     
    static final int
    REGION_END
     
    static final int
    RPAREN
     
    static final int
    SEMI
     
    static final org.antlr.runtime.Token
    SKIP
     
    static final int
    SLASH
     
    static final int
    STRING
     
    int
    subtemplateDepth
    To be able to properly track the inside/outside mode, we need to track how deeply nested we are in some templates.
    static final int
    SUPER
     
    static final int
    TEXT
     
    static final int
    TRUE
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    STLexer(org.antlr.runtime.CharStream input)
     
    STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken)
     
    STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken, char delimiterStartChar, char delimiterStopChar)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    org.antlr.runtime.Token
    _nextToken()
     
    protected void
    consume()
     
    void
    emit(org.antlr.runtime.Token token)
     
    String
    getSourceName()
     
    protected org.antlr.runtime.Token
    inside()
     
    static boolean
    isIDLetter(char c)
     
    static boolean
    isIDStartLetter(char c)
     
    static boolean
    isUnicodeLetter(char c)
     
    static boolean
    isWS(char c)
     
    void
    match(char x)
    Consume if x is next character on the input stream.
    org.antlr.runtime.Token
    newToken(int ttype)
     
    org.antlr.runtime.Token
    newToken(int ttype, String text)
     
    org.antlr.runtime.Token
    newToken(int ttype, String text, int pos)
     
    org.antlr.runtime.Token
    newTokenFromPreviousChar(int ttype)
     
    org.antlr.runtime.Token
    nextToken()
     
    protected org.antlr.runtime.Token
    outside()
     
    static String
    str(int c)
     

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • STLexer

      public STLexer(org.antlr.runtime.CharStream input)

    • STLexer

      public STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken)

    • STLexer

      public STLexer(ErrorManager errMgr, org.antlr.runtime.CharStream input, org.antlr.runtime.Token templateToken, char delimiterStartChar, char delimiterStopChar)

  • Method Details

    • nextToken

      public org.antlr.runtime.Token nextToken()
      Specified by:
      nextToken in interface org.antlr.runtime.TokenSource

    • match

      public void match(char x)
      Consume if x is next character on the input stream.

    • consume

      protected void consume()

    • emit

      public void emit(org.antlr.runtime.Token token)

    • _nextToken

      public org.antlr.runtime.Token _nextToken()

    • outside

      protected org.antlr.runtime.Token outside()

    • inside

      protected org.antlr.runtime.Token inside()

    • isIDStartLetter

      public static boolean isIDStartLetter(char c)

    • isIDLetter

      public static boolean isIDLetter(char c)

    • isWS

      public static boolean isWS(char c)

    • isUnicodeLetter

      public static boolean isUnicodeLetter(char c)

    • newToken

      public org.antlr.runtime.Token newToken(int ttype)

    • newTokenFromPreviousChar

      public org.antlr.runtime.Token newTokenFromPreviousChar(int ttype)

    • newToken

      public org.antlr.runtime.Token newToken(int ttype, String text, int pos)

    • newToken

      public org.antlr.runtime.Token newToken(int ttype, String text)

    • getSourceName

      public String getSourceName()
      Specified by:
      getSourceName in interface org.antlr.runtime.TokenSource

    • str

      public static String str(int c)