Package org.jparsec

Class Terminals

java.lang.Object

org.jparsec.Lexicon

org.jparsec.Terminals

public final class Terminals
extends Lexicon

Provides convenient API to build lexer and parsers for terminals. The following example is a parser snippet for Java generic type expression such as List<String>:

   
   Terminals terms = Terminals
       .operators("?", "<", ">", ",")
       .words(Scanners.IDENTIFIER)
       .keywords("super", "extends")
       .build();
   Parser<String> typeName = Terminals.identifier();
   Parser<?> wildcardWithUpperBound = terms.phrase("?", "extends");
   ...
   parser.from(terms.tokenizer(), Scanners.WHITESPACES.optional()).parse("List<String>");
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	Terminals.Builder	Builds Terminals instance by defining the words and keywords recognized.
static class	Terminals.CharLiteral	Entry point for parser and tokenizers of character literal.
static class	Terminals.DecimalLiteral	Entry point for parser and tokenizers of decimal number literal represented as String.
static class	Terminals.Identifier	Entry point for parser and tokenizers of regular identifier.
static class	Terminals.IntegerLiteral	Entry point for any arbitrary integer literal represented as a String.
static class	Terminals.LongLiteral	Entry point for parser and tokenizers of integral number literal represented as Long.
static class	Terminals.ScientificNumberLiteral	Entry point for parser and tokenizers of scientific notation literal.
static class	Terminals.StringLiteral	Entry point for parser and tokenizers of string literal.

Field Summary

Fields

Modifier and Type	Field	Description
static Parser<java.lang.String>	RESERVED	Parser that recognizes reserved word tokens.

Fields inherited from class org.jparsec.Lexicon

tokenizer, words

Constructor Summary

Constructors

Modifier	Constructor	Description
private	Terminals(Lexicon lexicon)

Method Summary

Modifier and Type	Method	Description
static Terminals	caseInsensitive(java.lang.String[] ops, java.lang.String[] keywords)	Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals	caseInsensitive(Parser<java.lang.String> wordScanner, java.lang.String[] ops, java.lang.String[] keywords)	Deprecated. Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals	caseInsensitive(Parser<java.lang.String> wordScanner, java.lang.String[] ops, java.lang.String[] keywords, java.util.function.Function<java.lang.String,?> wordMap)	Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.
static Terminals	caseSensitive(java.lang.String[] ops, java.lang.String[] keywords)	Deprecated. Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.
static Terminals	caseSensitive(Parser<java.lang.String> wordScanner, java.lang.String[] ops, java.lang.String[] keywords)	Deprecated. Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.
static Terminals	caseSensitive(Parser<java.lang.String> wordScanner, java.lang.String[] ops, java.lang.String[] keywords, java.util.function.Function<java.lang.String,?> wordMap)	Deprecated. Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.
(package private) static void	checkDup(java.lang.Iterable<java.lang.String> a, java.lang.Iterable<java.lang.String> b)
static Parser<java.lang.String>	fragment(java.lang.Object... tags)	Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.
(package private) static TokenMap<java.lang.String>	fromFragment(java.lang.Object... tags)	Returns a TokenMap object that only recognizes Tokens.Fragment token values tagged with one of tags.
static Parser<java.lang.String>	identifier()	Returns a Parser that recognizes identifiers (a.k.a words, variable names etc).
static Terminals	operators(java.lang.String... ops)	Returns a Terminals object for lexing the operators with names specified in ops.
static Terminals	operators(java.util.Collection<java.lang.String> ops)	Returns a Terminals object for lexing the operators with names specified in ops.
Terminals.Builder	words(Parser<java.lang.String> wordScanner)	Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).

Methods inherited from class org.jparsec.Lexicon

fallback, phrase, token, token, tokenizer, union, word

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RESERVED

public static final Parser<java.lang.String> RESERVED

Parser that recognizes reserved word tokens. i.e. Tokens.Fragment tokens tagged as Tokens.Tag.RESERVED. Tokens.Fragment.text() is returned as parser result.

Constructor Detail

Terminals

private Terminals(Lexicon lexicon)

Method Detail

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(java.lang.String[] ops,
                                        java.lang.String[] keywords)

Deprecated.

Use operators(ops) .words(Scanners.IDENTIFIER) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.

Parameters:

ops - the operator names.

keywords - the keyword names.

Returns:

the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(java.lang.String[] ops,
                                      java.lang.String[] keywords)

Deprecated.

Use operators(ops) .words(Scanners.IDENTIFIER) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

A word is defined as an alphanumeric string that starts with [_a - zA - Z], with 0 or more [0 - 9_a - zA - Z] following.

Parameters:

ops - the operator names.

keywords - the keyword names.

Returns:

the Terminals instance.

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(Parser<java.lang.String> wordScanner,
                                        java.lang.String[] ops,
                                        java.lang.String[] keywords)

Deprecated.

Use operators(ops) .words(wordScanner) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:

wordScanner - the scanner that returns a word in the language.

ops - the operator names.

keywords - the keyword names.

Returns:

the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(Parser<java.lang.String> wordScanner,
                                      java.lang.String[] ops,
                                      java.lang.String[] keywords)

Deprecated.

Use operators(ops) .words(wordScanner) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:

wordScanner - the scanner that returns a word in the language.

ops - the operator names.

keywords - the keyword names.

Returns:

the Terminals instance.

caseInsensitive

@Deprecated
public static Terminals caseInsensitive(Parser<java.lang.String> wordScanner,
                                        java.lang.String[] ops,
                                        java.lang.String[] keywords,
                                        java.util.function.Function<java.lang.String,?> wordMap)

Deprecated.

Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .caseInsensitiveKeywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case insensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:

wordScanner - the scanner that returns a word in the language.

ops - the operator names.

keywords - the keyword names.

wordMap - maps the text to a token value for non-keywords recognized by wordScanner.

Returns:

the Terminals instance.

caseSensitive

@Deprecated
public static Terminals caseSensitive(Parser<java.lang.String> wordScanner,
                                      java.lang.String[] ops,
                                      java.lang.String[] keywords,
                                      java.util.function.Function<java.lang.String,?> wordMap)

Deprecated.

Use operators(ops) .words(wordScanner) .tokenizeWordsWith(wordMap) .keywords(keywords) .build() instead.

Returns a Terminals object for lexing and parsing the operators with names specified in ops, and for lexing and parsing the keywords case sensitively. Parsers for operators and keywords can be obtained through Lexicon.token(java.lang.String...); parsers for identifiers through identifier().

In detail, keywords and operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. Words that are not among keywords are lexed as Fragment with Tokens.Tag.IDENTIFIER tag.

Parameters:

wordScanner - the scanner that returns a word in the language.

ops - the operator names.

keywords - the keyword names.

wordMap - maps the text to a token value for non-keywords recognized by wordScanner.

Returns:

the Terminals instance.

operators

public static Terminals operators(java.lang.String... ops)

Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").

If words and keywords need to be parsed, they can be configured via words(org.jparsec.Parser<java.lang.String>).

Parameters:

ops - the operator names.

Returns:

the Terminals instance.

operators

public static Terminals operators(java.util.Collection<java.lang.String> ops)

Returns a Terminals object for lexing the operators with names specified in ops. Operators are lexed as Tokens.Fragment with Tokens.Tag.RESERVED tag. For example, to get the parser for operator "?", simply call token("?").

If words and keywords need to be parsed, they can be configured via words(org.jparsec.Parser<java.lang.String>).

Parameters:

ops - the operator names.

Returns:

the Terminals instance.

Since:

2.2

words

public Terminals.Builder words(Parser<java.lang.String> wordScanner)

Starts to build a new Terminals instance that recognizes words not already recognized by this Terminals instance (typically operators).

By default identifiers are recognized through identifier() during token-level parsing phase. Use Terminals.Builder.tokenizeWordsWith(java.util.function.Function<java.lang.String, ?>) to tokenize differently, and choose an alternative token-level parser accordingly.

Parameters:

wordScanner - defines words recognized by the new instance

Since:

2.2

identifier

public static Parser<java.lang.String> identifier()

Returns a Parser that recognizes identifiers (a.k.a words, variable names etc). Equivalent to Terminals.Identifier.PARSER.

Since:

2.2

fragment

public static Parser<java.lang.String> fragment(java.lang.Object... tags)

Returns a Parser that recognizes Tokens.Fragment token values tagged with one of tags.

fromFragment

static TokenMap<java.lang.String> fromFragment(java.lang.Object... tags)

Returns a TokenMap object that only recognizes Tokens.Fragment token values tagged with one of tags.

checkDup

static void checkDup(java.lang.Iterable<java.lang.String> a,
                     java.lang.Iterable<java.lang.String> b)