Package com.google.googlejavaformat.java.javadoc

Class JavadocLexer

java.lang.Object
com.google.googlejavaformat.java.javadoc.JavadocLexer
final class JavadocLexer extends Object
Lexer for the Javadoc formatter.

  • Field Details

    • NON_UNIX_LINE_ENDING

      private static final Pattern NON_UNIX_LINE_ENDING

    • input

      private final CharStream input

    • braceDepth

      private final NestingCounter braceDepth

    • preDepth

      private final NestingCounter preDepth

    • codeDepth

      private final NestingCounter codeDepth

    • tableDepth

      private final NestingCounter tableDepth

    • somethingSinceNewline

      private boolean somethingSinceNewline

    • NEWLINE

      private static final com.google.common.base.CharMatcher NEWLINE

    • NEWLINE_PATTERN

      private static final Pattern NEWLINE_PATTERN

    • MOE_BEGIN_STRIP_COMMENT_PATTERN

      private static final Pattern MOE_BEGIN_STRIP_COMMENT_PATTERN

    • MOE_END_STRIP_COMMENT_PATTERN

      private static final Pattern MOE_END_STRIP_COMMENT_PATTERN

    • HTML_COMMENT_PATTERN

      private static final Pattern HTML_COMMENT_PATTERN

    • PRE_OPEN_PATTERN

      private static final Pattern PRE_OPEN_PATTERN

    • PRE_CLOSE_PATTERN

      private static final Pattern PRE_CLOSE_PATTERN

    • CODE_OPEN_PATTERN

      private static final Pattern CODE_OPEN_PATTERN

    • CODE_CLOSE_PATTERN

      private static final Pattern CODE_CLOSE_PATTERN

    • TABLE_OPEN_PATTERN

      private static final Pattern TABLE_OPEN_PATTERN

    • TABLE_CLOSE_PATTERN

      private static final Pattern TABLE_CLOSE_PATTERN

    • LIST_OPEN_PATTERN

      private static final Pattern LIST_OPEN_PATTERN

    • LIST_CLOSE_PATTERN

      private static final Pattern LIST_CLOSE_PATTERN

    • LIST_ITEM_OPEN_PATTERN

      private static final Pattern LIST_ITEM_OPEN_PATTERN

    • LIST_ITEM_CLOSE_PATTERN

      private static final Pattern LIST_ITEM_CLOSE_PATTERN

    • HEADER_OPEN_PATTERN

      private static final Pattern HEADER_OPEN_PATTERN

    • HEADER_CLOSE_PATTERN

      private static final Pattern HEADER_CLOSE_PATTERN

    • PARAGRAPH_OPEN_PATTERN

      private static final Pattern PARAGRAPH_OPEN_PATTERN

    • PARAGRAPH_CLOSE_PATTERN

      private static final Pattern PARAGRAPH_CLOSE_PATTERN

    • BLOCKQUOTE_OPEN_PATTERN

      private static final Pattern BLOCKQUOTE_OPEN_PATTERN

    • BLOCKQUOTE_CLOSE_PATTERN

      private static final Pattern BLOCKQUOTE_CLOSE_PATTERN

    • BR_PATTERN

      private static final Pattern BR_PATTERN

    • INLINE_TAG_OPEN_PATTERN

      private static final Pattern INLINE_TAG_OPEN_PATTERN

    • LITERAL_PATTERN

      private static final Pattern LITERAL_PATTERN

  • Constructor Details

    • JavadocLexer

      private JavadocLexer(CharStream input)

  • Method Details

    • lex

      static com.google.common.collect.ImmutableList<Token> lex(String input) throws JavadocLexer.LexException
      Takes a Javadoc comment, including ∕✱✱ and ✱∕, and returns tokens, including ∕✱✱ and ✱∕.
      Throws:
      JavadocLexer.LexException

    • normalizeLineEndings

      private static String normalizeLineEndings(String input)
      The lexer crashes on windows line endings, so for now just normalize to `\n`.

    • stripJavadocBeginAndEnd

      private static String stripJavadocBeginAndEnd(String input)

    • generateTokens

      private com.google.common.collect.ImmutableList<Token> generateTokens() throws JavadocLexer.LexException
      Throws:
      JavadocLexer.LexException

    • readToken

      private Token readToken() throws JavadocLexer.LexException
      Throws:
      JavadocLexer.LexException

    • consumeToken

      private Token.Type consumeToken() throws JavadocLexer.LexException
      Throws:
      JavadocLexer.LexException

    • preserveExistingFormatting

      private boolean preserveExistingFormatting()

    • checkMatchingTags

      private void checkMatchingTags() throws JavadocLexer.LexException
      Throws:
      JavadocLexer.LexException

    • joinAdjacentLiteralsAndAdjacentWhitespace

      private static com.google.common.collect.ImmutableList<Token> joinAdjacentLiteralsAndAdjacentWhitespace(List<Token> input)
      Join together adjacent literal tokens, and join together adjacent whitespace tokens.

      For literal tokens, this means something like ["<b>", "foo", "</b>"] => ["<b>foo</b>"]. See LITERAL_PATTERN for discussion of why those tokens are separate to begin with.

      Whitespace tokens are treated analogously. We don't really "want" to join whitespace tokens, but in the course of joining literals, we incidentally join whitespace, too. We do take advantage of the joining later on: It simplifies inferParagraphTags(java.util.List<com.google.googlejavaformat.java.javadoc.Token>).

      Note that we do not merge a literal token and a whitespace token together.

    • inferParagraphTags

      private static com.google.common.collect.ImmutableList<Token> inferParagraphTags(List<Token> input)
      Where the input has two consecutive line breaks between literals, insert a <p> tag between the literals.

      This method must be called after joinAdjacentLiteralsAndAdjacentWhitespace(java.util.List<com.google.googlejavaformat.java.javadoc.Token>), as it assumes that adjacent whitespace tokens have already been joined.

    • optionalizeSpacesAfterLinks

      private static com.google.common.collect.ImmutableList<Token> optionalizeSpacesAfterLinks(List<Token> input)
      Replaces whitespace after a href=...> token with an "optional link break." This allows us to output either <a href=foo>foo</a> or <a href=foo>\nfoo</a>, depending on how much space we have left on the line.

      This method must be called after joinAdjacentLiteralsAndAdjacentWhitespace(java.util.List<com.google.googlejavaformat.java.javadoc.Token>), as it assumes that adjacent whitespace tokens have already been joined.

    • deindentPreCodeBlocks

      private static com.google.common.collect.ImmutableList<Token> deindentPreCodeBlocks(List<Token> input)
      Adjust indentation inside `
      ` blocks.

 <p>Also trim leading and trailing blank lines, and move the trailing `` to its own line.

    • deindentPreCodeBlock

      private static void deindentPreCodeBlock(com.google.common.collect.ImmutableList.Builder<Token> output, com.google.common.collect.PeekingIterator<Token> tokens)

    • hasMultipleNewlines

      private static boolean hasMultipleNewlines(String s)

    • fullCommentPattern

      private static Pattern fullCommentPattern()

    • openTagPattern

      private static Pattern openTagPattern(String namePattern)

    • closeTagPattern

      private static Pattern closeTagPattern(String namePattern)