Package edu.umd.cs.findbugs

Class Tokenizer

java.lang.Object

edu.umd.cs.findbugs.Tokenizer

public class Tokenizer
extends java.lang.Object

A simple tokenizer for Java source text. This is not intended to be a compliant lexer; instead, it is for quick and dirty scanning.

See Also:

Token

Field Summary

Fields

Modifier and Type	Field	Description
private java.io.PushbackReader	reader
private static java.util.BitSet	single
private static java.util.BitSet	whiteSpace

Constructor Summary

Constructors

Constructor	Description
Tokenizer(java.io.Reader reader)	Constructor.

Method Summary

Modifier and Type	Method	Description
private Token	maybeComment()
private Token	munchString(int delimiter)
Token	next()	Get the next Token in the stream.
private Token	parseWord()
private void	skipWhitespace()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

whiteSpace

private static final java.util.BitSet whiteSpace

single

private static final java.util.BitSet single

reader

private final java.io.PushbackReader reader

Constructor Detail

Tokenizer

public Tokenizer(java.io.Reader reader)

Constructor.

Parameters:

reader - the Reader for the Java source text

Method Detail

next

public Token next()
           throws java.io.IOException

Get the next Token in the stream.

Returns:

the Token

Throws:

java.io.IOException

skipWhitespace

private void skipWhitespace()
                     throws java.io.IOException

Throws:

java.io.IOException

munchString

private Token munchString(int delimiter)
                   throws java.io.IOException

Throws:

java.io.IOException

maybeComment

private Token maybeComment()
                    throws java.io.IOException

Throws:

java.io.IOException

parseWord

private Token parseWord()
                 throws java.io.IOException

Throws:

java.io.IOException