Package org.apache.maven.jxr.util
Class SimpleWordTokenizer
- java.lang.Object
-
- org.apache.maven.jxr.util.SimpleWordTokenizer
-
public class SimpleWordTokenizer extends java.lang.ObjectThis is a small and fast word tokenizer. It has different characteristics from the normal Java tokenizer. It only considers clear words that are only ended with spaces as strings. EX: "Flight" would be a word but "Flight()" would not.
-
-
Field Summary
Fields Modifier and Type Field Description private static char[]BREAKERSprivate static java.util.regex.PatternNONBREAKERS
-
Constructor Summary
Constructors Constructor Description SimpleWordTokenizer()
-
Method Summary
All Methods Static Methods Concrete Methods Modifier and Type Method Description private static intgetStart(java.lang.String string)Go through the list of BREAKERS and find the closes one.private static booleanisBreaker(char c)Return true if the given char is considered a breaker.static java.util.List<StringEntry>tokenize(java.lang.String line)Breaks the given line into multiple tokens.private static java.util.List<StringEntry>tokenize(java.lang.String line, int start)Internal impl.static java.util.List<StringEntry>tokenize(java.lang.String line, java.lang.String find)Tokenize the given line but only return those tokens that match the parameterfind.
-
-
-
Method Detail
-
tokenize
public static java.util.List<StringEntry> tokenize(java.lang.String line)
Breaks the given line into multiple tokens.- Parameters:
line- line to tokenize- Returns:
- list of tokens
-
tokenize
public static java.util.List<StringEntry> tokenize(java.lang.String line, java.lang.String find)
Tokenize the given line but only return those tokens that match the parameterfind.- Parameters:
line- line to search infind- String to match- Returns:
- list of matching tokens
-
tokenize
private static java.util.List<StringEntry> tokenize(java.lang.String line, int start)
Internal impl. Specify the start and end.
-
getStart
private static int getStart(java.lang.String string)
Go through the list of BREAKERS and find the closes one.
-
isBreaker
private static boolean isBreaker(char c)
Return true if the given char is considered a breaker.
-
-