HOLE We add this arc to represent a hole.

POS_SEP We create transition between two adjacent tokens.

CharArraySet A simple class that stores Strings as char[]'s in a hash table.

GraphTokenFilter An abstract TokenFilter that exposes its input stream as a graph Call incrementBaseToken() to move the root of the graph to the next position in the TokenStream, incrementGraphToken() to move along the current graph, and incrementGraph() to reset to the next graph based at the current root.

LowerCaseFilter Normalizes token text to lower case.

StopFilter Removes stop words from a token stream.

TokenStreamToAutomaton Consumes a TokenStream and creates an Automaton where the transition labels are UTF8 bytes (or Unicode code points if unicodeArcs is true) from the TermToBytesRefAttribute.

WordlistLoader Loader for text files that represent a list of stopwords.

An Analyzer builds TokenStreams, which analyze text.

FilteringTokenFilter Abstract base class for TokenFilters that may remove tokens.

StopWordAnalyzer Base class for Analyzers that need to make use of stopword sets.

A TokenFilter is a TokenStream whose input is another TokenStream.

A TokenStream enumerates the sequence of tokens, either from Fields of a Document or from query text.