# Functions
Tokenize converts a string into a stream of tokens.
# Structs
TokenRange indicates the range of tokens that map to a particular checksum.
# Type aliases
Hash is a map of the hashes of a section of text to the token range covering that text.
TokenRanges is a list of TokenRange objects.
Tokens is a list of Token objects.