# Functions
NewBlock makes a new Block with prepared text and a Selector.
NewBlockWithParent makes a new Block with prepared text, a Selector, and a parent.
NewIterTokenizer creates a new iterTokenizer.
NewLinedBlock creates a Block with an already-known location.
TextToTokens converts a string to a slice of tokens.
UsingContractions sets the provided contractions.
UsingEmoticons sets the provided map of emoticons.
UsingIsUnsplittable gives a function that tests whether a token is splittable or not.
UsingPrefixes sets the provided prefixes.
UsingSanitizer sets the provided sanitizer.
UsingSpecialRE sets the provided special regex for unsplittable tokens.
UsingSplitCases sets the provided splitCases.
UsingSuffixes sets the provided suffixes.
# Variables
SentenceTokenizer splits text into sentences.
WordTokenizer splits text into words.
# Structs
A Block represents a section of text.
Info handles NLP-related tasks.
IterTokenizer splits a sentence into words.
No description provided by the author
TaggedWord is a word with an NLP context.
No description provided by the author
# Interfaces
No description provided by the author
# Type aliases
No description provided by the author
No description provided by the author