# README
parsing in golang
Simple lexing and parsing of lightweight languages, based on the assumption of whitespace separation of non-symbol tokens with C++-style single- (//) and multi-line (/.../) comments.
The lexer can be fine-tuned/extended by adding 'intercepts', the parser can be extended by adding rules to combine tokens.
Includes helper functions for goroutine-safe application wide stats counting and timing.
TODO:
- Extend Documentation
- Round-out Tests
- Add Examples
- Make stats/counters non-singleton so that user can make that decision
- Parser generators
# Packages
No description provided by the author
# Functions
CommonCommandLine parses common command line options.
FindFiles will iterate across the given list of paths or ".", finding files that match the given extension, and forwarding their fully-qualified paths to the supplied channel.
IsAlpha returns true for any character a-z or A-Z.
IsIdentifierContinuation returns true for any tokens that are allowed to continue an identifier (2nd character+).
IsNumeric will return true if the given character is a digit or period.
No description provided by the author
NewLexer constructs a new Lexer instance.
NewParser will construct a new parser instance and read-ahead the first two symbols.
NewTerminal will return a new Token with a friendly, lowercase name.
NewToken will return a new Token with the friendly name given, with an all uppercase name.
ParseFiles will locate any files matching 'extension' in the current pathlist and run the given parsing function on them using a worker pool.
PathList will either return the input list of paths if it is non-empty, or it will return a list consisting of just the current directory (".").
RaiseHandler is a default implementation of a Raise handler which prints the error to stderr..
# Variables
Intermediate classifications.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
SomethingToken denotes a class of token rather a match to a single explicit value.
Concurrency determines how many parse workers run simultaneously.
SomethingToken denotes a class of token rather a match to a single explicit value.
Tokens resolving to a single literal value.
EOFByte will be returned by Read and Peek when EOF is reached, but may also be returned if the stream contains binary data.
SomethingToken denotes a class of token rather a match to a single explicit value.
Tokens resolving to a single literal value.
SomethingToken denotes a class of token rather a match to a single explicit value.
Identifiers.
SomethingToken denotes a class of token rather a match to a single explicit value.
Noise.
Tokens resolving to a single literal value.
SomethingToken denotes a class of token rather a match to a single explicit value.
Symbols.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
ProjectPath is the top-level path under which any non-absolute, non ".", paths specified on the command line will be searched.
Tokens resolving to a single literal value.
Tokens resolving to a single literal value.
Stats enables reporting of statistics in output.
Literals.
SomethingToken denotes a class of token rather a match to a single explicit value.
TokenMap maps individual ASCII character values to base Token types.
Tokens resolving to a single literal value.
SomethingToken denotes a class of token rather a match to a single explicit value.
# Structs
Lexer tracks parsing of a source stream.
Parser is a core implementation of a lexing-based parser implementation.
No description provided by the author
Symbol describes a specific instance of a token within the source stream.
Token identifies a significant pattern in a code stream, from a specific keyword to an integer to whitespace to end-of-file.
# Type aliases
Intercept is a callback that attempts to apply grammar-specific rules to a character's token lookup before we apply the baseline rules.
InterceptTable is a dictionary of lists of intercepts.