HasUChars returns whether the lexer is at the beginning of a escaped Unicode character.

HasXChars returns whether the lexer is at the start of a escaped hexadecimal byte (e.g \xFE) XCHAR ::= '\x' HEX HEX.

IRIRef emits an IRIREF or returns an error if the input is invalid.

IsEndOfLine returns true if the rune is a Linefeed or a Carriage return.

EOF indicates the end of the an input.

ItemEOF is emitted when the end of the input is reached.

ItemError is emitted when there was an error lexing the input.

Item represents a unit emitted by the lexer.

ItemIterator iterates over the items emitted by a lexer.

Lexer converts a raw input into tokens.

A RuneWidth represents a consecutive string of runes with the same width and the number of runes is stored in count.

CheckRune is predicate signature for accepting valid runes on input.

CheckRuneRec is like CheckRune with Lexer as extra argument.

ItemType is used to set the type of a token.

StateFn represents the state of the scanner as a function that returns the next state.