/* foo */.

-- foo.

anything else.

"foo_bar""baz".

semicolon ending a query.

'foo', $$bar$$, $a$baz$a$.

Token is the minimal set of lexical tokens for SQL that we need to extract queries.