module Genlex: Genlextype token =
| |
Kwd of |
| |
Ident of |
| |
Int of |
| |
Float of |
| |
String of |
| |
Char of |
The type of tokens. The lexical classes are: Int and Float
for integer and floating-point numbers; String for
string literals, enclosed in double quotes; Char for
character literals, enclosed in single quotes; Ident for
identifiers (either sequences of letters, digits, underscores
and quotes, or sequences of 'operator characters' such as
+, *, etc); and Kwd for keywords (either identifiers or
single 'special characters' such as (, }, etc).
val make_lexer : string list -> char Stream.t -> token Stream.tConstruct the lexer function. The first argument is the list of
keywords. An identifier s is returned as Kwd s if s
belongs to this list, and as Ident s otherwise.
A special character s is returned as Kwd s if s
belongs to this list, and cause a lexical error (exception
Stream.Error with the offending lexeme as its parameter) otherwise.
Blanks and newlines are skipped. Comments delimited by (* and *)
are skipped as well, and can be nested. A Stream.Failure exception
is raised if end of stream is unexpectedly reached.