sascangen: simple sather scanner generator

basic useage

the input file consists any number of tokens and regular expressions of the form

Token ::= regular expression

sascangen will then generate to stdout a sather class which when given an a character iterator which conforms to $ELT{CHAR} will have an iterator scan!:INT which iterates over the tokens scanned from the $ELT{CHAR}. SCANNER has several user accessable members.

token:FSTR the text contained within the current token
line_number:INT the current line number
char_number:INT the current char number within the current line
each token is represented by an integer constant of the same name
create(elt:$ELT{CHAR}):SAME is the basic constructor
token_to_str(t:INT):STR will convert a token to its string representation. this is mainly useful for debugging.
scan!:INT the main scanner iterator

caveats/TODO

this release is of very alpha quality. it does not attempt to optimize the state table in any fashion. several more optimized interfaces (such as direct file lexing) could be implemented. the character and line number attributes should be optional somehow. even Ignore'd values are saved up in token. probably many bugs still exist. should create some sort of macro facility. does not work with unicode. generator speed could be improved dramatically.

special tokens

'Ignore' will be ignored in the resultant scanner and not returned to the application. this may be used for whitespace and comments.
'Error' will be returned for any text that does not match one of the given regular expressions.

expressions

_ matches any single character
[432a-z] matches a 4, 3, or 2 or a through z
[^a-z] matches NOT a through z
special meaning may be turned off with a \ as in [\-+/*] to match binary operators
strings surrounded by quotes (double or single) are matched verbatim
standard escapes may be used (ie \n for EOL)
parentheses will group expressions
| is alternation
+ means one or more of the preceding
* means zero or more of the preceding
? means one or zero of the preceding

example

this is the input file used by sascangen itself


CharClass ::= '[' (('\\' _)|[^\]\n\r\\])+ ']'
String ::= "'" ( '\\' _ | [^\n\\\r'])+ "'"
String ::= '"' ( '\\' _ | [^\n\\\r"])+ '"'
Ignore ::= "--" [^\n] * 
Ignore ::= [ \n]+
Semicolon ::= ';'
LParen ::= '('
RParen ::= ')'
Star ::= '*'
Plus ::= '+'
Option ::= '?'
Equals ::= "::="
UnderScore ::= '_'
Ident ::= [A-Za-z_]+
Pipe ::= '|'

contact info

Author: John Meacham john@foo.net
Homepage: http://synergy.foo.net/~john/computer/sather/

copyright

Copyright (c) 1999 John Meacham

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

The originator of this code may not be misrepresented, any derivative
works must be clearly marked as such.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.