Mon Jun 9 21:22:40 CEST 2008

new parser driver

It's a mess. There have been several occasions where i tried to
understand it but couldn't. So, how to fix this. There are 2 things to

* whenever a definition starts, the name, srcloc, and mode need to be
  recorded. -> implement as thunk.

* whenever a definition ends, the current expression needs to be
  combined with the stored header information and collected as a record.

the basic driver seems to work. it's a lot simpler to understand now:

(define (definer mode)
  (lambda (code expr)
    ((finalize-current) expr)
    (syntax-case code ()
      ((_ name . code+)
       (new-record #'name
                   (stx-srcloc #'name))
       (collect-next #'code+)))))

now need to adapt all the other macros to this new way of doing
things.. should be straightforward. nesting can be implemented with
dynamic scope and an exit continuation like for load.

ok, with some minor shuffling in what to return to the continuation
(which is now implemented as a prompt) it seems to work.

now locals: maybe i can get back to using (rpn-represent)? this
requires the function to return.. is this possible?

ok: rpn-next can _only_ return when all input is parsed. this way
parsing can still be nested locally without the driver loop needing to
restart parsing.

ok, got some generic nesting working, now do the same for load so all
nestings can compose.

something seems to be wrong with 'load' though: probably a
continuation barrier.. nope: the procedure embedded in the syntax was
of course wrapped as a syntax object, and my printer routine
automatically unwraps it..

locals: the problem with not allowing rpn-next to return is of course
that now it is no longer possible to modify the closing expression
(the lambda wrapper): this used to happen by returning. the solution
is to add yet another parameter that represents expression closure.

it's actually already there in the form of 'rpn-lambda'
but this makes it a bit complicated..

the following modification should do it: allow 'locals-tx' to modify
rpn-lambda, and reset rpn-lambda in the forth parser so every
definition can start from a clear wrapper.

ok.. the thing is this: building an expression one wants to be able to
insert nesting expressions above and below: passing on just the inner
expression is a bad idea. maybe this needs to change? 3-value parser

hmm.. alternatively, write the parser in terms of scat threaded state
updates, but that might go too far and lead to bootstrap problems.

the problem is now to make sure that wrappings are only used
once. this needs an interface:

pfff... i'm getting myself into lowlevel mess again because one
feature doesn't fit into the simple abstraction. what about making
'expr' an expression generator: a list of functions that can be
composed and evaluated.


expr = a cursor inbetween:

(outer . inner)

this should really solve all parsing needs.

ok. got it working with a bit of juggling with rpn-lambda:

  * at every rpn-compile, the current expression wrapper is set from rpn-lambda.
  * expressions are allowed to override the current expression wrapper
  * if entry is not trough rpn-compile, you need to initialize the wrapper!

next problem: the locals macro seems to have a problem with non-2stack
states. solved: wasn't fixed after abstract state update was changed.