Tue May 27 11:06:38 CEST 2008

parser cleanup

EDIT: this is a mess.. i've reached the conclusion that the 3 layers
are necessary, so it might be best to think a bit about the best way
to represent the highest layer..

gut feeling says current problems with parser (the non-concatenative
part of Forth syntax) are rooted in the way it's implemented.

phase 1: what is ':' ?

this, in addition to creating a new label, will terminate the previous
definition. following the colorForth model, there is no interpret
mode, so definitions run upto the next word. those 2 behaviours need
to be split up.

( simplifying, why not have macros with multiple entry points? this
would not be too hard to solve really. )

so, let's have a look at the single-assignment language for Purrr
parsing. the problem to solve is:

   How to implement a Forth-style syntax (without interpret mode) on
   top of the purely concatenative Macro syntax used by Purrr.

To re-iterate why this is a problem:

  * I'm convinced purely functional and purely compositional macros
    are a good idea: they behave well as a data structure for program
    representation and code generation (compiler structure).

  * Forth is only a thin layer on top of this, mostly to solve
    definition of names in a source file using a familiar
    syntax. Forth syntax in itself is a good user interface. However,
    the design of the Forth language is firmly rooted in reflectivity,
    employing an image based incrementally extensible word dictionary,
    something which i'm trying to unroll into a language tower.

  * There is already a base syntax using s-expressions which
    translates directly to Scheme. The problem now is to find a way to
    translate (linear) forth into tree-structured Macro expressions.

One solution is to

  * create an intermediate language syntax that has all the features
    needed in Forth, but uses s-expressions and single assignment.

  * map this language to an explicit list of definitions

  * find a mapper from linear forth -> s-expressions

This should work for the 3 levels of code:

    - macros
    - words (instantiated macros)
    - parser (Forth source transform patterns)

First attempt in forth/single.ss

Maybe the real focus should be to embed s-expressions in the Forth
language. This gives parse-time lists.

Care should be taken not to create yet another level that's hard to
metaprogram. This needs to sink in a bit.