Tue Sep 1 22:03:26 CEST 2009

Your generator is my macro

Popular notions

As pointed out to me recently, I should not forget the existence of
the popular[1].

To be fair, the site is quite broad.  But I can't help but get the
feeling that a lot of the complexity is because of deficiencies in
some high level programming languages, mostly Java.

It seems that lisp-style macros would make this kind of code
generation a lot less painful.  To me (and my target domain) most of
this seems a bit moot.  I would be interesting to learn about
something more powerful and well-designed than hygienic macros and

A spectrum of metaprogramming techniques

As mentioned in this LtU post[2], there is a spectrum of code
generation techniques:

   rigid and static                                    flexible and dynamic
<--passive generators---active generators--macros--compilers---interpreters-->

I tend to stick to the macros-compilers part, with excursions into
active code generation and interpreters.  Let me give my list. 

- AST interpreters (PF, embedded Scheme)
- byte code interpreters
- compilers -> C / asm / LLVM / VM (control / data structure flattening)
- typed/untyped heterogenous staged metaprogramming (MetaOCaml / Staapl)
- typed homogenous staged metaprogramming (MetaOCaml) and
  untyped lisp macros / AST tx tools (compile to Scheme/Lisp/OCaml/Haskell)
- active code generators (simple gen of header files, macro defs, struct inits, ...)
- passive generators (emacs keyboard macros)

Some remarks

The last one: passive generators, should be used with care.  I use
them only as `sanitizers': converting permanently to a new data
format, and then taking that as a root.

Active generators I call simple configuration file -> init file /
interface definition file conversion.  Essentially flat direct

Lisp macros and AST tx tools are really powerful in-language
metaprogramming.  Missing link: (gradually) typed macros.  Typed
homogenous staged programming fills the void made by restricted typed
macros.  I'm not sure about the usefulness of this yet.

Heterogenous staged programming (HSP) is useful for separating
high-level program specification from implementation as a high -> low
map.  This is a restricted form of compilation.  HSP comes close to
`real' language design and implementation.

Full high level to low level (ASM) compilers are best left to experts.
Compilation to intermediate level is probably best phrased as HSP (for
C) or macros (for higher level targets).

Interpreters: anything that doesn't `stage' a solution is an
interpreter.  Most complex data structures can be seen as languages.
Interpreters are best written in high level languages.

[1] http://www.codegeneration.net
[2] http://lambda-the-ultimate.org/node/1495#comment-17349