Mon Aug 17 10:01:30 CEST 2009

New interpreter

I started working on a new PF interpreter in [1].  The key changes are

  1. proper tail calls 
  2. a safe memory model

The new memory model is a combination of a graph memory with a simple
asynchronouse garbage collector (GC) for code and variable box
storage, and a clearly separated tree memory with linear lists (2
stacks + free lists) and a synchronous reference counting (RC) based
leaf object (= packets) manager.

This architecture is based on the assumption that for applications PF
is intended for (DSP scripting), compilation and variable allocation
happen only at program load time, while the ``infinite loop at the
end'' is usually linear, and benefits from the synchronous
finalization property of an RC-based memory manager, which leads to
smaller heap sizes and better data locality: ideal for DSP

Instead of separating these in a staged solution where a compiler
executable generates an executable with proper linear or static memory
management, the new PF engine has both of them in the same interpreter
image.  The compiler uses GC, while the compiled programs do not.

In other words: when your script loads you can use all kinds of
macro-fu to generate the DSP code.  Then you can essentially turn off
the asynchronous GC and run the resulting program.  This cycle can be
repeated without leaving the application.

The new PF core is implemented in a dynamically typed expression
language EX, which is essentially Scheme running on a C stack.  The EX
language is also used to define SC, a proper Scheme interpreter that
could be used as a metaprogramming system for PF code.

The main goals at this moment are simplicity of implementation,
reflection (very useful for debugging), and small code size.  Current
PF primitive C functionality (i.e. data processing, video IO and
OpenGL) is not ported yet.  This will be done as needed.

[1] http://zwizwa.be/darcs/libprim