Sun Nov 19 12:57:01 EST 2006

clean up readme : the stuff that nobody reads

What is this?

First of all, it's actually an application, but it's packaged
as a library to ease integration in other systems. Currently
there are 2 operational interfaces: as a unix command and as a
puredata object.

Second of all, it's actually a language. The application core
is written in C, and can be used in programs that have a C
language binding. Most of the core functions are available as
primitives in a language based on Forth. These primitives can
be used to build things related to media processing.

Where does it come from?

This project is a second incarnation of a media processing
library called pf, which is an extension for the computer music 
program Pure Data.


Since one of the goals of pf is to serve as a 'media glue language'
there are a lot of ways to connect to other programs or objects.

The pf object for Pure Data connects Packet Forth with original
pf, pidp and 0.12.x pf protocol compatible objects. Most of this
works, or at least should work. You can help by filing a bug report
to tom@zwizwa.be or the pure data mailing list if you find
a broken or missing feature in the pf version of pf.

The unix command acts as an interactive interpreter to the
forth dialect. When connected to a terminal it lauches a readline
enabled command line console. The line editor runs in a separate 
thread, and is polled by the worker thread, which runs the user 
definable 'schedule' and 'sync' functions.

If the input is not a terminal the readline part is skipped and the 
stream is parsed in blocking mode, just like would be done when
the argument is a file. This way you can send forth commands to
a pf instance, i.e. from perl or an external gui.

Packet forth supports streams. These can be files, sockets, fifos or
the input/output of inferior processes. Load and save of packet data in the
native format is supported for all streams. yuv4mpeg is supported,
so you can use it in conjuction with programs that support this (veejay,
mjpegtools, mplayer and some smaller utilities.)

Pf can be mapped to guile. This effectively enables all the powers
of pf combined with scheme syntax, garbage collection and all kinds
of sillyness.


Most library dependencies are wrapped in plugins to decouple
them from the core. See the file PLUGINS for more information.

Why the fork?

I don't really have an explanation for this. But i enjoyed the 
year of turbulence which has brought it to what it is now.
And remember.. there is no fork.

Why the forth?

Quirky as it is, it stimulates the brain in different, and
even orthogonal pleasure dimensions. It is the ultimate in 
one word hacking, and man, have you ever got lost in a 3 line 

Why not an existing scripting language?

Because those that i considered (python and scheme) were too
heavyweight and did not fit some requirements. The most important
reason is of course lack of control about internal workings.
Forking an existing language is more complicated than writing
a forth from scratch. Other languages will eventually be 
supported using direct api mappings.

I didn't even remotely think of considering perl as a core
language, but it does work very nice together with perl just 
doing 'open PF, "|pf";', or in conjunction with sockets.
This would be the way to connect an external gui to pf.


Packet forth does not have a garbage collector, for the simple
reason that it is not necessary. Packets are managed using
reference counts, and because they cannot refer to other
packets themselves, this is sufficient. Add to that the 
fact that a real-time garbage collector is not an easy 
thing to write, i've opted to leave it out entirely.

Packets use reference counts and copy on write.
All other atoms and containers (lists) are copied. Abstract 
objects (i.e. make-xv) need explicit freeing. Most objects 
are for io and need explicit resource cleanup anyway.

Note that memory allocated for packets does not get freed.
Instead all packets use a FIFO reuse stack per unique type
symbol to ensure the memory is 'as hot as possible'.

For normal operation this is not a problem, but if you
start creating a whole lot of packets of different types 
(i.e. loading different image sizes) you need to manually
clear the type cache when you are done using a type. This 
is not 'intended use' of pf and actually considered a 
good thing, because it keeps things simple and predictable. 
Pf does not solve all your problems :)

Portability & Requirements

I set out to make libpf as portable as possible, but
unfortunately i had to drop this constraint to keep it small
and simple. At the moment, GNU libc is required, and gcc-3.x

So compiling on a GNU or similar system other than GNU/Linux
should work with relatively minor effort (OSX/fink is one of the
intended targets), as is MSW/cygwin. Porting to a non-GNU
system might be quite an ifdef ride though.


I did my best to eliminate the most obvious buffer overflows,
but i can't guarante anything. Remember that almost all
data files are actually executable, (which can be worked around),
and pf can execute arbitrary programs and directly modify memory
and disk when running as root. Sending forth code over the network 
is a very powerful mechanism, but incredibly insecure. If you intend
to use pf as a server or as part of a server exposed to the internet 
or another non-trusted network, be careful. It is very promiscuous.