PDP Developer Documentation

There is not yet much developer information, partly because pdp is not that big and since the goals are - not completely clear yet, a lot will probably change on the inside in the future. I believe it is - not too hard to figure out how it works, once you get started somewhere. This document is a minimalistic - attempt to provide that starting point. For full prototypes see the header files. I suggest you have a look at the pdp_base base class, and some simple - modules: pdp_add, pdp_noise and pdp_gain for examples. - -

PDP architecture

Architecture is a big word, but pdp is organized as modules. A packet pool module (a reuse pool memory manager), - a packet class, a processing queue module, a high level type conversion module, an image packet class, and some - low level modules for image type conversion, image resampling and all sorts of other image processing. Besides that - there are 2 extension libraries: pdp_scaf, a cellular automata extension and pdp_opengl, a 3d rendering extension. - These are separate because of portability issues. The different pdp_* externs in the main pdp library use the - core modules' functionality to minimize code duplication. I'm relatively happy with how it fits together, - but some things need to change for future plans. Most objects are written in the object oriented c style of pd. - To prevent namespace conflicts, (almost) all routines start with the pdp_ prefix. The second name is the name of the - object or module they belong to. The first argument is always a pointer to an object or an integer (for packets). - - -

PD ties

PDP is written as an extension for PD. One of the goals of pdp is to evolve to a separate library that can - be reused in other software. The architecture will be split into two parts. A pd-independent part (the packet classes, - the packet pool, the type conversion system and the forth system) and a part with pd specific stuff (the process queue and interfaces to the - pd system like the base classes and the pd communication protocol). In order to do this the packet class will probably - evolve to a proper object model, supporting run time attribute binding (inspired by the python object model). - -

There are some things that put a stamp on the current pdp design. Most importantly pd's processor object model and - communication protocol. (i.e. the fact that pd only supports unidirectional messaging creates the awkward concept - of a "passing packet" to eliminate excessive data copying.) - -

In pd, the pdp messaging protocol is implemented as pd messages. The protocol is however 3 phase. - With a read only register phase, a read/write register phase and a process phase. This functionality - is part of the base class or the forth processor object. The dpd protocol is entirely different, - and is used in the opengl library. It is - not based on parallel dataflow but serial context passing. - -

Packets

PDP introduces a new atom: the data packet. This can contain all kinds of data. Images (16bit/8bit), cellular - automata (1bit), matrices (real/complex float/double), opengl textures and 3d rendering contexts. Packets - are stored in a pool to ensure fast reuse, and to enable sharing. The paradigm is centered around a - combination of an object oriented approach and a dataflow approach. -

The methods operating on packets - (pdp_packet_*) are mainly for administrative purposes: memory management (construction, registering, copying) - and getting or setting info. -

All processing is done in the pd modules. Processors can be defined using - the forth scripting language, but this is still experimental. The forth system can be accessed - from the guile library. -

There is a central mechanism for packet type conversion. This is to facilitate the combination of different - media types. Whenever a packet class is constructed (i.e. in an extension library), a number of conversion - routines should be defined to convert the added type to one or some of the main pdp types. - - - - - - -

PDP API Overview

pdp_packet_* -
new	construct a raw packet (depreciated) -
new_*	construct packet of specific type/subtype/... -
mark_unused	release -
mark_passing	conditional release (release on first copy ro/rw) -
copy_ro	readonly (shared) copy -
copy_rw	private copy -
clone_rw	private copy (copies only meta data, not the content) -
header	get the raw header (t_pdp *) -
data	get the raw data (void *) -
pass_if_valid	send a packet to pd outlet, if it is valid, and mark unused -
replace_if_valid	delete packet and replace with new one, if new is valid -
copy_ro_or_drop	copy readonly, or don't copy if dest slot is full + send drop notify -
copy_rw_or_drop	same, but private copy -
get_description	retrieve type info -
convert_ro	same as copy_ro, but with an automatic conversion matching a type template -
convert_rw	same as convert_ro, but producing a private copy -

The pool object methods. All the packets are stored in a central packet pool. - - -

pdp_pool_* -
collect_garbage	manually free all unused resources in packet pool -

- -

The process queue object methods. PDP supports a separate processing thread. - - -

pdp_queue_* -
add	add a process method + callback -
finish	wait until a specific task is done -
wait	wait until processing queue is done -

- -

The control methods. General pdp control messages. - - -

pdp_control_* -
notify_drop	notify that a packet has been dropped -

- -

The type mediator methods. - -

pdp_type_* -
description_match	check if two type templates match -
register_conversion	register a type conversion program - - -

- - -

NOTE: it is advised to derive your module from the pdp base class defined in pdp_base.h - instead of communicating directly with the pdp core - - - -

pdp_base class

pdp_imageproc_* modules

pdp_llconv call

Pure Data Packet

Introduction

Pure Data Packet (PDP) is an extension library for the computer music -program Pure Data (PD), by Miller Puckette and -others. Its goal is to provide a way to use arbitrary data types (data -packets) as messages that can be passed around inside PD, along side the -standard PD numbers and symbol types. In short it puts any data object on -the same level as a float or a symbol. - -

PDP runs on Linux and OSX. The OSX version depends on Fink, which is not in the "point & -click" stage yet, so setting it up will require some efford. There is no -windows version. The reason for this is simple: i don't use windows myself. -Porting would require writing code for input/output and getting the -libraries PDP depends on to work. If anyone is willing to do this, just let -me know. PDP can run without X Window, using SDL. - -

Currently, PDP's focus is on images and video, but there is no reason it -should stay like that. There is limited support for matrix processing -included in the main library (like Jitter or Gridflow). There is an -extension library for 1D and 2D binary cellular automata, opengl rendering -(like Gem). Some plans include audio buffers (like Vasp), ascii packets, -text buffers, ... Finally there's a library that enables you to connect a -scheme interpreter (guile) to PD/PDP. For more image processing objects, -have a look at Yves Degoyon's PiDiP library. - -

Getting Started

Packets and Types

PDP is centered around the concept of packets and operations on -packets. There are several types of packets. The default type for most -objects is image/P411/320x240. This is a single video -frame, encoded in the internal 16bit YUV format, measuring 320 by 240 -pixels. Another image type is the grayscale image -image/grey/320x240. Important notes: All image processing objects that -combine two or more packets need to be fed with the same packet types, i.e. -encoding (P411/grey) and dimensions need to be the same. Image dimensions need to be a -multiple of 8x8. - -

The -bitmap/*/* type is another image representation type -supporting several encodings. I.e. bitmap/rgb/*, -bitmap/rgba/*, bitmap/yv12/*, ... - -This type cannot be processed directly by most of the image processing -objects, but it can be used to store in delay lines, or to send over the -network. It's main use is to support all kinds of input/output devices, and -opengl textures, without introducing too many conversions, but it can serve -as a space and bandwidth saver too (especially -bitmap/yv12/*). - -

One of the interesting -features in PD is the possibility of connecting everything with everything. -If you want to generalize this to all kinds of media objects, the complexity -of managing the different types starts to grow quite fast. Therefore PDP has -a type conversion system that can take care of most of the conversions -using the [pdp_convert] object. You can manually convert -packets to a certain type by specifying a type template as a creation -argument. I.e. [pdp_convert image/grey/*] will convert -any packet to a greyscale image. Most of the conversion will become -automatic later on. - -

An example: You can use the basic PDP library together with the -cellular automata library and the opengl rendering library to use a cellular -automaton as an input to a video processing chain. You can convert the -processed image to a texture that can be applied to a 3d object, which then -can be drawn to the screen, captured as a texture, converted back to an -image, which can then be converted to a sound, processed and converted back -to an image, etc... You get the point. The possibilities are endless. - - - -

PDP Forth Scripting

Introduction

This document describes the rationale behind the pdp forth scripting - language, aka "pdp's rpn calculator". - - -

The point is to be able to decouple the packet processors from the pd - object model, in order to be able to port the functional part of pdp to other - systems. Therefore pdp needs its own processor model. - -

A requirement for this is that it is both flexible and simple. pd's object - model is very flexible, but it is hard to port to other architextures, because - of the tight bindings to the pd kernel. What is needed is a way to abstract the - functionality of a pdp object in such a way that a pd class can be - generated from the pdp processor model. - -

There are a lot of ways to solve this problem. One is to extend a packet - class to support methods. This seems like a very clean solution, however - when a processor has more than one packet as internal state, this poses - a problem. It would require to define an operation like biquad (which has 1 input - packet and 2 extra state packets) as a method of a packet collection. To - do this properly requires a much more advanced object model, or a separate - object model for processors. - - -

In short: it is not always clear if a packet processor can be seen as a metod - 'of' or an operation 'on' a single packet, so extending a packet with methods - would require a separate packet processor class anyway. Therefore we do not - define packet operations as methods of packets. (no object oriented solution) - -

Another approach is to write operators in a pure functional way: a processor - accepts a list of arguments and returns a new list of arguments. To do this - cleanly it would require the calling style to be pass by value: i.e. the arguments - passed will not be modified themselves. Since most of the image processors in - pdp all use in-place processing for performance reasons (minimal L2 cache - usage), this would require - a lot of workarounds or a big kludge mixing const and non const references - (like it would be done in C++). - -

Since one of the goals is to automate - the generation of wrappers of pdp processors (i.e. pd classes, python classes, - scheme functions, ...) the interface can best be kept as simple as possible. - (no pure functional solution) - -

The most natural solution, given the underlying code base, seems to be to embrace - an in place processing approach. What comes to mind is to use a data stack to solve - the communication problem. I.e. the forth language model. A packet operation is then - a forth word that manipulates a data stack. A data stack is nothing more than a list - of atoms containing the basic data building blocks: floats, ints, symbols and packets. - -

An additional advantage is that dataflow and forth mix very well. This would enable - the possibility to create new words by chaining old ones, without the disadvantage - that pd abstractions using pdp objects have: passive packets are contained in internal - processor registers longer than necessary, leading to inefficient cace memory - usage. The forth uses a copy on write strategy. Together with in place processing - this is optimal for effects chains without fanout. - -

Several practical problems need to be solved to implement this. The first being - description of the stack effect. Each primitive word has a mandatory description of - the number of stack elements it consumes and produces. - -

The forth words will support - polymorphy: each word has a type template to describe the type of atoms it can operate - on. The type is determined by some word on the stack, or a symbol indicating the type for - generators. - -

To solve the additional problem of mapping forth words to processors, the concept - of a forth process is introduced. A forth process has a stack (representing it's - state), an init method that constructs a stack, a process method that operates - on the stack and some description of how to map inputs to stack and stack to output. - The mapping between a forth process and a pd object is represented by a pdmap. - -

There is one last class of objects that don't fit the forth description - very well: it is the input/output classes. These will probably stay as special - cases, since they really need an internal state represented as an object, - incorporating them into the system would require them to be defined as an - object (quicktime packet, v4l packet, xv packet, ...). More later. - -

Implementation

- -Packet Forth - - -

-Introduction - -You know, there's something strange going on with forth. In the -last 2 years, my attitude towards forth has changed from being intrigued -by that language with the strange succession of words, through the process -of trying to write several forths myself, and inspecting a miracle of engineering: -ColorForth. Chuck Moore is a genious. - - All in all forth is a hack. It is very much determined by design -decisions mostly aimed at keeping it simple. It is very pragmatic. -But, it is a hack that solves one of those hard problems: finding a good -compression scheme for human and machine readable concepts. Aka code and -data. Packet Forth takes forth somewhat out of it's original "can't be too simple" context -by turning it into a more highlevel, typed language, while forth is usually -situated at the bit and machine instruction level. Somehow this typed approach -makes it feel more contemporary. Direct consequences are that it's slower, but less -error prone. Different, more in tune with other tools in the basket, and i'm looking -at perl, python and guile. - - - In case this is not convincing as a teaser to read more, let me give you a pragmatic example. -When i work with pf, i have usually a terminal running a pf session with some -things loaded. I want do try something. Mostly this is just a couple of commands. -But i will need them again later, so i make a new composite word. Then i try again, -etc. When i got something workin, i paste it to an editor. Or, i can work the other -way around, and send a line of commands to the interpreter when i'm working on -a forth program in the editor (emacs). This just works. The fact that it's forth, -doesn't matter that much, only that it leads to the smallest red tape overhead. Most -things are just one word. One symbol per abstract action. - - -

-Commands and Command Composition - - -Forth is commands. Each command does something with the current things. -The current things are kept on a stack. The stack can be printed using -the command .S, which starts with a -. like most words that send -something to the current output stream. The command consisting of -a single . prints the top atom -on the stack and discards it. Try this at the console: - - - -> 1 -> .S -<1> 1 -> - - - -The first command will load the integer number 1 -Now, let's get some more fuel. - - - -> 2 3 -> .S -<3> 1 2 3 -> - - - - -We now have 3 items in our bag. The hottest one is 3. It is on the top of -the stack. If a command is executed that consumes just one item, it will -be the 3. But we're not going to do that. We're going to try something -more exiting, which is to execute the command +: - - - -> + -> .S -<2> 1 5 -> - - - -It seems the 2 and 3 magically disappeared from the stack and were -replaced by a 5! Luckily, the 1 is still there, not touched by the evil -fivifier. Actually, + is not really a fivifier. -And besides, Packet Forth -does not come with fivifiers (tsss). So why don't we just extend it? - - - -Before we start the chapter on how to add a fivifier to Packet Forth, -we first make sure there is an item worth of fivification in the ready -position, meaning the top of the data stack. We do this using the command -'swap'. - - - -> swap -> .S -<2> 5 1 -> - - - -Now, the most important thing about forth in general is the ability to -extend the language with fivifiers, but also any other function or -operation you can think of that is a combination of known operations. A -new operation can be defined using the combo : - and ;. The first word -following the semicolon will be the name of the composite command, all the -other symbols until a semicolon will be part of the command definition. - - - -Let's define a fivifier. Admit it. The thing you've been waiting for. -Just type this at the '>' prompt and brace for horror. - - - -: vijver drop 1 1 + 1 + 1 + 1 + ; - - - -Now, this really is an evil fivifier. In adition to being pretty -rude as a fivifier, by discarding the first item instead of treating it -with proper respect, it also goes against the holy doctrine of the Church of -Chuck, which says thou shall factor! -Anyway. Lets check if we're really dealing with a fivifier. - - - - -> vijver -> .S -<2> 5 5 - - - -The top 1 is gone, and a five is put there instead. The fivifier seems -to be operational. -Now, let's clean this fivifier up a little. The recurrent pattern that -catches the eye above is '1 +'. This is actually way to verbose for an -operation with such small effect. Let's define this operation, 'to add one -to the item on the top of the stack', to be hitheron know as 'x': - - - -: x 1 + ; - - - -This means we can redefine the fivifier using the word 'x' we just -created. You may call me mad, but this is what is called compression of -ideomatic sequences. - - - -: vijver drop 1 x x x x ; - - - -Ok, enough sillyness. This however does illustrate the key concepts. -Forth is about words and stacks and this enables very easy factoring. -Factoring (chopping the large chunks into smaller bits) is essential to -writing understandable code in any programming language. Forth makes it -easy and almost a requirement, so why not: - - - -: y x x ; -: z y y ; -: vijver drop 1 z ; - - - -If you can read that, you can read any forth program with a bit of -practice. Of course, giving words short names like this can get pretty -confusing. The entire program from scratch, but with more -descriptive names: - - - -: increment 1 + ; -: add-2 increment increment ; -: add-4 add-2 add-2 ; -: vijver drop 1 add-4 ; - -

- -

-Implicit vs. Explicit - - -The advantage of using a stack is that intermediates are not named. -They are implicit, so -they don't get in the way when you are trying out things. What i mean with -intermediates is the values that are passed from one function to another. -This is also a disadvantage, because it is less redundant: there's more -information in your head. - - - -I was thinking about the big difference between forth and languages -that do have named parameters and local variabled, because that's what -it's all about in the end. If you look at this from a data-flow pov, -symbols allow 'random access linking', which effectively introduces a -complex data flow. In forth, the data flow is a very simple fifo one, -which enables very short simple functions, but which makes writing longer -functions harder. For lanugages that name local symbols and function -arguments, it is the other way around: it is harder to write small -functions, because the overhead of the names themselves becomes larger, -but it's easier to write longer functions, because the symbols enable you -to randomly connect to the caller's namespace. The same goes for data -flow languages like Pure Data. It's easy to create a mess, and factoring -requires a lot if [inlet] and [outlet] objects, which effectively increase -the object count. Which one is best is really a matter of taste. Forth is -less verbose and serves very well in its role as command language. Forth's -syntax is optimized for lots of small functions. It allows to write -extremely modular code, because it force you to 'linearize' your code and -this makes it trivial to factor. - - - -

- - -

- -The Gory Details - - -The language is characterized by it's imperative nature. Everything is -a succession of commands. Because that's basicly all there is, it -serves good purpose to know how a forth works. - - - -Packet Forth's interpreter translates human readable code and data -(text messages) to an internal data representation (data atoms like -ints, floats, packets, lists, ...). If it concerns a symbol, the symbol is -looked up in the dictionary of commands. Then, depending on wheter we are -in interprete or compile mode, the command is executed or compiled. - - - -Forth is actually 2 interpreters. The inner interpreter executes the forth -code residing in the dictionary. Forth code consists of lists of execution -tokens. An execution token points to some machine code (which can come -from a c program) and some data (which can be forth code). Now, that's -a mounthful, and everything will become clear later. -The outer interpreter translates symbolic, indirect commands to direct -commands (xt) by looking them up in the dictionary. In packet forth, the -parser is another element that translates a stream of text input to a -stream of forth atoms. - - - -Compiled code is referenced by an execution token (xt). Compiling code is -done by converting a sequence of symbols and data items to a sequence of -xts and data items. This is a very important concept. -The most important thing to remember is that a command or instruction for -the inner interpreter is an execution token (xt). It represents a direct -link to an action the forth can perform. -Ignoring the difference between outer interpreter and the parser, the -important thing to remember is that there are 2 levels 'code'. One is -human readable (as a string or a list of atoms), the other one is -compiled. When i say 'x is interpreted' later, it means the outer -interpreter. - - - - -