Packet Forth

+ +Packet Forth + + +

+Introduction + +You know, there's something strange going on with forth. In the +last 2 years, my attitude towards forth has changed from being intrigued +by that language with the strange succession of words, through the process +of trying to write several forths myself, and inspecting a miracle of engineering: +ColorForth. Chuck Moore is a genious. + + All in all forth is a hack. It is very much determined by design +decisions mostly aimed at keeping it simple. It is very pragmatic. +But, it is a hack that solves one of those hard problems: finding a good +compression scheme for human and machine readable concepts. Aka code and +data. Packet Forth takes forth somewhat out of it's original "can't be too simple" context +by turning it into a more highlevel, typed language. Forth is usually +situated at the bit and machine instruction level. Somehow this makes it feel +more contemporary. More in tune with other tools in the basket, and i'm looking +at perl, python and guile. + + + In case this is not convincing as a teaser to read more, let me give you a pragmatic example. +When i work with pf, i have usually a terminal running a pf session with some +things loaded. I want do try something. Mostly this is just a couple of commands. +But i will need them again later, so i make a new composite word. Then i try again, +etc. When i got something workin, i paste it to an editor. Or, i can work the other +way around, and send a line of commands to the interpreter when i'm working on +a forth program in the editor (emacs). This just works. The fact that it's forth, +doesn't matter that much, only that it leads to the smallest red tape overhead. Most +things are just one word. One symbol per abstract action. + + +Packets and Forth are the golden twins of the 21st certury ! + + +jaja! + +

+ +

+Commands and Command Composition + + +Forth is commands. Each command does something with the current things. +The current things are kept on a stack. The stack can be printed using +the command .S, which starts with a +. like most words that send +something to the current output stream. The command consisting of +a single . prints the top atom +on the stack and discards it. Try this at the console: + + + +> 1 +> .S +<1> 1 +> + + + +The first command will load the integer number 1 +Now, let's get some more fuel. + + + +> 2 3 +> .S +<3> 1 2 3 +> + + + + +We now have 3 items in our bag. The hottest one is 3. It is on the top of +the stack. If a command is executed that consumes just one item, it will +be the 3. But we're not going to do that. We're going to try something +more exiting, which is to execute the command +: + + + +> + +> .S +<2> 1 5 +> + + + +It seems the 2 and 3 magically disappeared from the stack and were +replaced by a 5! Luckily, the 1 is still there, not touched by the evil +fivifier. Actually, + is not really a fivifier. +And besides, Packet Forth +does not come with fivifiers (tsss). So why don't we just extend it? + + + +Before we start the chapter on how to add a fivifier to Packet Forth, +we first make sure there is an item worth of fivification in the ready +position, meaning the top of the data stack. We do this using the command +'swap'. + + + +> swap +> .S +<2> 5 1 +> + + + +Now, the most important thing about forth in general is the ability to +extend the language with fivifiers, but also any other function or +operation you can think of that is a combination of known operations. A +new operation can be defined using the combo : + and ;. The first word +following the semicolon will be the name of the composite command, all the +other symbols until a semicolon will be part of the command definition. + + + +Let's define a fivifier. Admit it. The thing you've been waiting for. +Just type this at the '>' prompt and brace for horror. + + + +: vijver drop 1 1 + 1 + 1 + 1 + ; + + + +Now, this really is an evil fivifier. In adition to being pretty +rude as a fivifier, by discarding the first item instead of treating it +with proper respect, it also goes against the holy doctrine of the Church of +Chuck, which says thou shall factor! +Anyway. Lets check if we're really dealing with a fivifier. + + + + +> vijver +> .S +<2> 5 5 + + + +The top 1 is gone, and a five is put there instead. The fivifier seems +to be operational. +Now, let's clean this fivifier up a little. The recurrent pattern that +catches the eye above is '1 +'. This is actually way to verbose for an +operation with such small effect. Let's define this operation, 'to add one +to the item on the top of the stack', to be hitheron know as 'x': + + + +: x 1 + ; + + + +This means we can redefine the fivifier using the word 'x' we just +created. You may call me mad, but this is what is called compression of +ideomatic sequences. + + + +: vijver drop 1 x x x x ; + + + +Ok, enough sillyness. This however does illustrate the key concepts. +Forth is about words and stacks and this enables very easy factoring. +Factoring (chopping the large chunks into smaller bits) is essential to +writing understandable code in any programming language. Forth makes it +easy and almost a requirement, so why not: + + + +: y x x ; +: z y y ; +: vijver drop 1 z ; + + + +If you can read that, you can read any forth program with a bit of +practice. Of course, giving words short names like this can get pretty +confusing. The entire program from scratch, but with more +descriptive names: + + + +: increment 1 + ; +: add-2 increment increment ; +: add-4 add-2 add-2 ; +: vijver drop 1 add-4 ; + +

+ +

+Implicit vs. Explicit + + +The advantage of using a stack is that intermediates are not named. +They are implicit, so +they don't get in the way when you are trying out things. What i mean with +intermediates is the values that are passed from one function to another. +This is also a disadvantage, because it is less redundant: there's more +information in your head. + + + +I was thinking about the big difference between forth and languages +that do have named parameters and local variabled, because that's what +it's all about in the end. If you look at this from a data-flow pov, +symbols allow 'random access linking', which effectively introduces a +complex data flow. In forth, the data flow is a very simple fifo one, +which enables very short simple functions, but which makes writing longer +functions harder. For lanugages that name local symbols and function +arguments, it is the other way around: it is harder to write small +functions, because the overhead of the names themselves becomes larger, +but it's easier to write longer functions, because the symbols enable you +to randomly connect to the caller's namespace. The same goes for data +flow languages like Pure Data. It's easy to create a mess, and factoring +requires a lot if [inlet] and [outlet] objects, which effectively increase +the object count. Which one is best is really a matter of taste. Forth is +less verbose and serves very well in its role as command language. Forth's +syntax is optimized for lots of small functions. It allows to write +extremely modular code, because it force you to 'linearize' your code and +this makes it trivial to factor. + + + +

+ + +

+ +The Gory Details + + +The language is characterized by it's imperative nature. Everything is +a succession of commands. Because that's basicly all there is, it +serves good purpose to know how a forth works. + + + +Packet Forth's interpreter translates human readable code and data +(text messages) to an internal data representation (data atoms like +ints, floats, packets, lists, ...). If it concerns a symbol, the symbol is +looked up in the dictionary of commands. Then, depending on wheter we are +in interprete or compile mode, the command is executed or compiled. + + + +Forth is actually 2 interpreters. The inner interpreter executes the forth +code residing in the dictionary. Forth code consists of lists of execution +tokens. An execution token points to some machine code (which can come +from a c program) and some data (which can be forth code). Now, that's +a mounthful, and everything will become clear later. +The outer interpreter translates symbolic, indirect commands to direct +commands (xt) by looking them up in the dictionary. In packet forth, the +parser is another element that translates a stream of text input to a +stream of forth atoms. + + + +Compiled code is referenced by an execution token (xt). Compiling code is +done by converting a sequence of symbols and data items to a sequence of +xts and data items. This is a very important concept. +The most important thing to remember is that a command or instruction for +the inner interpreter is an execution token (xt). It represents a direct +link to an action the forth can perform. +Ignoring the difference between outer interpreter and the parser, the +important thing to remember is that there are 2 levels 'code'. One is +human readable (as a string or a list of atoms), the other one is +compiled. When i say 'x is interpreted' later, it means the outer +interpreter. + + + + +