#lang slideshow

(define (img file)
  (let* ((p (bitmap file))
         (s (/ 570 (pict-height p))))
    (display (list p s))
    (scale p s)))

;; todo:
;; - demo formant
;; - demo modulation


;; the joy of starting from scratch
;; - a lot of fun to rethink basic assumptions
;; - in this case: get rid of C
;; - originally, 

;; Learn how to make a synthesizer
;; Two parts:
;; - quirky: 8-bit PIC18F and Forth
;; - proper: EDSL and C code generation

;; backdrop: got into electronics because i wanted to make
;; synthesizers. spent a couple of years writing DSP algorithms
;; directly in C,C++ and discovered scripting languages.  ever since
;; I've been looking for ways to build DSP applications using more
;; higher level languages.

;; - this is really about Forth
;; - how minimalism is attractive to me
;; - how it got me into compilers
;; - and how building a synth made it possible to turn staapl into a social thing
;; - (and how focusing on the compiler put it back into obscurity)
;; - now i have this thing, so what should i do with it?
;; - i've continued developing the synth (midi, analog)
;; - i'm still looking for what to do with the forth: there's something about writing minimal code
;; - i'd like to extract this into a simpler thing some time

;; - but what it really has started, is looking into programming language implementation, and code generation for embedded systems
;; - one spinioff is the use of typed embedded languages (SISO, DSPM)


;; elegance - Forth is worth trying out
;; sharing passion for simplicity
;; what if we do it all from scratch?
;; project takes on a life of its own
;; it got a little out of hand
;; there is simplicity hidden in complexity
;; some infrastructure is needed


(slide
 #:title "Synths and Metaprogramming"
 (t "tom@zwizwa.be")
 (t "http://zwizwa.be"))

(slide
 #:title "Synths and Metaprogramming"
 (t "Music, sound & image, electronics ...")
 (t "... but mostly code")
 (t "and the joy of starting from scratch")
 )

(slide
 #:title "How to build a synthesizer"
 (item "demo 1: Minimalist synth in PIC18 Forth")
 (item "demo 2: Emacs Modular Synth: Scheme DSL -> C"))

(slide
 #:title "Study 1987-2000"
 (item "1987-1989 MSX BASIC, Z80 assembler")
 (item "1989-1991 MS-DOS, Pascal, DEBUG.COM, 8086 asm & arch")
 (item "1991-1993 Borland Turbo C/C++ & Debugger, 80386 asm & arch, 2D/3D graphics, sound")
 (item "1993-1997 Electronic music, hardware synthesizers, analog & digital electronics")
 (item "1997-1998 Linux: RedHat, Suse, Debian, system administration")
 (item "1998-2000 Synth engine, C++, Matlab, Perl")
 )

(slide
 #:title "Study 2000-2010"
 (item "2000-2002 Pure Data, computer music & signal processing, GCC/GDB, Linux development, wrote creb")
 (item "2002-2004 Python, emacs, video processing, wrote PDP")
 (item "2004-2006 Forth, Scheme, language & VM design, wrote PacketForth")
 (item "2006-2008 OpenWRT and embedded Linux, Microchip PIC asm & arch, ColorForth, compiler design, wrote Staapl, sweb")
 (item "2008-2010 Scheme (Racket) and programming language theory, wrote libprim")
 )

(slide
 #:title "Study 2010-2017"
 (item "2010-2012 Haskell, type theory, and embedded language design, wrote DSPM")
 (item "2012-2013 back to DSP theory, wrote RAI: a language for DSP algorithm design")
 (item "2013-2014 Python Qt GUI and MySQL, SQLite database, wrote Pyla, Staapl development")
 (item "2014-2017 Erlang, MyHDL (+ VHDL/Verilog), Analog Electronics, RAI & Staapl tweaks, wrote SISO")
 )

(slide
 #:title "Synths and Metaprogramming"
 (item "DSP code in C is horrible - find something better")
 (item "Pure Data: C modules, graphical composition")
 (item "BYO scripting language: Packet Forth")
 (item "From Forth to compilation")
 (item "Scheme to the rescue")
 (item "Haskell, typed EDSL"))

(slide
 #:title "Staapl: Forth on Microchip PIC18"
 (item "PIC18 is close to a hardware stack machine")
 (item "Use Racket Scheme's macro system as a 'programmable compiler'")
 (item "8-bit Forth dialect taylored to hardware")
 (item "Peephole optimizing compiler for PIC18")
 (item "Interaction system for incremental compilation")
 )
 
(slide
 #:title "Minimal Synth in PIC18 Forth"
 (item "No DSP. Just events.")
 (item "Timer interrupts, 3 x 16bit HW timer")
 (item "Toggle output in ISR based on current 'mixer' and toggle/noise")
 (item "Mixers: square, XMOD2, XMOD3, formant")
 (item "Timbre = update parameters at +- 200Hz")
 (item "Notes = update timbre via MIDI or internal sequencer")
 )




(slide
 #:title "PIC18 is almost a Stack Machine"
 (item "A hardware return stack")
 (item "An accumulator / top-of-stack register")
 (item "3 Pointer register with auto increment/decrement"))

;(slide
; #:title "uC Metaprogramming"
; (t "stop being productive")
; (t "go for interesting and fun")
; (t "build everything from scratch")
; (t "build multiple iterations")
; (t "give it some time"))

(slide
 #:title "PIC Forth evolution"
 (item "2004/3 first Forth compiler code in C")
 (item "2006/2 rewrite in Guile Scheme")
 (item "2007/1 rewrite in Racket (then PLT Scheme)")
 (item "enabled by grants 2005-2007")
 (item "later, ongoing refactoring during consulting downtime")
 (item "synth as test suite")

 )

(slide
 #:title "Breadboard Workshops, 2005-2007"
 ;; (scale (bitmap "100_0483.jpg") 0.75)
 (img "100_0483.jpg")
 (t "Huddersfield residency with GOTO10 2005-2007"))

(slide
 #:title "CATkit - 2007"
 ;; (scale (bitmap "img_1142.jpg") 0.5)
 (img "img_1142.jpg")
 (t "Later, workshops used board")
 )

(slide
 #:title "ForthTV - 2007"
 ;; (scale (bitmap "516734190_5fc608e13e_o.jpg") 0.4)
 (img "516734190_5fc608e13e_o.jpg")
 (t "META workshop Amsterdam, PIC tv out")
 )

(slide
 #:title "KRIkit / Meshy - 2007"
 ;; (scale (bitmap "img_1168.jpg") 0.5)
 (img "img_1168.jpg")
 (t "Amsterdam residency with GOTO10 2007")
 )

(slide
 #:title "After residencies, 2007-2012"
 (item "Work: back to software consulting, embedded systems")
 (item "Play: back to books: languages and compilers + EDSL experiments"))

(slide
 #:title "PIC18 Forth USB driver 2012"
 ;; (scale (bitmap "IMG_20170529_125659.jpg") 0.25)
 (img "IMG_20170529_125659.jpg")
 (t "Bought tools, fixed some nasty bugs")
 )

(slide
 #:title "RAI, 2012-2014"
 (item "take Racket Scheme DSL seriously")
 (item "build a language to build a digital modular synthesizer")
 (item "focus on the practical -- typed EDSL (Haskell) was too difficult")
 (item "no dependencies, no runtime")
 (item "interoperate: VST, Pure Data, linux stand-alone, embedded CM4")
 )


(slide
 #:title "MIDI, EIDAC, VCO, VCF - 2014"
 ;; (scale (bitmap "IMG_20170529_104601.jpg") 0.25)
 (img "IMG_20170529_104601.jpg")
 (t "After some analog work, picked up synth again")
 )

(slide
 #:title "EIDAC, Sawtooth VCO - 2014"
 ;; (scale (bitmap "pic_saw-v2-201405.jpg") 0.4)
 (img "pic_saw-v2-201405.jpg")
 (t "Exponential sawtooth: works, but no calib")
 )

(slide
 #:title "Ladder VCF - 2014"
 ;; (scale (bitmap "ladder-201406.jpg") 0.4)
 (img "ladder-201406.jpg")
 (t "Diode ladder: doesn't work yet")
 )

(slide
 #:title "Haskell 2014 - 2017"
 (item "EDSL based on structure of RAI")
 (item "Signal generators as Applicative Functors")
 (item "Base language: Tagless Final typed DSL -- Oleg Kiselyov, Jacques Carette, Chung-chieh Shan")
 ;; (scale (bitmap "ladder-201406.jpg") 0.4)
 )


#|

So.. spent an entire day tracing back history, and got completely
confused and distracted.  How to make this work?  Make some sound.
Either use USB, or MIDI.





How is this relevant?

Art (music, media-art) and technology (DSP, scripting languages,
metaprogramming, electronics).


Demo:
- midi keyboard
- pic18 board, hooked up to mixer & speakers
- some sequence, composing?

Topics:
- the language (macros and instantiation)

- the base language for pic18
- the pic18 compiler (two-pass peephole optimizer)
- forth and the "compression force" (factoring, protocol col oriented programming)

TODO:
- synth: get a board working with MIDI
- synth: get the old tutorial working again -> extract basic ideas! (generators)
- language: get the old tutorial working again
- compier: clean up flunk presentation for compiler part

General guidelines:
- keep the despair out of it -- the compiler is ad-hoc, period
- don't linger on the "is this practical" point



Compared to the C-world, Staapl can seem impractical.  What attracts
me though is two important point I still seek to articulate better:

- compiler is a little quirky but quite simple.  main interest in
  driving this foward is to make the pattern matcher be typed.
  especially: how to pick the pseudo instructions? the design stems
  from a time at which i did not see the benefit of algebraic
  datatypes wrt. future growth.

- a forth program has a high degree of "inlinability" and
  "refactorability".  much more so than a regular applicative program.
  apart from all this, could the "corset" be used as a more general
  funnel to have programmers write smaller code?

- there is something about eliminating random access that is not fully
  explored in industry.  in hardware, it is a serious bottleneck.
  there is an echo of a solution to this problem in the idea of forth.

Combination of both: toward a typed stack machine language.




PIC18 as a proto-stack-machine

This is about the ideas of radical simplicity coming out of the Forth
world.  After an inititial infatuation phase, my goal has always been
to extract Forths essential oil and find a way to mix it with today's
de-facto ecosystem: C on ARM barebone or RTOS, and some Linux stack.

The Staapl system is built with generality in mind, but is focused on
the PIC18.  The PIC18 architecture is an evolution of the original PIC
architecture, taylored for C.  Still, C is quite inefficient on this
small chip.

The approach taken in Staapl is to define a simple Stack language on
top of the PIC18 architecture, resembling a "machine Forth".





Tilting at Windmills

In this particular story, the imaginary enemy is the C programming
language family tree.  Before diving into any of this I want to make
clear what is my current position.

I love C.  And I hate C.  And I maybe that is what a mature
relationship eventually should become.  Despite all of C's flaws, it
is still the de facto standard for deeply embedded software, and I do
not believe this is going to change any time soon, especially because
C and computer architecture co-evolved.  The processors we use today
are designed to efficiently run C code.

Regarding embedded software, my point is that you should use C only
when there is no other sane way to get the low-level control you need.
But otherwise you should use a high level language with garbage
collection and preferably a strong type system.  My current favorites
are Racket, Erlang, Haskell, Rust, Lua and to some degree Python.

However, there is a gap between assembly language and C that is filled
by another approach: Forth.  If you write a lot of low-level C and
assembly code and have not explored Forth, I think you should.  Just
to get a feel.  Forth is very different.

I dove into Forth around 2004.  Still, I cannot find a satisfactory
articulate explanation about what is so attractive about the idea of
Forth.  But I think it is possible to get an intuitive understand of
this by looking at the life work of Chuck Moore.  His main idea is
radical simplicity when it comes to hardware and software design.

One way of looking at it, is that writing code is "compression by
subdivision and reuse".  What a stack language does, is to allow you
to push this compression very far through the use of implicit local
context -- the data stack.

Essentially, at the bottom line, you are not moving data around
unnecessarily.  What you learn to do when writing Forth code, is to
design the coupling between words in such a way that the number of
stack shuffling operations becomes minimized.  This I believe is the
main reason why Forth can lead to such compact code.

There is obviously a cost to this: it takes more time to write Forth
code.  But the result is almost always elegant.  And this is the
surprising part, maybe.  If you do get the compression right, the code
will look beautiful and simple.  And it will be small.

What I find sad about Chuck's approach makes sense only in Chuck's
world.  It works if you can build an entire product this way, without
the need for code interfaces to the outside world.  It works for
systems that are isolated at communication protocol level.

What I do think he is right about, is that Forth is about hardware.
Forth is a machine language for a stack CPU, and it can lead to
processors that are very simple.

This is where the Staapl project originated: to build a machine
language for the PIC18 that is close to the native machine language,
and see where that goes.






The PIC18 as a Stack Machine

The architecture of the PIC18 is easily mapped to the following
architecture:

- A hardware code stack

- An accumulator

- 3 Pointer register with auto increment/decrement

These map almost directly to:

- Return stack and data stack
- Two pointer registers to use for transfers




A Macro Forth


Compression




The sheep: basic principles

- Use hardware to create something unique: pic18 has 3 x 16 bit timers
  which can create fairly high resolution sound pitches.

- No DSP, just events at 3 different time scales:

  - There is not much room for traditional 8 or 16 bit DAC as this
    would require processing at a fairly high rate (say 48kHz).
    Instead, sound is generated purely by making transitions based on
    interrupt events from the timers.

    From a signal analysis p.o.v., the effective sampling rate of the
    output signal is 2MHz (in the original 8MHz osc design), which
    means there will not be much aliasing present.

  - At around 200Hz, the synth's configuration can be updated to
    perform timbre evolution.  (generator loops).

  - At around 10Hz, rhythmic events can switch the configuration of
    the timbre units.  These can be generated on-chip, or come from
    MIDI.



How to move this forward?

- Complete the MIDI implementation

- Add more sound generators, and/or write an interpreter for those
  that can be updated over midi.

- Add analog components

|#