RAI - Racket Abstract Interpretation Development log for the RAI project. The goal of this project is to explore code generation and validation using techniques based on Abstract Interpretation, implemented in Racket. For more information, visit http://zwizwa.be/rai This log was forked from: http://zwizwa.be/-/meta which contains more information about the Haskell leg of the project. Entry: Move to git / github / racket package Date: Sun Jun 2 19:13:02 EDT 2013 Starting on a fresh repo. http://zwizwa.be/rai raco pkg install github://github.com/zwizwa/rai/master parent dev log: http://zwizwa.be/-/meta parent darcs repo: http://zwizwa.be/darcs/meta Entry: TODO Date: Sun Jun 2 19:14:08 EDT 2013 Maybe good to make a list of the small things that need to be done. - FIXES: - ai-stream.rkt (write in terms of ai-array.rkt output?) - delay line allocation: phase 1 -> phase 2. - make time block explicit (subsampling) and fixed size - proper feedback operator - compute transfer function as ratfunc. - FEATURES: - constant arrays. - presets - state initialization - generate GUI from description - loop fusion? (check out Feldspar) Entry: Add a `where' form? Date: Sun Jun 2 19:27:46 EDT 2013 This would allow specification of inputs/outputs in a more straightforward way, such that output arity is visible at the syntax level. E.g. (proc (in ...) (out ...) (where ((a ...) (b ...) ...))) Entry: Run generated C code from Scheme Date: Mon Jun 3 19:48:43 EDT 2013 3 ways to run code in Scheme: - provide separate interpretation, like ai-stream.rkt (reference implementation) - compile the pre-C imperative intermediate language to Scheme - load a compiled C module Entry: John Clements Course Date: Mon Jun 10 07:57:55 EDT 2013 http://www.brinckerhoff.org/JBCsite/index.html http://www.brinckerhoff.org/clements/ http://www.brinckerhoff.org/clements/csc123-fa12/index.html Entry: Next Date: Mon Jun 10 08:08:37 EDT 2013 Low on time and energy. What needs to happen next? - LLVM C/C++ bindings - Load C code in Scheme - Scheme semantics based on ai-array form output Entry: KVR post Date: Tue Jun 11 08:43:08 EDT 2013 http://www.kvraudio.com/forum/viewtopic.php?p=5385673#5385673 Hi Frank, What I find most interesting in language design is the basic principles or "axioms", being 1) primitive operations, and 2) composition/abstraction mechanisms to build new operations IIUC, your patch language is functional, except for the primitives, which have state. Some questions about the composition mechanism: - How do you differentiate between "class" and "instance" for the primitives? I.e. there is Osc1 and Osc2. Are these fixed instances of a shared oscillator class? - Is there an abstraction mechanism? I.e. is there a way to take a couple of modules and create a new black box to be used in another patch? Cheers Tom Entry: Pd-dev, music-dsp, racket list post Date: Wed Jun 12 11:28:54 EDT 2013 Hi List, I've been cleaning it up a bit. Still quite raw in the details, but it basically works. added synth-lib.rkt basic synth kit and synth.rkt example synthesizer: - envelopes - parameter control dezippers - parameter control scales (exponential, "squashed" exponential, ...) - SVF filter - supersaw osc - anti-aliased saw osc - saturation functions - FDN reverb tail How I use it at this time, Linux + Pd: make sp_host.pd_linux # build binary Pd wrapper pd sp_test.pd & # run Pd patch built around sp_host make livecode # continuously compile synth.rkt -> synth.sp binary module then edit synth.rkt or any of its dependencies and the binary code running in Pd will be updated, leaving state intact if the type didn't change. some docs at http://zwizwa.be/rai For headless Windows VST plugin: "make synth.dll" This needs a Linux->Windows cross compiler; follow the error messages. Cheers Tom Entry: LLVM? Date: Wed Jun 12 13:28:34 EDT 2013 Maybe an LLVM target would be a nice addition. Running code in-image without external C compiler dependencies is attractive, though is it worth the trouble? Entry: Industry integration: validated, bottom-up design Date: Wed Jun 12 13:46:55 EDT 2013 I'd like to be able to integrate what I've learned up to now (the knowledge that lead to RAI) and steer it more into a direction that is usable in an industrial setting. At this point it is usable for designing music DSP software on PCs or mobile devices, but not yet so usable for machine mapping as would be encountered in a typical embedded development settings, i.e. the projects that pay the bills. One important element is target dependency. It might make more sense to construct a tool that helps manual machine mapping of an existing algorithm, than to attempt to work top-down as I'm doing for the synth work. At least for code running on a quirky DSP chip there is going to be a significant amount of squeezing involved to map to a particular target. It doesn't seem that the trend of quirky hardware is not going to change much in the near future, as power consumption and device cost constraints will keep much of the manual design approach alive. For the big guys, there are plenty of expensive tools available. Have a look at the MathWorks product range. More, bigger tools can be found in the ASIC industry. For the small guys doing one-off jobs, these tools just too expensive. What I've been wondering for 10 years is how far can one get with open tools to make the lone wolf embedded developer more productive? I want to stress this: none of the ideas here are new. What is new is to keep it simple, small and malleable. A large constraint in reality is the introduction of exotic tool chains. My conclusion is that people hate it: it's too big a risk to use yet another quirky code generator whenever a new person joins the team. A big constraint for designing tools is to make sure they can get out of the way, i.e. that you can switch them OFF and drop down to writing straight ASM or C code. In a nutshell: VALIDATION of a low-level implementation is more important than GENERATION from high to low level. In practice, in the RAI framework, this would boil down to finding a set of primitives, probably based on the machine's primitive, AND a set of machine constraints and resource management annotations. This is not a simple problem. To keep it simple, a most straightforward approach is to start with a reduced assembler language, and see how to put a functional semantics on top of that. So what about this for future direction: - Aiming at validation, use the abstract interpretation trick to approximate the semantics of a real device. - Pick a target to test this: dsPIC or ARM DSP / Cortex M4 Entry: What target to pick? Date: Wed Jun 12 13:57:31 EDT 2013 dsPIC: + combined with PIC18 support in Staapl, I could aim at 1 vendor (Microchip) / 1 tool (Racket) approach. + DIP available, good for grassroots electronics projects - typical quirky DSP arch with lots of constraints - no free optimizing C compiler for both PIC18 and dsPIC. ARM Cortex M4 [1] + industry standard, multiple vendors: TI Stellaris, STM32 F3,F4 + good open tooling support + plenty of boards - no DIP available [1] http://en.wikipedia.org/wiki/ARM_Cortex-M#Cortex-M4 Entry: Idiomatic C Date: Wed Jun 12 14:50:53 EDT 2013 It would be interesting to find the subset of C that can accomodate the basic stream language's semantics. Entry: Merging ideas: Staapl and RAI Date: Wed Jun 12 16:53:54 EDT 2013 Both have an element of bottom-up design, but for RAI, the bottom is a small, well-defined language, while for Staapl, bottom is the machine language. Making this work for Staapl was one of the goals, but has not been reached yet. I wonder where the middle ground is here: design a machine abstraction close enough to the real machine to allow for hand-tuned allocation, and high level enough to have a decent semantics. Another thing is to properly gauge the importance of resource efficiency vs. safety/correctness. If resource efficiency is not a main driver, code gen might be a better option. Entry: Project management: future of RAI Date: Thu Jun 13 09:53:11 EDT 2013 After about 5 months the goals are reached. RAI is good enough for its initial intended purpose. The rest seems to be "long tail" maintenance, refinement, cleanup and gaining more insight through concrete use cases. I find myself in a bit of an impasse at this point, as the strongly mobilizing curiosity is no longer there. Maybe it is important at this point to realize that, and keep what there is now as a stable core. I.e. build extensions as external projects. Entry: A simplified simulator Date: Thu Jun 13 14:36:57 EDT 2013 Problem: find a good approximation to a chip's instruction set that can be embedded in a functional language. This attempts to connect two worlds: - Easy translation to "real" device semantics - Easy translation from pure functional constructs The problem here is resource management. For the particular problem of simplifying the dsPIC, this would be to abstract X/Y data memory, accumulator and addressing modes. Entry: Composing state machines Date: Fri Jun 14 10:06:26 EDT 2013 Note these are synchronous state machines, which are a bit simpler than event-based ones. The basic model I use for the audio DSP work is the state space model (SSM) or state space representation, which has nice properties if it is linear: http://en.wikipedia.org/wiki/State_space_representation Goal: represent a stream operator in Haskell in an abstract way, but allow for C code generation from the description. A stream operator consists of an initial state value and update function with types: s (s,i) -> (s,o) Ignoring the explicit state, the straightforward way to represent a stream operator as a pure function is something like this: data S a = S a (S a) S i -> S o However, the state info is lost here. So the SSM rep can be mapped to the stream rep, but not vice versa. When composing two SSMs (ignoring the initial state) you get something like: ( (s1,a) -> (s1,b) ) -> ( (s2,b) -> (s2,c) ) -> ((s1,s2),a) -> ((s1,s2), c) This is troublesome, because the state "grows". The whole thing is here: http://zwizwa.be/darcs/meta/dspm/SSM.hs The composition would fit neatly in the Arrow abstraction except for that dangling state vector. It is possible to use existential types to hide the states. The trouble with that is they become quite inaccessible. In the end, after composition, I really want a type (s,i) -> (s,o) where s is a large composition of per-SSM state vectors, because that information is used to generate C code. I also want this to fit in the Arrow abstraction, so it integrates better into Haskell. For code generation, the SSM idea is combined with the "final tagless" representation trick to perform abstract interpretation. See: http://zwizwa.be/darcs/meta/dspm/Control.hs http://zwizwa.be/darcs/meta/dspm/Data.hs http://zwizwa.be/darcs/meta/dspm/Struct.hs All this +- works. To work out how exactly to use the existential types, I did some type directed programming to just see where the types lead me, but in the end I got confused. It all seems a bit clumsy. I took that idea and went back to Scheme (Racket) to do what I wanted to do in the first place; to build a tool that I can use effectively to write audio DSP code (*). This is the result up to now: http://zwizwa.be/rai/ Doing it in Scheme allowed me to "emulate" types and type classes. An interesting exercise in itself, but still, in Haskell it would be a lot simpler. Now that I know that the basic idea works, I'd like find a good way to port it back to Haskell, probably on top of Feldspar. http://hackage.haskell.org/package/feldspar-language Entry: Fluxus / LLVM / Pd Date: Mon Jun 17 10:53:31 EDT 2013 So, what's next? A big fork in the road: - Target "real" DSPs, i.e. TI C2000, MicroChip dsPIC, ... - Integrate into host tools like Pd, Fluxus, ... Former seems a bit far off at this point. It would be great to have a real project drive this. Hybrid analog synth based on dsPIC? I ordered a C2000 eval board, which might lead to an interesting avenue. For the latter, it would still be possible to use an external .c to .so compilation step, but it is really awkward. It might be a lot better to bite the bullet and go for a JITtable LLVM approach. The question is where to make the Scheme <-> C bridge for the LLVM bindings. As I understand, C++ isn't so easy from Racket FFI[1]. [1] https://groups.google.com/forum/?fromgroups#!topic/racket-users/GFrGBbczTc0 Entry: Fluxus install Date: Mon Jun 17 11:11:12 EDT 2013 Probably best to just follow instructions and install to /usr/local * Racket from git git clone git://git.racket-lang.org/plt cd plt ./configure --enable-shared --prefix=/usr/local make sudo make install * Fluxus from git git clone git://git.savannah.nongnu.org/fluxus.git cd fluxus scons NOTE: I could not get this to work without symlinking bin & lib & include from the git-racket tree into /usr/local Entry: Live coding Date: Mon Jun 17 16:49:40 EDT 2013 Back to the life coding idea. I want to stick to the Scheme approach: language works better than graphical programming, but I also want Bret Victor's number boxes[1], essentially Pd's number boxes in code. I wonder if this is possible in Fluxus. [1] http://worrydream.com/#!/InventingOnPrinciple Entry: Fluxus question Date: Mon Jun 17 16:56:07 EDT 2013 fluxus@lists.pawfal.org Hi List, I recently saw Brent Victor's "Inventing On Principle" http://worrydream.com/#!/InventingOnPrinciple How hard would it be to make those "number sliders" work in Fluxus? That is, if it's not already possible.. Basically, hover over a number in the editor and allow the mouse to change its range. Cheers Tom Entry: Cortex-M4F Date: Sat Aug 24 15:40:51 EDT 2013 Got the Tiva Launchpad[1] with a TM4C123GH6PMI[2]. Goal: write DSP library code for the Cortex-M4 DSP and/or FP. It would be nice to be able to attach it to some audio codec, though for now it seems simplest to just use a USB audio interface to patch it into a test system. TI Tivaware can be used for dev dependency, since I won't need to redistribute anything. People that need to reproduce the dev env can just download SW-TM4C-1.1.exe themselves. This probably also means I can use Roadmap: - toolset setup - get bare-bones DSP code running in RAM / Flash - USB audio interface for testing (using tivaware) - Some HW codec? Old soundcard patch? Sparkfun[3]? PWM? software S/D? [1] http://www.ti.com/tool/ek-tm4c123gxl [2] http://www.ti.com/product/tm4c123gh6pm [3] https://www.sparkfun.com/products/9365 Entry: MinGW Date: Sun Aug 25 00:19:11 EDT 2013 So it builds on cygwin using cygwin-hosted MinGW, and stand-alone MinGW. However, the build scripts use shell commands. Entry: Live coding Date: Mon Sep 2 12:50:07 EDT 2013 Time to think about user interfaces. I'd like to implement a feature from Bret Victor's talk[1], where constants embedded in code can have a pop-up slider to change the value, with the result changing immediately. While this slider thing is neat, what is really important is a fast update between editor save and audible effect., i.e. a very fast compilation cycle for tick() methods. The C compiler probably needs to be removed from the loop, and possibly, all optimization could be switched off: code could run interpreted at first. The bottleneck is the speed at wich the partial evaluation runs. Let's try to benchmark this first. After that, an interpreted mechanism could be built for the C-like language. For (time (reload)) in this I get about 1.0 - 1.3 seconds: #lang racket/base (require "ai-array-c.rkt") (provide (all-defined-out)) (define (reload) (define ns (make-base-namespace)) (eval `(begin (require "ai-array-c.rkt") (require (file "synth.rkt")) (display (ai-array-c main #:nsi main-nsi))) ns)) About half the time is spent in the Scheme compilation phase. Re-running the AI interpretation step gives: (time (void (ai-array-c main #:nsi main-nsi))) ;; => 0.7 sec This seems like a lot. Trouble might be in the symbol lookup. Hashes could make things faster, maybe. But improving the Scheme compilation time isn't something I see happening, so it might nog be worth the bother. Adding the C compiler to this is fast if optimizations are turned off. To get better performance, the problem should be tackled somewhere else: compile to a more dynamic structure, and modify that stucture's parameters instead of recompiling everything. [1] http://www.youtube.com/watch?v=PUv66718DII#t=496 Entry: gdb & gcc/as Date: Sun Sep 15 17:12:50 EDT 2013 Got basics for gcc/as compilation and gdb interaction working. The idea is to make some kind of "console" where asm code can be tested interactively, either on real hardware or in the qemu emulator. I started getting into brainless hack mode trying to parse the gdb output. Looks like some emacs code uses a regexp massager to translate the format to json, and then uses a json parser[1]. I do wonder, for a syntax as simple as that, there has to be a nice racket tool that can do it. E.g. a packrat parser.. EDIT: It wasn't so hard to write a recursive descent parser using `peek-char' and `read-char'. [1] https://github.com/dov/dov-env/blob/master/emacs/gdb-mi.el Entry: qemu emulator Date: Sun Sep 15 17:13:42 EDT 2013 apt-get install qemu-sysem Got this error: tom@zoo:~$ qemu-system-arm -cpu cortex-m4 -nographic -monitor null -serial null -semihosting qemu-system-arm: symbol lookup error: qemu-system-arm: undefined symbol: libusb_get_port_numbers Probably due to this. Removing.. tom@zoo:/usr/local/lib$ ls -al *usb* lrwxrwxrwx 1 root root 76 Aug 11 2012 libmchpusb-1.0.so -> /opt/xc/microchip/mplabx/mplab_ide/mplablibs/modules/lib/libusb-1.0.so.0.0.0 lrwxrwxrwx 1 root root 17 May 21 2012 libusb-1.0.so.0 -> libmchpusb-1.0.so Looks like cortex-m4 is not supported. Works with cortex-m3 though. Next is memory regions: $ qemu-system-arm -cpu cortex-m3 -nographic -monitor null -serial null -semihosting -kernel console.axf -gdb tcp::1234 qemu: fatal: Trying to execute code outside RAM or ROM at 0x20000100 R00=20000100 R01=20000100 R02=ffffffff R03=00000000 R04=01000078 R05=00000000 R06=00000000 R07=00000000 R08=00000000 R09=00000000 R10=00000000 R11=00000000 R12=00000000 R13=200000e8 R14=0000037f R15=20000100 PSR=60000153 -ZC- A svc32 FPSCR: 00000000 Aborted My guess is that it just has 128MB RAM at 0. So to make this work, it needs an elf that starts at 0? So yes, that works, but apparently a simple .bin also works. [1] http://dr.barik.net/kb/Assembly [2] http://balau82.wordpress.com/2010/11/04/qemu-arm-semihosting/ [3] http://cgi.cs.indiana.edu/~geobrown/stm32/Main/Simulation Entry: gdb trickery Date: Sun Sep 15 18:54:50 EDT 2013 - parsing gdb data structures (OK) - uploading code to ram (OK, arbitrary bytes) - set temp breakpoint + run Entry: State machine transformer Date: Sat Sep 28 09:17:49 EDT 2013 The basic tasks: - environment capture for all yield points. - liveness analysis: does life-time cross yield points? How much of this can be done with abstract interpretation? Probably a lot. The interesting part is going to be how to do loops. Analysis phases: - identify all variables (e.g. flatten) - identify all control paths It doesn't fit in the standard rai template since it needs to capture variable bindings. Or does it? Let's just give it a try. It seems that this is really not so hard to do once the right representation has been found. What about avoiding control structures in a first iteration and just concentrating on the idea of continuation? The existing SSA transformer can be used to build nested environments. However, it's not really in an abstract reusable form, and it doesn't directly abstract `let' forms, only as side effects.. It might be best to solve the problem in isolation first: build a minimalistic scheme interpreter / compiler. A simple example program: with - named let - ordinary let* - primitive invocation - yield - tail call without: - non-tail calls (let next ((a 0)) (let* ((a (inc a))) (yield a) (next a))) It seems best to focus on let* instead of let, as it allows sequential operation, which maps better to low-level code. Named let is a bit annoying, but it's the only way to make loops without explicit loop statements and explicit lambdas. The language is not supposed to be higher-order. Interesting is that `yield' is the only non-tail position call in a body. ( let* rhs could have a proper non-tail call, which we ignore for now ) From this it might make sense to make `yield' into function semantics, i.e. always returning a value. This leads to the program: (define program '(let next ((a 0)) (let* ((a (inc a)) (_ (yield a))) (next a)))) Where we currently ignore yield's rv. Now, a compilation would involve walking the tree and splitting each yield into: - what came before -> treat yield as the inner expression / break off - what comes after -> reconstruct the environment = main problem So this is the same as "representing the continuation". The context needs to be restored. This includes control context. Waiting for that flash to happen... Where is the difficulty? Not thinking properly about the control structures. Reconstructing the environment is easy. The difficulty is in implementing the "variable update" behind the tail-recursive call. So what about doing this all in a dumb way: * rename each variable to a unique name and store them in a flat name space. optimize later for accessibility * implement next in terms of set! and goto This seems really simple and solves the whole problem. The rest is optimization. Entry: LV2: Turtle RDF Date: Sat Oct 5 15:23:22 EDT 2013 So it took a bit of digging to figure out what this is all about, but at least it seems quite straightforward to generate Turtle RDF from a canonical nested list/assoc-list Scheme data structure. Next: the C part. Entry: Kernel in Rust? Date: Sun Oct 6 10:01:37 EDT 2013 For the dynwav project, some core routines are not "just DSP". It might be good to use Rust for this. I wonder if it's possible to write deterministic code in Rust, meaning: for a particular thread, no I/O and no memory allocation. Anyway, an important part to get right first is "streams", which should probably be implemented as chunked buffers. Dynwav has a time->frequency tradeoff that reflects as a variable buffer size between FFT calls. Entry: Next? Date: Sun Dec 1 10:26:37 EST 2013 Probably try LV2 in a simple host, and make this part of the build process. Entry: LV2 torture test Date: Thu Dec 5 18:29:41 EST 2013 http://carlh.net/plugins/torture.php Entry: slideshow Date: Wed Dec 18 01:09:08 EST 2013 Document sput, making a slideshow with bode diagrams. [1] http://docs.racket-lang.org/slideshow/ Entry: Testing LV2 Date: Fri Dec 27 21:57:32 EST 2013 It might be interesting to have a LV2 host interface for testing authored plugins. sudo apt-get install libslv2-dev [2] sudo apt-get install swh-lv2 invada-studio-plugins-lv2 EDIT: Working with [2] I'm not sure that I like it very much. API is quite stateful, ad-hoc in the memory management and uses complete types and inline functions like LV2 does too. EDIT2: For later: find out if it is easier to use RDF directly. I'm new to this world of "simple complex" abstractions so I don't feel in a good position to decide. Maybe first get the job done, then look at elegance later? [1] http://lists.puredata.info/pipermail/pd-list/2013-07/103407.html [2] http://drobilla.net/software/slv2/ Entry: FFI for libslv2 Date: Sat Dec 28 15:35:30 EST 2013 Going forward, though slow.. Lot's of red tape. Next: pointers and arrays of floats. Entry: Float vectors Date: Sat Dec 28 16:34:02 EST 2013 It seems that variable-size arrays are problematic due to the need to define a C type for each array size. Is this so? Yes, but doesn't look problematic: all just functions / no macros. It seems simplest to work like this: - once size is known => (define f10 (make-array-type float 10)) - allocate memory using (malloc f10) Actually what is needed is ffi/vector and `make-f32vector` http://docs.racket-lang.org/foreign/homogeneous-vectors.html Entry: Next: hard problems Date: Thu Jan 9 22:38:37 EST 2014 - "extend from below" -> see also racket conference - how to have multiple low-level bases implement a shared functional base? e.g. approximation of x->exp(x) in different contexts fixed point Cortex M4F. - hierarchy of modules? this really smells like haskell type classes. can it be formalized? are racket modules sufficient? [1] - formalize subsampling - formalize delay lines (+- ok? it's already constructor based) - fix transfer function computation (see DSP book?): separate computatuion of polygon / poles (once)from evaluation of rational function (many times, e.g. for plot). - plug visualization into fluxus - make the "code dial" work. structured editor? this probably requires some kind of jit. or compile to some VM where constants are not inlined, or get "out-lined" whenever they are dialled. - solve the multi-layer mix -> distribute problem (only nonlinear!). this requires representation of "map" as in Feldspar: essentially: don't ever unpack lists. once unpacked, they gain individual identity and can no longer be represented as a separate array. the goal could be to undo gratuitous unpacking? [1] http://repository.readscheme.org/ftp/papers/sw2005/garcia.pdf Entry: Delay lines, commitation of implementation and structural transformation Date: Fri Jan 10 01:01:37 EST 2014 Delay lines are structural shortcuts. Meaning, they are intrinsically different. Is there a way to generalize this concept, i.e. straighten it out, removing the loops? Commutation of structural transformations. Commutation is such an interesting crutch. It is the embellishment of an implementation hack. If you can abstract the thing that needs a hack in a clean transformation, you can get away with limiting the iteractions of the hack. Concretely: suppose s represents and ordinary feedback system, and s' is a specification of a delay line system. The map D : s' -> s Represents the _conceptual_ part: a delay line is a generator of a large system with pass-through delays. Then an interpretation / implementation of such a system is optimized if there is a commutation between the implementation map I and the delay map D. I (D s') -> D' (I s) or I D ~ D' I Note that D does not have to be equal to D'. Maybe it's time to switch back to Haskell for a while to flesh this out... EDIT: Conclusion: - don't ever perform unnecessary unpacking: keep high level operations on the high level. - hack: re-abstract unnecessary unpacking? Entry: Removing the scheme reference implementation? Date: Sat Jan 11 15:55:09 CET 2014 Main reason is that it doesn't really work very well. Let's stick to the C implementation as reference loaded into racket through FFI, and keep the other interpretations for analysis. See test-librai.txt Entry: Evolve or rewrite? Date: Tue Jan 14 02:18:05 CET 2014 It is not clear if some of the bad abstractions need a rewrite or if they can evolve. An essential point to keep in mind is to not trash the code that is written in the language. If a parallel implementation should arise at some point, it should be able to use the DSP code written in the current system, be it with some translation step. Most of the design is complicated by hacks that are currently bolted on top: delay lines, control-rate subsampling. Entry: Write an executable tutorial Date: Tue Jan 14 02:26:40 CET 2014 Executable so it doesn't get out of date. Sput could be that. Entry: What is important? Date: Tue Jan 14 14:20:04 CET 2014 Theory: solve subsampling, delays through model transformation. This should be kept out of the Scheme core for now, and probably best be done in Haskell. Practice: - integrate loop transformations. - syntax cleanup - generated synth gui in PD (done) - use structs instead of float arrays Entry: Cleanups Date: Tue Jan 14 15:04:14 CET 2014 - Turn everything into call by name? - The autolifting is still quite weird. The reason for this is the way state threading is implemented. (it's ok, see next post) Entry: About implicit indexing Date: Tue Jan 14 16:04:05 CET 2014 The type annotation mechanism is a bit unorthodox. It supports a mechanism of implicit indexing. The reason for using implicit indexing is that it solves the behind-the-scenes state threading in a very elegant way. Causal stream operators may appear inside spatial loops. For each iteration, a separate "instance" of a causal stream operator needs to be allocated. Shifting from the scalar to the grid view makes it trivial to solve this problem. Abstracting such state management is a major feature of the language: it makes the language purely functional at the stream level, instead of object-oriented at the scalar level. I accidentally bumped into this approach as originally making every operation auto-lifted seemed like a good idea. It is not. Currently the loop contstruct is relatively orthodox, but internally the auto-lifting is still used to track grid indexing. It would be interesting to formalize this approach in Haskell. Entry: Loop transformations Date: Wed Jan 15 10:45:16 CET 2014 How to specify algorithms that require temporary array storage? Something is missing in the idea. Something can't be expessed.. It seems best to try to tackle this when the problem arises again in practice. Entry: Fixed point math Date: Wed Jan 15 11:50:19 CET 2014 Why is this interesting? Phasors and saturated add. Entry: pd: find all sliders in current canvas Date: Wed Jan 15 15:02:31 CET 2014 t_gobj *o; for (o=c->gl_list; o; o=o->g_next) { const char *name = class_getname(o->g_pd); if (!strcmp("hsl", name)) { printf("slider %p\n", o); } } Entry: Structs instead of float arrays Date: Thu Jan 16 15:07:09 CET 2014 At the .g.h level everything is fine. The problem is in the main_* wrappers. Essentially, .sp modules need type annotation. Entry: From floats arrays to generic types? Date: Fri Jan 17 16:45:29 CET 2014 It sounded good to just use floats, however in practice this will be very difficult to accomodate. Some library code is necessary to link up parameter connectivity. Not a quick fix! EDIT: This needs a design. More functionality needs to be handled in a separate library. Entry: Changes to rai.h Date: Sun Jan 19 11:34:07 CET 2014 essentially, all-float interface is simple, but too limiting. abstract it away in access methods. rai_info_control points rai_info_param (instead of index) rai_info_param as byte_offset member (pointing into flat container struct). Entry: SP structures Date: Sun Jan 19 20:56:15 CET 2014 It seems best to stick to the sp structures as default. Leave the static C interfaces for where it makes sense, i.e. on deeply embedded stuff. To make this work, make it possible to share the functionality in main_sp.c TODO: - librai.c -> operate on a (linked) rai_info - sp.c -> load binary .sp into rai_info (+- done: rai.c rai_sp.c) Entry: Next? Date: Mon Jan 20 12:16:58 CET 2014 - Initial state values. DONE! - fix libproc.rkt Entry: Status Date: Mon Jan 20 20:36:03 CET 2014 Apart from some minor details (and maybe I broke VST?) it basically works. The rest is incremental maintenance. So time to have some fun. Entry: GUI in racket Date: Mon Jan 20 20:37:03 CET 2014 Time for a reactive programming gui in racket. Entry: Quick compile & run Date: Mon Jan 20 21:43:27 CET 2014 If full plugin functionality (parameterizability) is not necessary, it might be straightforward to run a C proc from racket. Let's give it a try using the integrator, running in a single loop. EDIT: Done. Simplest approach: - compile stream processor to proc_class - instantiate proc_class inside -run function (abstract away state) Entry: A nice use case for rust? Date: Mon Jan 20 21:54:32 CET 2014 An audio event scheduler! Basically, a granular synth would be an interesting use case. It requires significant "systems" programming apart from pure number crunching. Let's do it on fractional time too, to get rid of the 1-sample aliasing artifacts. Entry: Racket gui experiments Date: Mon Jan 20 22:23:16 CET 2014 Time to create some guis for filter design. Entry: C with holes Date: Tue Jan 21 23:49:46 CET 2014 What about turning all constants into parameters for debug? (I.e. to do the "code dial" thingy..) Entry: Data structures. Date: Sat Jan 25 22:46:27 CET 2014 Possible optimization, exposing more representation freedom to the C compiler: If state is kept as local variables, it might be possible to avoid making the structs explicit in the .g.h This would work for a single-threaded application, or if the processor runs in a blocking thread. Entry: Next Date: Sun Jan 26 12:01:35 CET 2014 - fix libproc.rkt Found one x vs. x-pointer mixup. It seems best to add compile time size to proc.h api. EDIT: most problems go away after plugging libproc.rkt into the methods of proc.h instead of working with the datastructure directly. Probably only for gui control meta-data. Entry: DSP interfaces for cortex Date: Sun Jan 26 18:22:37 CET 2014 CMSIS - Cortex Microcontroller Software Interface Standard http://www.arm.com/products/processors/cortex-m/cortex-microcontroller-software-interface-standard.php CMSIS DSP Software Library http://www.keil.com/pack/doc/CMSIS/DSP/html/index.html Audio Weaver™- the Only Cross-Platform Audio Design Tool for ARM http://www.dspconcepts.com/products/audio-weaver Entry: Next: pattern music Date: Sun Jan 26 19:13:34 CET 2014 To further design synths and filters, it's necessary to connect them to a pattern sequencer with easy control. That problem is one of organization.. Entry: ai-stream.rkt replacement Date: Sun Jan 26 23:39:29 CET 2014 So it's now possible to implement ai-stream in terms of ai-proc.rkt Is that a good idea? It might actualy be more interesting to tap into the compilation step right before generation of C code. I've added an .il generation rule in rules.mk At this point it is too much duplication of effort. Best to keep the C compiler in the loop. Using rai without a working C compiler could be fixed later. Let's stick to building ai-stream.rkt on top of ai-proc.rkt, i.e. just adding a thin layer that does list <-> f32vector conversion, and possibly auto-lifting of constants, and unwinding of streams / sequences. Entry: Compiler temp dir Date: Mon Jan 27 11:51:39 CET 2014 FIXED Entry: compalg.rkt Date: Mon Jan 27 13:40:39 CET 2014 The file compalg.rkt is an example of a computer algebra worksheet. This is essentially a target language module: #lang s-exp "stream.rkt" with some extra functions from the library: (require "compalg-lib.rkt") This library overrides `ai-function-printer', printing all functions in symbolic form. Entry: In-image sequencer Date: Mon Jan 27 15:44:03 CET 2014 So let's make a sequencer for .sp modules. This would take Pd as a dependency out of the synth test loop. ( Later the C compiler could be removed if the .il is interpreted / compiled to Scheme ) Entry: Prototyping & Production platforms Date: Tue Jan 28 12:41:33 CET 2014 Time to fix the different target platforms. I'd like to support: PROTOTYPING: - Racket through rsound `signal-play' (see test-ai-network.rkt) Later this can have a VM interpretation in scheme, e.g. based on the intermediate .il form. For now straight C is good enough. This isn't particularly fast, but probably good enough. To speed things up, code can run at larger blocksize, while blocksize = 1 is a possibility for better interfacing with the rsound `network' DSL. - Pure Data: + "production" object with statically linked proc classes + dynamic loading of .sp class files + Pd gui creation + state-preserving reload - Jack stand-alone or in-image in Racket. Should be the same as the Pd module. For in-image it might be interesting to add a Racket gui. PRODUCTION: - VST & LV2. Straightforward. Probably best on top of libproc.c as this allows more code reuse. Static generation is not a problem. - Cortex M4F DSP. Static generation with local variables as state. Entry: Next: isolated problems Date: Tue Jan 28 12:46:47 CET 2014 - Add pointer indirection to array of objects. - Jack Entry: Pointer indirection Date: Tue Jan 28 19:41:06 CET 2014 Currently doesn't work: access by name is used. I'm wondering, since all applications at this point: - either use the proc.h dynamic abstraction - or need to be done in a special way It might be best to generate the proc.h data structures from Scheme. It's probably easier to write a custom Scheme generator than to get creative with the C preprocessor. EDIT: do this later - not essential. Entry: Next Date: Sun Feb 16 15:52:56 CET 2014 Axo? Cortex M4 DSP? Fixed point analysis? GUI? Entry: Axo reboot Date: Thu Sep 18 21:23:22 CEST 2014 1. Making a patch: simple without paramter input. 2. Uploading a patch on command line. Where is the protocol defined? Header: "Axo" followed by: P6 # param change W8 # write w20 # write file to SD T0 # change preset M3 # midi command B12 # Virtual control buttons C4 # create sdcard fle A4 # append data to sdcard file S0 # stop patch D0 # goto dfu mode d0 # read directory listing s0 # start patch p0 # ping c0 # close sdcard file EDIT: - axocl.py can start/stop, load, dump - minimal xpatch.cpp OK Entry: Axo floating point Date: Sat Sep 20 12:01:44 CEST 2014 The STM32F405[1] has floating point, so let's try that first. [1] http://www.st.com/web/en/catalog/mmc/FM141/SC1169/SS1577/LN1035 Entry: Axo Pex Date: Sat Sep 20 12:26:13 CEST 2014 Actually to make this useful, first implement PEx and midi. Entry: john clemens, racketcon 2014 - sound, why is it so imperative Date: Sun Sep 21 00:09:22 CEST 2014 BROKEN LINK from live event https://www.youtube.com/watch?v=ORQdUnUtgIA&feature=player_detailpage#t=8580 look for part two here later https://www.youtube.com/watch?v=oe2tK5Lxybc Entry: Macros in low level languages Date: Sun Sep 21 00:33:04 CEST 2014 http://www.brinckerhoff.org/blog/2014/06/25/macros-more-important-in-low-level-languages/ Entry: Pickup Date: Sun Sep 21 10:25:24 CEST 2014 Where did things get left last time? IIRC there are some hard problems to solve at the language level. - Feldspar-like loop transformations, i.e. fold,spread operations (global->feature->global). - Initial values - Typing - bottom-up extending / machine mapping Entry: Bottom up for M4F Date: Sun Sep 21 10:33:07 CEST 2014 Basically this means: represent the machine primitives and present a bridge between those and the (fixed) rai primitives. The interesting question here is machine mapping: replace the assembler programmer. So, it should be straightforward to support a M4F -> RAI prim mapping. Is it also possible to implement the inverse in a deterministic way. I.e. implement an instruction mapper that is guaranteed to recover native instructions? If this is possible, it might be a better approach. Required knowledge: Muchnick's chapter on peeplhole optimizers[1]. [1] http://www.amazon.com/Advanced-Compiler-Design-Implementation-Muchnick/dp/1558603204 Entry: Reverse macros Date: Sun Sep 21 10:41:07 CEST 2014 So, give an example. The M4F intrinsic QADD for saturated add of two integers. The problem with this instruction is that it is very inefficient to implement using rai primitives. Saturate on 31 bits might be more efficient since the overflow condition is not lost. So there is definitely a mismatch that makes it even impossible with the current set to implement any reverse matching. Entry: What is the real problem to solve? Date: Sun Sep 21 10:48:25 CEST 2014 Writing safe machine code. What does this boil down to? - Safe memory and type management - Safe semantics The former isn't hard to do in generated code, though the latter is harder since there is only machine semantics. Solution is to write an abstract interpretation of the machine code data processing primitives. How hard is it to generalize this? I can imagine working bottom-up, i.e. starting with the machine's computing primtives, and adding a semantics on top. Currently in rai, the primitives are too high level. Too polite :) The problem I'm trying to solve is to write an "assisted assembler". Entry: Assisted assembler a.k.a. "smart emulators". Date: Sun Sep 21 10:59:47 CEST 2014 So let's assume that machine mapping to a fixed set of primitives is too difficult. Let's start out with a concrete machine. I currently have 2 I care about: - dsPIC33 - ARM Cortex M4F Let's build a tool that allows building checked assemblers for new targets. A checked assembler allows construction of fully optimized assembly programs (manual resource allocation) while keeping semantic mapping in check. Essentially a semantic check is also ad-hoc specified, but some essential parts consist of transfer function computation, parameter sensitivity, noise levels, stability, ... What about making this into a live-coding instrument as well? Entry: Control and computation Date: Sun Sep 21 11:36:21 CEST 2014 I don't want annotation to a standard assembler. I do not want to do register allocation, just instruction selection. This is a DSL and has the bad selling points of a DSL that it's a new language... Maybe the problem in rai is that it needs a way to separate control from data. Keep control as part of the language, but allow the primitive data flow set to be extensible. Entry: Too complicated Date: Sun Sep 21 11:47:51 CEST 2014 So is there anything that can be done to make this incremental? First, let's fix the dataflow primitives. Just looking at the instructions available, the impression I have is that this problem is just too complicated: a machine's instruction set leaves a lot of room for specific optimization. So what is the process that a human assembly programmer is performing? Knowing the instruction set and knowing where the data is and how it needs to be programmed, find a way to do it with a minimal amount of "fuss". Entry: Goals, problems and orthogonality Date: Sun Sep 21 13:55:59 CEST 2014 The problem domain for RAI is time-recursive algorithms. The kind that does not map well onto parallelism, unless it is "dumb duplication" as parallel or pipelined serial filters. Most DSP algorithms in the field are probably time-block based, or if they are not, the operations at each control step are "wide" enough such that the core routines are still matrix routines. The problem I see is that vector instructions impose a lot of structure in the data layout. The main problem is not doing the actual computation, but getting the data in the right spot. DSP implementation is a *ROUTING PROBLEM*. And routing problems are combinatorial problems. What I really want is an algebra to express all these forms of routing and permutation. DSP is a *COMBINATOR PROBLEM*. The best way to go seems to pick a non-trivial problem and learn from it. I have one: pitch/timbre controlled dynamic wavetable synthesis. It's sufficiently ad-hoc to not fit a particular mold. Entry: Primitive pool Date: Sun Sep 21 14:04:56 CEST 2014 What is needed is a primitive pool. A collection of primitives that can be implemented in terms of each other. This allows the interface to remain the same: simply add primitives, but add a mechanism for "default implementations". In first iteration this can be directed, but later might be optimized differently. Put it differently: Add primitives to the language whenever they are supported on a particular platform, then allow for them to be implemented by a library function whenver they are encountered. This allows: - target compilation to use native instruction - abstract interpretation to use the library functions Entry: Dataflow explorer Date: Sun Sep 21 14:33:37 CEST 2014 So, let's try to build something to explore an instruction set. Essentially, an assembler + simulator + primitive allocator. Something to explore the building of core routines to gain intuition for a new architecture. Entry: Cortex M4F Fixed point and Floating Point Unit : ARM online docs Date: Sun Sep 21 14:46:59 CEST 2014 Floating [1][2] - 32 single precision registers - mode settings for subnormals and NaN - instructions and cycle counts [2] - 1 extra pipeline delay if result is used in next instruction - convert float/int = 1 cycle - ADD, MUL = 1 cycle, MAC = 3 cycles - DIV, SQRT = 14 cycles (slots can be filled up with integer instructions) Fixed [3]: - 16 registers - all instructions single cycle except divide (2-12) - no pipeline delay when result is used in next instruction (??) [1] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0439d/BEHFGGGE.html [2] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0439d/BEHFGGGE.html [3] http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0439d/BEHFGGGE.html Entry: Fixed point Date: Mon Sep 22 15:48:13 CEST 2014 So, go over the M4F fixed point instructions and see how it best maps to existing rai functions. Then extend the primitives to a sensible goldielocks subset. - saturation - count leading zeros - mac - mul w x w-> w (same-wordsize multiplication) - pack / unpack - extend - bitfield Notes: - selecting multiplications: sign, top/bottom, accumulate, - additions: - Q / GE setting? - halved results (is this for FFT?) - exchange - dual So apart from saturation these are all easily expressed as combinations of primitives. Most of the effort in the DSP instructions is in routing operands and results. Moving stuff around... So the question: - how much of this can practically be done by the compiler? are there any compilers accessible to me at this point that actually do a good job at this? - if the "routing optimization" problem is in need for human assistance, maybe best to start from that point: make things into a "puzzle" that automates most of the red tape, but leaves the cleverness to the human operator. Practically, for getting fixed point things to run on the axo, it's probably best to work with what's there and add saturation, bit count (rounded log 2). then see if a postprocessing step can insert intrinsics or leave it entirely up to the compiler. i'm interested in the idea of "augmented assembler" but not sure if i want to get into general compiler things like register allocation and instruction selection. Entry: Testing ARM fixpoint Date: Mon Sep 22 18:32:22 CEST 2014 Using arm-linux-eabi-gcc and qemu-arm-static. Maybe this can even hook into jack? If not, use TCP to decouple archs. Entry: algebra Date: Sun Sep 28 15:51:30 CEST 2014 Next? Work on the language part. Find a closer link between idea and expression of idea. That's where the beef is. It's always possible to fill more time with low-level optimizations.. This is better left to experts. Let's make some noise. Entry: Live coding assembler Date: Sun Sep 28 19:09:32 CEST 2014 Basically, how short can the type-to-sound delay be made? It seems to always be important here to separate algorithm design from efficient implementation. Optimization costs time. Optimized compilation can not be instantaneous. So a live coding instrument is necessarily an interpreter. What if the interpreter is the CPU? Live code assembly? Since making music goes better for me when I limit myself, maybe the same goes for writing code? Start with M4F fixed point assembly, and build a live coding language on top of that. The end result will be useful, working, optimal code. So the idea is to make this work first for raw assembly with instantaneous response. Then to build abstractions on top of it. Fast DSP code is register-optimized DSP code, so that's what this does. Since it is dataflow, maybe do this as a graphical language? Entry: Live editor Date: Sun Sep 28 19:28:02 CEST 2014 Fluxus would probably be a good platform to do this. Though emacs should work as well. Every key stroke should be doing something. No, not every keystroke. It should be possible to edit a number without having the intermediate steps be visible, so at least it should be possible to turn live on/off easily. frtime? Entry: Editor Date: Sun Sep 28 19:43:41 CEST 2014 Lighttable has some of these features, but damn it depends on a lot of stuff. I want something simple to start with. So start from scratch. As usual. What is currently the easiest way to have a racket program update a text display? Let's just try curses and see where that goes. Start with a structured editor on curses[1]. racket -pm neil/charterm/demo But that's not all. I also want graphics with fast updates. Looks like openGL[2] is what is needed. [1] http://www.neilvandyke.org/racket-charterm/ [2] http://docs.racket-lang.org/sgl/main.html Entry: LLVM Date: Sun Sep 28 21:11:43 CEST 2014 The thing to focus on for rai is LLVM. I'd like to use rai for data-intensive modeling as well. Something that needs horsepower on the host cpu. Most of these things do not need "structural" flexibility that racket offers. The result is state machines performing math ops. Let's find something that does that. Entry: Purpose - something new Date: Tue Sep 30 15:34:13 CEST 2014 Yeah eventually I'll get to building that awesome 303 / 808 clone but let's face it, not much novelty in there.. Something like these new-old acoustics fwd by Antti [1][2][3][4]. [1] https://www.youtube.com/watch?v=c0xxnFqdBCE [2] https://www.youtube.com/watch?v=xSsXihuHcoo [3] https://www.youtube.com/watch?v=EDQgU1CPpis [4] https://www.youtube.com/watch?v=Kh1KMvyer4E Entry: tree reorg Date: Thu Oct 2 15:41:22 CEST 2014 moving rai/rai/* to rai/* -> something changed in the default directory structure of packages. not sure what/why - this is all trial and error. [master] tom@tx:~/rai$ ls -al total 60 drwxr-xr-x 8 tom tom 4096 Oct 2 15:40 . drwxr-xr-x 42 tom tom 4096 Sep 25 15:38 .. drwxr-xr-x 2 tom tom 4096 Sep 22 13:26 bin drwxr-xr-x 2 tom tom 4096 Oct 2 15:08 doc drwxr-xr-x 8 tom tom 4096 Oct 2 15:40 .git -rw-r--r-- 1 tom tom 104 Sep 28 2013 .gitignore drwxr-xr-x 2 tom tom 4096 Oct 2 15:08 gui -rw-r--r-- 1 tom tom 517 May 31 2013 info.rkt -rw-r--r-- 1 tom tom 1533 Jun 12 2013 LICENSE -rw-r--r-- 1 tom tom 216 Sep 27 14:07 Makefile drwxr-xr-x 3 tom tom 12288 Oct 2 15:40 rai -rw-r--r-- 1 tom tom 199 May 31 2013 README.md drwxr-xr-x 2 tom tom 4096 Oct 2 15:40 slv2 [master] tom@tx:~/rai$ git commit -am 'tree restructure' [master db0edc1] tree restructure 73 files changed, 25 insertions(+), 382 deletions(-) rename rai/ai-array-c.rkt => ai-array-c.rkt (100%) rename rai/ai-array-scheme.rkt => ai-array-scheme.rkt (100%) rename rai/ai-array.rkt => ai-array.rkt (100%) rename rai/ai-autodiff.rkt => ai-autodiff.rkt (100%) rename rai/ai-complex.rkt => ai-complex.rkt (100%) rename rai/ai-eval.rkt => ai-eval.rkt (100%) rename rai/ai-freq.rkt => ai-freq.rkt (100%) rename rai/ai-linpar.rkt => ai-linpar.rkt (100%) rename rai/ai-network.rkt => ai-network.rkt (100%) rename rai/ai-poly.rkt => ai-poly.rkt (100%) rename rai/ai-proc.rkt => ai-proc.rkt (100%) rename rai/ai-stream.rkt => ai-stream.rkt (100%) rename rai/ai-symbolic.rkt => ai-symbolic.rkt (100%) rename rai/ai-taylor.rkt => ai-taylor.rkt (100%) rename rai/arm-dsp.rkt => arm-dsp.rkt (100%) rename rai/compalg-lib.rkt => compalg-lib.rkt (100%) rename rai/compalg.rkt => compalg.rkt (100%) rename rai/debug.rkt => debug.rkt (100%) rename rai/demo.rkt => demo.rkt (100%) rename {rai/doc => doc}/ai-ssa-example.rkt (100%) rename {rai/doc => doc}/rai.scrbl (100%) rename rai/exp.rkt => exp.rkt (100%) rename rai/f32vector.rkt => f32vector.rkt (100%) rename rai/fam-sp.rkt => fam-sp.rkt (100%) rename rai/gnuplot.rkt => gnuplot.rkt (100%) rename rai/libproc.rkt => libproc.rkt (100%) rename rai/main_lv2.rkt => main_lv2.rkt (100%) rename rai/matrix-lib.rkt => matrix-lib.rkt (100%) rename rai/meta.rkt => meta.rkt (100%) rename rai/number.rkt => number.rkt (100%) rename rai/old-ai-stream.rkt => old-ai-stream.rkt (100%) rename rai/peval.rkt => peval.rkt (100%) rename rai/prim.rkt => prim.rkt (100%) delete mode 100644 rai/bcr2000.c delete mode 100644 rai/test-lang.rkt rename rai/{ => test}/test-ai-array.rkt (100%) rename rai/{ => test}/test-ai-autodiff.rkt (100%) rename rai/{ => test}/test-ai-c.rkt (100%) rename rai/{ => test}/test-ai-freq.rkt (100%) rename rai/{ => test}/test-ai-linpar.rkt (100%) rename rai/{ => test}/test-ai-network.rkt (100%) rename rai/{ => test}/test-ai-proc.rkt (100%) rename rai/{ => test}/test-ai-stream.rkt (100%) rename rai/{ => test}/test-ai-taylor.rkt (100%) rename rai/{ => test}/test-ai.rkt (100%) rename rai/{ => test}/test-array-scheme.rkt (100%) rename rai/{ => test}/test-axo.rkt (100%) rename rai/{ => test}/test-delay.rkt (100%) create mode 100644 rai/test/test-lang.rkt rename rai/{ => test}/test-lc2.rkt (100%) rename rai/{ => test}/test-libproc.rkt (100%) rename rai/{ => test}/test-map.rkt (100%) rename rai/{ => test}/test-matrix-lib.rkt (100%) rename rai/{ => test}/test-series.rkt (100%) rename rai/{ => test}/test-type.rkt (100%) rename rai/{ => test}/test-win32.c (100%) delete mode 100644 rai/test_benchmark.rkt delete mode 100644 rai/test_jack.rkt delete mode 100644 rai/test_pd.rkt delete mode 100644 rai/test_pd1.rkt rename rai/sp-host.rkt => sp-host.rkt (100%) rename rai/stream-lib.rkt => stream-lib.rkt (100%) rename rai/stream-meta.rkt => stream-meta.rkt (100%) rename rai/stream-native.rkt => stream-native.rkt (100%) rename rai/stream-syntax.rkt => stream-syntax.rkt (100%) rename rai/stream.rkt => stream.rkt (100%) rename rai/stream2c.rkt => stream2c.rkt (100%) rename rai/synth-lib.rkt => synth-lib.rkt (100%) rename rai/synth.rkt => synth.rkt (100%) rename rai/tools.rkt => tools.rkt (100%) rename rai/type.rkt => type.rkt (100%) rename rai/yield.rkt => yield.rkt (100%) [master] tom@tx:~/rai$ Entry: restructure Date: Fri Oct 3 18:36:21 CEST 2014 rai/*.rkt -> simple require path rai/src -> *.h *.c rai/test -> test in terms of rai/... Entry: Remove the s-exp thing Date: Thu Oct 9 08:44:36 CDT 2014 (module reader syntax/module-reader #:language `rai/stream) Entry: Moving things forward Date: Sat Nov 29 13:27:41 EST 2014 Haven't had a lot of clarity lately - life taking too much energy. Looks like the coming week will be open to get some things out of the way and make some noise! Thigs to do: - Pattern sequencer / rhythm generator - OSC frontend Entry: 4 on the floor Date: Sat Nov 29 13:34:59 EST 2014 Let's get a simple bass drum working. Involves the whole chain: - sequencer - OSC or MIDI - bass drum synth - audio output Entry: Sequencer from first principle Date: Sat Nov 29 14:13:43 EST 2014 Basically what I'm working towards anyway is analog synthesis, so let's make a sequencer as such. What is a sequencer? A cyclic array with a phase. So it can be driven as a phasor. Really, that's all there is to it. From the system's point of view, this is a table playback synth. I'm re-inventing Pd.. How to make table playback work in RAI? Maybe it's easier to start algorythmically first. Entry: Bresenham's Algorithm Date: Sat Nov 29 14:28:49 EST 2014 Basically I'm thinking about the Euclidian Rhythms idea[1]. Is there a way to express this directly? It seems looking at it from a Bresenham's line drawing pov. removes the "off-line" part of the algorithm and turns it into an incremental process. Let's see. The basic idea[2] is to make a decision each time to jump or not jump. This is done based on phase. At each step an increment is added to the phase (the slope of the line) and every time we pass through 0, a jump is made. Essentially, this is the same as an integrating delta/sigma modulator. I believe this is also mentioned in [1]? No in [3]. So it seems Euclid, Bresenham and SD are all related[4]. [1] http://cgm.cs.mcgill.ca/~godfried/publications/banff.pdf [2] http://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm [3] http://en.wikipedia.org/wiki/Delta-sigma_modulation [4] http://www.cs.tau.ac.il/~nachum/calendar-book/papers/bresenham.pdf Entry: Bresenham phasor/pulsor? Date: Sat Nov 29 16:13:30 EST 2014 Is a phasor enough? No. What is needed is a pulsor: something that sends a 1 on transition and 0 otherwise. Let's add that to the library. What this needs is a different primitive. `wrap01' as used in phasor is not primitive, or "primitive enough". There needs to be a wrap01 that also produces the carry bit. I.e. div/rem. I'm already using: (define (wrap01 x) (- x (floor x))) Entry: Using integers Date: Sat Nov 29 16:37:41 EST 2014 In practice, using just floating point is too limiting. Is there a way to introduce integers without making the core language typed? Typing can easily be inferred, so is there a way to fake the use of integers? Is typing actually necessary at that level? Why can't a number just be a number, leaving the float/int distinction to compile time? Let's try that first. Entry: Core language Date: Sat Nov 29 18:17:35 EST 2014 The dataflow primitives should be optional: they do not require "machinery". One of the major problems is to implement a set of N operators out of P primitives, where N > P, and there are many sets of P of different sizes. Algorithms written in set of N should be implementable in terms of a wide set of different P without too much trouble. Entry: bug: 'm' as variable name causes problems Date: Sat Nov 29 22:46:02 EST 2014 (module dsp rai/stream (provide (all-defined-out)) (define (bres (s) (inc m)) (let ((next (+ s inc))) (values (mod next m) (quot s m) )))) This is most likely due to 'm' being used in one of the C macros. Yep. Entry: Rhythms Date: Sat Nov 29 22:56:42 EST 2014 So bresenham sequence is done. How to "oversample" it? Simple: add a clock input. This needs a timer abstraction. Entry: Timers Date: Sun Nov 30 09:37:04 EST 2014 Needed: an abstraction to implement (clocked) timers without doing div/mod. It is currently very awkward to express conditional statements. Probably because they are dataflow, not control flow.. One problem is that `values' cannot be used inside of `if'. Maybe that should be changed? The other is that I find it very hard to express the logic of a clocked timer: i.e. only do somthing when clock is high. Entry: Float / Int Date: Sun Nov 30 10:31:31 EST 2014 Time to fix it properly. Where is the typing happening? Time to decouple that? Probably not.. Very interwoven and would require a separate representation - possibly making things more complicated. It seems that the primitives just need to be typed properly. Maybe best to add some explicit conversions? In order to do this properly, first set up a test. EDIT: Was already there in the form of cast/n primitive. However, that isn't correct: it decouples unification completely. It works in some cases because there is enough type information coming in before the cast. Fix this when it becomes a problem? Entry: Envelopes Date: Mon Dec 1 11:44:08 EST 2014 Rewrite envelopes in terms of "digital logic" primitives? Entry: Generative Piece Date: Mon Dec 1 11:53:35 EST 2014 There is enough infrastructure now to make a generative piece. Start with the livecode patch? Comments in test-lc2.rkt say it doesn't work well and should probably use jack directly. I'm more inclined to use pd. IIRC that did work last time. Try src/sp_test.pd This needs src/sp_host.pd_linux .pdrc: -path /home/tom/git/rai/src Can't load synth.sp which is part of kmook project. Shall I just include the synth in rai? No let's keep that separate for now. Later move demo into rai project. pd.local sp_test.pd Entry: Not Pd Date: Mon Dec 1 19:01:17 EST 2014 I want to just run it on the command line. Probably best to start with pulseaudio. Entry: pulseaudio and jack Date: Mon Dec 1 19:15:39 EST 2014 The short of it seems to be: don't[1]. Maybe add a pulseaudio transport. rsound already has this? I believe it worked before but no longer does.. Maybe that was just jack. [1] http://jackaudio.org/faq/pulseaudio_and_jack.html [2] http://freedesktop.org/software/pulseaudio/doxygen/pacat-simple_8c-example.html Entry: linux and audio Date: Mon Dec 1 19:26:39 EST 2014 Looks like this just got more and more messy.. Stick to one thing? That would be jack. rsound tries: - pcm_dsnoop - pcm_dmix - pulse - jack Entry: native pulseaudio Date: Mon Dec 1 19:33:00 EST 2014 Is it hard to just connect to pulse? Nope. See main_pulse.c and the %.pulse rule. No more excuses! EDIT: No more inspiration either.. So let's fix some bugs. Entry: Duplicating outputs Date: Tue Dec 2 11:11:41 EST 2014 Fix by adding another copy operation? I have a faint memory if this being problematic for other reasons though.. This is what I mean. Why is this line commented out? ;; (out-nodes (copy-nodes semantics out-nodes-packed)) Entry: Integers Date: Tue Dec 2 11:26:42 EST 2014 Trying to make a 32bit overflowing phasor and it seems the typig falls short here. Can't express it such that state is integer. (define (phasor32 (s) (period32)) (let* ((s_next (+ s period32))) (* (/ 1 #x7FFFFFF) (float s)))) Entry: 1 - 1 ? Date: Tue Dec 2 11:41:49 EST 2014 I don't remember what this is about: float32_t r3 = p_copy(1.0); float32_t r4 = p_sub(r3,1.0); It's part of the delay nodes - used for modulo 2^n indexing. Entry: Types.. Date: Tue Dec 2 11:49:17 EST 2014 So there is a problem. Because of the implicit nature of state, I've opted to use an ad-hoc typing method[1]. However this seems to make it impossible to express certain constraints. Currently still lacking insight to tackle this. E.g. I want to express that a base type is Int, while it might be eventually typed as a grid. This seemingly needs some more infrastructure in the unification. I.e. some dormant annotation that kicks in when the recursive unification makes it to the leaf nodes of the type graph. [1] entry://20140114-160405 Entry: Grid types explained (again) - Where is my morphism? Date: Tue Dec 2 13:35:16 EST 2014 The `traditional' way of typing in a functional language is to deconstruct containers, i.e. a type (C t) will yield values of t that can be individually presented to a mapped or folded function. In RAI, the loop operator "unpacks" values, but their types are kept as grids. Some magic happens whichs lifts all operations and values from scalars to grids of dimension d to dimension d+1. This is done in order to make it simpler to construct state associated to causal sequence processors. Somehow this representation encodes the morphism between (abstract) causal stream operators and programs implementing those in terms of iterated functions. But it is weird and inconsistent in the way I word it. However it works fine to express what I want, which is to track the coordinates and dimensionality associated to each (named) node, in order to add hidden nodes to contain state. I find myself swimming in confusion about how to make this more concrete or formal. It feels that I discovered something that already exists, but in a slightly different form and I don't recognize it. So what is wrong with this picture? Why are there two sides to this: scalars (there is an "unpack" operation as in:) Entry: loop and values should commute Date: Tue Dec 2 14:15:52 EST 2014 I have a feeling this is currently an arbitrary limitation. The problem seems to be the (+ accu body) in the `mix' macro: (define-syntax mix (syntax-rules () ((_ (i ...) ;; same as `loop' index spec vs body) (loop (i ...) ((accu 0)) vs (+ accu body))) ((_ size streams body) (mix size () streams body)))) I have a feeling I'm going to keep running into special cases like this until there is a more formal way of thinking about grids.. Solve it with a mix2 form for now.. The problem seems to be that there are two mechanisms at work here: grids and the `values' construct. Again the packed vs. unpacked thing... Jumping between levels of abstraction. Entry: Stereo as grids? Date: Tue Dec 2 14:48:12 EST 2014 Is there even a way to express this? (define (main samplerate) (loop (c (nb_channels 2)) () () (float c))) While it does actually see there are two channels through #define proc_size_out PROC_NB_EL(struct proc_out, float*) There is a crash because the array doesn't get indexed properly: for (int i0 = 0; i0 < 2; i0 ++) { out->r10[t] = p_copy(r9); } Instead of: for (int i0 = 0; i0 < 2; i0 ++) { out->r10[i0][t] = p_copy(r9[i0]); } Also the buffer allocation doesn't work either. There are a bunch of ad-hoc assumptions that make this not work out. It seems that all transformation happen in ai-array.rkt (before ai-array-c) (define (annotate-def lc) (lambda (v) (match v ((list-rest '! v index) `(! ,v ,@index ,@(time-coords v))) (v (if (external-node? v) `(! ,v ,@(time-coords v)) ;; *** `(,(node-base-type v) ,v)))))) (define (annotate-ref lc) (lambda (v) (match v ((list-rest '@ v index) `(@ ,v ,@index ,@(time-coords v))) (v (if (external-node? v) `(@ ,v ,@(time-coords v)) ;; *** v))))) My guess is that the lines marked *** should also have (loop-context-indices lc). So that works for the output, but it turns out that the `internal-node?` hack is actually not consistent. float32_t r9[2]; for (int r7 = 0; r7 < 2; r7 ++) { float32_t r8 = p_copy(r7); r9[r7] = p_copy(r8); } for (int i0 = 0; i0 < 2; i0 ++) { out->r10[i0][t] = p_copy(r9); } Not all internal nodes are scalars. Which is something I already ran into at another point. I'm thinking that it might be better to avoid this kind of condensed hack and just make the loop transformations explicit. Try to solve that as a separate problem. Or leave it to something like LLVM. The insight I'm really missing is these loop transformations. In the end, RAI array compiler does only two things: - Insert state storate and updates. - Eliminate temporary storage through reuse (internal nodes hack). Make this explicit. Entry: Big problem? Date: Tue Dec 2 16:25:40 EST 2014 So to keep this realistic: how to fix this without doing a full rewrite? The only thing that needs to be figured out is how to type a node. More precisely: the explicit types are available. The question is: which storage dimensions can be optimized? E.g. giv a node type (a b Int) Which one of the the dimensions can be condensed? none: (a b Int) b: (a Int) a: (b Int) both: Int The problem is that loop transformations can influence this. E.g. loop merging. Entry: How to think about loops? Date: Tue Dec 2 16:32:07 EST 2014 This problem has been bugging me for what... 20 years? It looks like I've reached a point where I can't ignore it any more. There are no hacks that can solve this.. It needs insight.. I need a loop transformation algebra. In Feldspar this is called Vector Fusion[1]. Maybe a good way to move forward is to express the feedback generation part in Feldspar? [1] http://www.cse.chalmers.se/~ms/MemoCode.pdf Entry: Next: two loop problems Date: Tue Dec 2 17:20:21 EST 2014 - Incorrect references for the stereo grid. - Problems with intermediate results I ran into while doing reverbs. Both need a smarter way to determine what parts of a grid node can be re-used. How to start? The problem is equating "zero-dimensional" with "internal node". At some point the knowledge pops up that storage is indexed, e.g. (! rx ry). At this point it should also be noted that further references are indexed. That knowledge is lost in the ad-hoc handling.. Why is it known? It's at definition point, so it's obviously known. What about making the "virtual type" and "implementation type" explicit? I.e. for input/output nodes, both are the same, but for internal nodes they can be different. Roamap: - Write an example and print out types - Find a place to put the extra annotation - Replace the current 'internal-node?' based guessing. Entry: Can't rewrite yet Date: Tue Dec 2 20:29:27 EST 2014 The structure of the implementation is probably "beyond repair". To fix this means to rewrite it, and it seems best to do that once the typing can be expressed in Haskell. To use a bad metaphor, this has gotten "too big to fail", i.e. it's too useful to throw away until a full replacement is ready. Entry: Fixing array annotation Date: Wed Dec 3 11:27:38 EST 2014 - add `imptype' dictionary - for each definition (through `annotate-def') add an element to this dictionary. - for each reference (through `annotate-ref') refer to the dictionary Entry: Still not fixed.. Date: Wed Dec 3 12:30:15 EST 2014 (define (megasaw6) (let ((f .001)) (loop (c (nb_channels 2)) () () (mix (v (nb_osc 50)) () (let* ((v1 (* 0.001 (float v))) (v2 (* v1 v1)) (v3 (* 0.02 (float c))) (g 0.0005) ) (* g (saw-d3 (* f (+ 1 v1 v2 v3))))))))) (define (main samplerate) (megasaw6)) Simplifying: (define (test6) (loop (c (nb_channels 2)) () () (mix (v (nb_osc 50)) () (float v)))) for (int t = 0; t < t_endx; t ++) { struct proc_si * restrict si = (struct proc_si *)(&state[(t^0)&1]); struct proc_so * restrict so = (struct proc_so *)(&state[(t^1)&1]); float32_t r0 = p_copy(t); float32_t r1 = p_copy(t_endx); float32_t r3 = p_copy(1.0); float32_t r4 = p_sub(r3,1.0); so->r6 = p_sub(si->r5,1.0); float32_t r12[2]; for (int r7 = 0; r7 < 2; r7 ++) { float32_t r9 = p_copy(0.0); for (int r8 = 0; r8 < 50; r8 ++) { float32_t r10 = p_copy(r8); float32_t r11 = p_add(r9,r10); r9 = p_copy(r11); } r12[r7] = p_copy(r9[r7]); // *** } for (int i0 = 0; i0 < 2; i0 ++) { out->r13[i0][t] = p_copy(r12[i0]); } } Problem here is r9 in the *** marked line. This is probably defined as a (! ..) operator but still has the grid type. Essentially we just need to replace (! . is) with (@ . is). EDIT: Almost: if there's a second loop, the index variable name changes so always take it from current loop context. Entry: Is the fdn problem solved now? Date: Wed Dec 3 14:30:55 EST 2014 I ran into a problem expressing intermediate results when implementing FDNs as loops. Is this still a problem? Entry: Live coding : make every number a parameter Date: Wed Dec 3 21:13:54 EST 2014 Basically, update continuously. However that's not realistic because compilation takes too long, but as an intermediate point: turn all numbers into parameters and update them continously from an editor. Or is it? Why is compilation so slow? What about doing the same as in Staapl: add a namespace? Trouble is that I want to change the "language" on the fly. Entry: emacs Date: Wed Dec 3 22:16:50 EST 2014 Live coding in emacs: - Make saving simpler - add up/down arrow editing ;; logarithmic number editing (require 'thingatpt) (defun my-m-up () (interactive) (my-scale-number-at-point 1.1)) (defun my-m-down () (interactive) (my-scale-number-at-point 0.9)) (defun my-scale-number-at-point (factor) (let* ((word (thing-at-point 'symbol)) (num (string-to-number word))) (when (not (zerop num)) ;; Only operate on numbers (let* ((updated (* num factor)) ;; Round digits through exponential notation. (snum (format "%.1e" updated)) ;; But keep ordinary number format (snum (number-to-string (string-to-number snum))) (bounds (bounds-of-thing-at-point 'symbol)) (start (car bounds)) (stop (cdr bounds))) (delete-region start stop) (insert snum))))) (global-set-key (kbd "") 'my-m-up) (global-set-key (kbd "") 'my-m-down) Combine this with M-F3 as save. http://www.emacswiki.org/emacs/ThingAtPoint Entry: Snarfing params Date: Thu Dec 4 17:17:47 EST 2014 - Create a special form that names each parameter - Keep parsing at high rate until the structure changes. If it doesn't change, send only parameter values. - Listen for parameters on stdin. So got a macro form to gather parameters, but it's not integrating well into ai-lambda / ai-app. Probably doing something unorthodox here that should be solved somewhere else. There is already an explicit treewalker, right? Solved by adding to stream-lib.rkt which has correct bindings. Next: - defaults / initial values. - param updates? EDIT: Done initial values. Entry: Emacs side Date: Fri Dec 5 18:18:08 EST 2014 To get faster updates, probably every edit action in an emacs buffer should synchronously trigger sending out parametesr. Why not parse the s-expression in emacs directly? Not really that hard.. (thing-at-point 'sentence) returns a function EDIT: ;; RAI (defun my-send-nums () (interactive) (let* ((cmds '()) (str (thing-at-point 'sentence)) (expr (read str))) (labels ((gather (it) (cond ((listp it) (mapc #'gather it)) ((numberp it) (let ((cmd (format "p%d %f;\n" (length cmds) it))) (push cmd cmds)))))) (gather expr) (message (apply #'concat (reverse cmds)))))) Entry: emacs udp Date: Sun Dec 7 19:46:38 EST 2014 (defun osc-make-client (host port) (make-network-process :name "OSCclient" :coding 'binary :host host :service port :type 'datagram)) https://github.com/mlang/emacs-lisp/blob/master/osc.el Entry: one number at a time Date: Sun Dec 7 19:58:45 EST 2014 so in pd we have: - inc/dec current number - gather all numbers from current expression - send over udp socket currently it sends all parameters. how to send only the current one? probably not necessary.. It's probably also better to explicitly tag the numbers that should be parameters using a "p" tag. Define this as a syntax parameter in scheme? Basically just something to reserve the name. Or maybe something like quote/unquote/quasiquote? These are not used for anything and supported in emacs. Unquote seems best as it has no standard meaning outside of quasiquote. Yeah can't figure out unquote ("not inside quasiquote") without going deeper. Let's use quote: it's already optional for numbers. Entry: Structural addresses? Date: Sun Dec 7 21:23:53 EST 2014 So what about using structural addresses instead of sequential ones? That would keep naming consistent when just changing the quotes. EDIT: Numbering the nodes in a tree traversal should be enough. Entry: Live coding Date: Tue Dec 9 09:48:53 EST 2014 So, two mechanisms: 1. Change structure -> recompile -> reload 2. Change param -> send message from emacs Second one works well. First one needs a fix. I believe that test-lc2.rkt is obsolete. This was in the Makefile: livecode: $(RACKET) fam-sp.rkt synth.rkt ./pd_notify.sh Entry: BCR-2000 setup Date: Tue Jan 20 17:01:21 EST 2015 Let's get some awkwardness out of the way. I'm always messing with this shit and quite tired of it. Have it connected to raspi. Need it to broadcast over local net so anything can hook into it. Need to add midi learn to all apps (incuding fucking TV volume control) and probably should have inc/dec mode only. So: - set up inc/dec mode - broadcast midi from raspi Entry: socat broadcast midi Date: Tue Jan 20 17:03:10 EST 2015 UDP-SENDTO Run it from udev when SUBSYSTEM=sound ACTION=add echo $DEVNAME | grep /dev/midi && socat -u file:$DEVNAME UDP-SENDTO:239.0.0.1:12345 # socat -u UDP4-RECV:10000,ip-add-membership=239.0.0.1:192.168.11.238 - |hd # socat -u UDP4-RECV:10000,ip-add-membership=239.0.0.1:0.0.0.0 - |hd Entry: next? Date: Fri Feb 13 18:25:46 EST 2015 What's next? I've been thinking about porting the model to Haskell. Not sure if this is really useful at this point. Maybe best to clean up the model internally first. There are some ugly ad-hoc things in there that are quite useful in practice, but would need some effort to type properly: - delay lines - control rate hacks - fixed primitives - parameter range specs Look at stream-syntax.rkt for hints of these. Currently no inspiration.. Probably best to move to application? Entry: Abstracting primitives Date: Fri Feb 13 18:39:23 EST 2015 One of the interesting changes is to make the set of primitives programmable, to be able to target different machines. It might be too constraining to keep a single base language. E.g. allow implementation in base language but allow mapping to different primitive cores. [1] entry://20141129-181735 Entry: Application Date: Fri Feb 13 19:21:50 EST 2015 Problem right now is really user interface. I've got all these synths I want to build but I keep getting stuck in finding ways to make them playable / explorable. Core OS will probably be in Rust eventually. I had a couple of days of deep-dive and I feel I'm ready. Should probably think about ARM-targeting to keep things lean? For now thought the problem is graphics and UI/UX. Rai can be used to generate computations for visualizations. I might take a detour finishing logic analyzer visualizations. Entry: live coding Date: Tue Feb 17 01:53:59 EST 2015 Added notes to emacs/rai.el to point the way. Entry: next? Date: Tue Feb 17 02:40:50 EST 2015 Ride the livecode wave. Eliminate obstacles and see what happens. Entry: Merging live coding strategies Date: Wed Feb 18 03:01:56 EST 2015 See doodle.lc2 Currently the main_pulse.c checks a file's time stamp, but this could better be the executable instead of source, i.e. let somebody else watch source and compile it, then let program restart. Entry: rai image Date: Fri Feb 20 21:58:50 EST 2015 I forget so much... With geiser it's possible to display racket's images in interaction. I did use this before to print plots. Where is that code? See rai/compalg.rkt Entry: Next? Date: Sun Feb 22 17:58:29 EST 2015 What is the problem? Why is there no music coming out of my speakers? I played a bit with the new emacs stuff and it's nice, but it's still cerebral. How to dig back into that non-cerebral thing? I keep saying i want knobs, but i have this big box of knobs sitting right next to me and I'm not using it. Somehow the _setup_ problem is still the big one. How to set the table such that I can sit down and play without having to _think_, ruining the mood.. Some systematic thing that seems to work: - just whip things up - find bounds and scales and map them to knobs - start exploring the field Entry: Workflow Date: Thu Feb 26 02:16:59 EST 2015 Combine the emacs editing with macros (visual programming!) and the bcr2000 knob box. Basically, use emacs "infinite" log scale for setting the limits of parameters. Entry: Visualize Date: Tue Mar 3 23:50:04 EST 2015 Made basic draft of 2D gl widget collection. Point is to visualize. Keep controls in hardware knobs for now, possibly with on-screen feedback, but don't worry about editing with the mouse. Entry: The Death of Optimizing Compilers Date: Sun Apr 19 20:58:03 EDT 2015 [1] https://lobste.rs/s/slioj7/the_death_of_optimizing_compilers Entry: Next? Date: Sat Jul 11 01:25:53 EDT 2015 4 months of madness preventing any work... So what's next here? The remaining problems are not really technical. They are mostly fear about not having any musical ideas when using this, and then getting lost in ambitious abstraction and typification.. Might be good to ask some annoying questions. What do I really want to accomplish here? Is the point making sound, or is the point growing in understanding of computer programming, as a goal in itself? Entry: Haskell types Date: Sun Jul 19 13:40:33 EDT 2015 It would be nice to write out the core of this as Haskell operators, to see how they commute. In the current racket implementation all of this is a big ball of mud. - composition of state machines with state 'accumulation'. - introduction of block-rate and other sample rate up/down conversion - spatial operations (essentially feldspar) - abstraction of data primitives (with control being fixed). I'm moving this back into meta/dspm where it belongs. [1] http://zwizwa.be/darcs/meta/dsmp/DSPM_RAI.txt Entry: GCN Date: Sun Jan 24 23:45:15 EST 2016 https://en.wikipedia.org/wiki/Graphics_Core_Next https://en.wikipedia.org/wiki/AMD_Radeon_Rx_200_series I have this: Radeon R9 270 Nov 13, 2013 Curaçao PRO GCN 1.0 Might be interesting to start playing a bit with programming this thing. All fairly recent developments: asm/dasm: https://community.amd.com/thread/190822 Entry: Driver application Date: Mon Jan 25 15:35:25 EST 2016 Trouble these days is really the mindset. Too skewed towards "real life" -- payed work pulling me into economic mindset. So what I do have is a headset on frequently. Just playing nothing, with noise cancelling on to get rid of the traffic noise. What about putting some ambient on there? Simple things. As for platform, it needs to be portable. Old cell phone would work fine. Entry: Interface Date: Mon Feb 8 22:48:51 EST 2016 Looking for inspiration. Need to find a way to experiment with structural elements and parameter range finding without having to type. Touch screens come to mind, but I also have this wacom tablet i never used.. Let's make a driver for it and see where it leads. Entry: Mix matrix Date: Sat Feb 13 09:25:36 EST 2016 Always intrigued me, but never really knew what to do with it. What about this: - start with clean slate - add module, and automatically extend the current mix matrix - visualize the matrix in a module diagram? - prune mix matrix - assign controller knobs (autolearn) - knob range finding I don't think I can repeat this often enough: the problem is user interface for composition, which involves: - instantiation - connection - parameter mapping That's really all. Do that in a way that doesn't trigger logical (language) thought. Entry: probably best to switch to OSC Date: Sun Feb 14 02:35:52 EST 2016 Here's an emacs library: https://delysid.org/emacs/osc.html Entry: To solve Date: Tue Feb 16 20:01:38 EST 2016 Vector intermediates. That's a big one. ( Make me think of applicative functors... ) Entry: Doodling rai/lc Date: Tue Feb 16 20:02:44 EST 2016 One synth a day to get back into the groove. What I forgot is that the exploratory part is really essential: once you take the local view of a single parameter, it is usually easy to imagine some kind of next processing step. However, the iterative / layered approach is *essential* to discovering these paths through soundscapes. ( Like deep learning :) Entry: retrigger state reset Date: Tue Feb 16 20:42:32 EST 2016 It would be nice to add a conditional state reset. A nice emerging property is the natural transient that happens for many of these systems. E.g.: reverb, supersaw (into distortion to limit amplitude due to initial constructive interference). Entry: Axoloti scales Date: Wed Feb 17 21:54:19 EST 2016 audio, control voltages, en parameterFrac32SMap: normal range -0x0800000 tot 0x07FFFFFF (4 bit headroom). This range is mapped in UI to -64.0 tot 64.0. Promotes use of nice numbers as opposed to fractions (a sort of percentage..) Compare to analog modular synth, +-64 Volts, 1 volt per semitone. Entry: Spice: TODO Date: Wed Feb 17 23:42:13 EST 2016 1. Assume Gv-i=0 needs to be solved numerically. 2. Later, see if the gaussian elimination can be done structurally Methods that use structure are usually iterative, so implement conjugate gradient? This is a tracking problem, so iterative solution might actually work better. Plan: - Express the multiplication by G as a program - Use the conjugate gradient method to solve the system equation - The update G(k) -> G(k+1), e.g. the computation of the derivatives, can likely be expressed as an iterative update in its own right. As long as the time step is small, all these iterative steps should be quite precise. The problem is reaching the initial state. For some circuits, this might be possible to do by powering them up slowly. Computation of derivatives: - Use autodiff to obtain program that computes derivative - To evaluate the trancendentals, use a single iteration that uses the previous value as a starting point. Question though, is will that drift? The only transcendental I need is exp, for diodes and ebers-moll. I have some evaluators for that. A possible approach is to pipeline a non-drifting version over N steps, and then replace the iterative approximation. This comes back the the essential need for "control rate" processing. md5://c360a9013f7276bfa1f860bea697c374 Entry: AKFE Date: Thu Feb 18 00:52:46 EST 2016 a knob for everything Entry: Midi knobs and structural editing Date: Fri Apr 8 09:58:17 EDT 2016 I've got the midi equipment organized in an Erlang program (studio). Next: assign knobs to numeric values in code. Do this the same way as before: traverse the tree, giving structural addresses. Doodling a bit, it seems what is necessary next is a way to relate the parsed expression back to text. Maybe this is all easier to do in racket instead of emacs. emacs -> racket: code snippet racket -> emacs: cut & replace Best to focus on edits instead of sending the whole thing. Emacs has a notion of "current expression". This is initialized through: beginning-of-thing thing-at-point Messages in more detail: emacs -> racket - operate on current number - update full expression racket -> emacs - update part or full expression Entry: Structural editor Date: Fri Apr 8 10:54:04 EDT 2016 Let's do this properly. Any change to the emacs buffer will propagate to racket. Ideally it should parse incrementally, but there doesn't seem to be a good reason to do it different than the dumb way: send it the whole thing on every keystroke. No this will be too complicated. It is important to keep the end result in mind: allow numeric parameter updates without making structural changes. This is a necessary optimization: incremental compilation isn't fast enough to keep up with a 50 Hz update rate. So the basic idea is to keep two models in sync: - Emacs buffer with textual representation - Compiled form + parameter values. To make this work, start both from the same point = the source file. Go back to that point whenever an invariant is broken. Distinguish these kinds of edits: - Number control events, e.g. from an external midi controller or using emacs to modify the current number under edit. This is structure-preserving, but needs to update the number literals in the representation. - Any other arbitrary code change. If it parses to the same structure a parameter update can be performed. Else a recompilation is necessary. I wonder if geiser can be used to do this. Likely there are at least some communication primitives to be reused. Entry: s-expr coords Date: Fri Apr 8 19:54:42 EDT 2016 Using existing emacs navigation it might be possible to use s-expression coordinates for parameter names. Entry: Ardour 5 inline display Date: Sat Aug 13 10:58:06 EDT 2016 Display plugin gui inside mixer strip. (currently display only) https://community.ardour.org/node/13842 Entry: Raw audio & packet loss Date: Mon Jan 2 22:37:23 EST 2017 Perform resampling at the receiving end to regulate buffer depth. If data is missing, just drop it with mixing and recover buffer depth again through regulation. This might be an interesting thing to build as it has a lot of different components that are useful elsewhere. Entry: oleg on stream fusion Date: Tue Jan 3 01:34:30 EST 2017 https://speakerdeck.com/biboudis/stream-fusion-to-completeness Entry: Haskell Date: Sat Mar 18 09:54:03 EDT 2017 Time to translate the core back to Haskell. Most work on streams is done. What is left: - internal buffers, i.e. "spatial algorithms" - delay lines - annotation (e.g. units, UI information) - LLVM generation Regarding LLVM: does it make sense to abandon C / GCC? Entry: Bit rot Date: Tue May 30 07:58:17 EDT 2017 last working version racket-6.8 .sp.elf problem: /usr/bin/ld: /var/tmp/proc_14961451551496145155942.sp.elf: Not enough room for program headers, try linking with -N /usr/bin/ld: final link failed: Bad value Entry: Fix build to support build VM + separate target to run the code Date: Tue May 30 08:04:51 EDT 2017 Do this on zoo, zoe, tp. Entry: entry point? Date: Tue May 30 09:21:13 EDT 2017 This is hard to remember... It should be part of "make" cat README.md .... Follow the white rabbit in test/doodle.rkt Entry: tp / core Date: Tue May 30 18:07:43 CEST 2017 - put links tp->core to racket-6.8 and ~/.racket - install libpulse-dev on tp Entry: nixos: clean build Date: Sat Nov 18 12:20:52 EST 2017 Added a default.nix Use "nix-shell --pure" to build it. TODO: install the package source somewhere. "raco pkg install --link rai" won't work if .racket is under rai/ What I want is 1. for things to build cleanly inside "nix-shell --pure" 2. (later) wrap it as a package Here, 2 might not be so important because this is really a compiler, i.e. it is an "environment". Maybe install ".rai.deps" next to "rai"? Entry: re-evaluate Date: Mon Mar 11 09:23:10 EDT 2019 So... I've had quite a bit of time working with Haskell and Rust, and was wondering if it makes sense to continue with racket, or to just go the fully typed way. One thing that might still be useful, is to use racket as a #lang frontend to some Haskell code. I want a front-end for monadic languages, and other than the compiling to categories approach, I do not find a good way. Maybe that should be checked out first? Entry: connect to exo Date: Fri May 10 11:45:47 EDT 2019 - keep the setup as is (make doodle.lc2) to allow stand alone use of the demo - provide a daemon to capture those udp events and route them to exo - use the push_change reload code instead of the make loop Entry: fixable? Date: Fri May 10 14:50:43 EDT 2019 Maybe it is actually possible to introduce intermediates in the way explored in rtl.txt So what is the difference? There was something weird about how rai uses combinators as contexts. This isn't necessarily a bad idea, but I ran into a problem with intermediate arrays. It will take a bit of time to reload the idea and implementation. Too tired for that now... EDIT: See explanation in 20141202-133516 I still can't wrap my head around it. And not sleeping properly isn't helping either.. The previous idea is probably ok: go ahead and build some synths. It will later on be easy to translate them into a more general setting. Entry: Unfused Date: Sun May 12 09:15:52 EDT 2019 Create an unfused "full grid" language first. Then look at what comes out, and solve fusion in a second pass. EDIT: Worked a bit on Language.Grid in asm-tools-grid. Reached the conclusion that all the interesting bits are about post-processing and loop transformations. EDIT: So I think it is fixable. But it will require a lot of work. Both on the Haskell side (where it might be simpler to just go ahead in Haskell), and on the RAI side. It does seem beneficial to keep both avenues open. Entry: synth.rkt Date: Mon May 13 10:57:11 EDT 2019 I don't understand how the voice allocator works. proc.c proc_voice_init_from_proc takes "voice_gate" and "voice_freq" params. do the usual jack and pulse wrapper have these activated? Maybe it's only in pd? tom@panda:~/rai$ grep -re proc_voice_init_from_proc * doc/rai.txt:proc.c proc_voice_init_from_proc takes "voice_gate" and "voice_freq" src/main_pd.c: if (0 == proc_voice_init_from_proc(x->proc_instance, &x->voice)) { src/proc.c:int proc_voice_init_from_proc(struct proc_instance *p, struct proc_voice *v) { src/proc.h:int proc_voice_init_from_proc(struct proc_instance *p, struct proc_voice *v); Ok so just look at the pd code. It still needs to be added to the other two. Entry: Pin racket version in Nix for integration in exo Date: Thu Nov 14 10:47:57 EST 2019 Which one was it? Last working version racket-6.8 Is this available in nixpkgs as is? It seems best to modify nixpkgs directly. But only _add_ stuff. Do not remove the old version. So maybe don't modify nixpkgs? Policy: only do bugfixes on nixpkgs. Put other nix files in exo/nix/ See exo/nix/nixpkgs-extra/racket tom@panda:~/exo/nix/nixpkgs-extra$ /nix/store/wljspgn904figapd273490wmqnr5nqvv-racket-6.8/bin/racket Welcome to Racket v6.8. > Entry: Pulseaudio Date: Mon Nov 18 10:28:18 EST 2019 Some practicals. How to create a pulseaudio effect? Use module-ladspa-sink https://askubuntu.com/questions/43950/how-can-i-apply-a-ladspa-plugin-to-a-pulseaudio-stream Entry: Midi jack Date: Mon Apr 20 16:41:05 EDT 2020 So I've set up the studio again. Settled on jack + jack_midi is the thing that just works. All the rest is too cumbersome to map and translate. So how to get that integrated into RAI? Entry: Cleanup Date: Thu May 28 11:45:58 EDT 2020 I'd like to use this for a secret project and integrate it with exo. Some things that need work: - resampler to run algos faster internally - bring it to more recent racket version I don't remember what is the problem with the version issue. Currently at 6.8 Let's spend some time on it. Entry: Racket updates Date: Thu May 28 11:48:12 EDT 2020 Last working is 6.8 Maybe insteall a couple of other versions and see how it goes? Maybe just bump to 6.9? That was already installed. Let's try. How to install the dependencies? It's going to be simpler to temporarily disable rsound and portaudio until the version is bumped. 6.9 actually seems to work. 7.7 also. I wonder what went wrong last time? 7.2 from my current nix also works I guess this is no longer a problem. Entry: Next Date: Thu May 28 14:06:36 EDT 2020 Integrate into exo. I want to run this as a service to avoid the Racket startup delay, and do some interactive stuff. Maybe some kind of notebook style thing. https://github.com/rmculpepper/iracket Entry: Parameterizing modules Date: Sat Jun 6 13:57:03 EDT 2020 Currently I don't think it is possible to parameterize ai-freq interpretations, so we go through a compilation step using a namespace. (define-namespace-anchor anchor) (define (svf fun f q) ;; Otherwise: define-values: assignment disallowed; cannot re-define a constant (define expr `(begin (module test rai/stream (provide ,fun) (define (,fun (s1 s2) (in)) (let* ((f ,f) (q ,q) (s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2)))) (require 'test) ,fun)) (pretty-write expr) (eval expr (namespace-anchor->namespace anchor))) This seems to be a very clumsy way to do it. The function symbol cannot be reused. Preliminary conclusion is that it might be best to not spend too much time on this... Maybe I'm just not getting it. There is ai-lambda. EDIT: Yeah I was not getting it! This isn't bad at all. Just need to read the code again and play with it. So it still doesn't seem to work. Some contract violation on number somewhere in the syntax expansion. Let's figure this out... (define (svf f q) (ai-lambda-feedback (((s1 0) (s2 0)) ;; state params + init (in) ;; inputs ()) ;; time parameters, not used (let* ((s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2)))) EDIT: Getting really confused now... time for a break. Why are these two different? (module test rai/stream (provide svf1) (define (svf1 (s1 s2) (in)) (let* ((f 0.1) (q 0.3) (s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2)))) (require 'test) (define svf2 (ai-lambda-feedback (((s1 0) (s2 0)) ;; state params + init (in) ;; inputs ()) ;; time parameters, not used (let* ((f 0.1) (q 0.1) (s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2)))) The behave differently with test function: > (test svf1) (amplitude 1) (amplitude 1) # > (test svf2) (amplitude 1) (amplitude 1) ; *: contract violation ; expected: number? ; given: (ai-z 0 #) ; argument position: 2nd ; [,bt for context] > > (require "test.rkt") (ai-lambda (s1 s2 in) (let* ((f 0.1) (q 0.3) (s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2))) (ai-lambda (in) (ai-app ai-feedback/n '((s1 . 0) (s2 . 0)) `(,in) '() update)) (ai-lambda (s1 s2 in) (let* ((f 0.1) (q 0.1) (s2 (+ s2 (* f s1))) (s1 (- s1 (* f (+ (* q s1) s2 in))))) (values s1 s2 s2))) (ai-lambda (in) (ai-app ai-feedback/n '((s1 . 0) (s2 . 0)) `(,in) '() update)) It must have something to do with symbol capture... EDIT: I really don't see it. (test svf1) (ai-semantics #) ((freq-semantics #) (state-names ((s1 . 0) (s2 . 0))) (in-nodes (#(struct:ai-z 0 #))) (time-names ()) (update #)) (ai-semantics #) (ai-semantics #) (ai-semantics #) (ai-semantics #) # > (test svf2) (ai-semantics #) ((freq-semantics #) (state-names ((s1 . 0) (s2 . 0))) (in-nodes (#(struct:ai-z 0 #))) (time-names ()) (update #)) (ai-semantics #) ; *: contract violation ; expected: number? ; given: (ai-z 0 #) ; argument position: 2nd ; [,bt for context] > Entry: Again... Date: Sat Jun 6 17:06:16 EDT 2020 This shit is hard to debug. So the problem is on line 120 in ai-freq.rkt (so-test (values-list (apply (ai-function-proc update) freq-semantics si-test))) Can I trigger this with a simpler function maybe? Because following all the indirections is quite a challenge. I missed that the problem was in the multiplication: *: contract violation But I can trigger it with summation also. I wonder if this has something to do with the identity of structs? e.g. rai/xxx is not the same as "xxx.rkt" I see the problem. Symbols are taken from context, so need to be remapped manually to ai-add etc.. Maybe this also needs a different #%app ? This is not the road to travel.... Re-export racket-define, and parameterize that way. Entry: Hooked up to exo Date: Mon Jun 8 02:07:35 EDT 2020 Together with basic web interface and bode diagram display. So what next? Entry: next Date: Mon Jun 22 23:51:12 EDT 2020 - summarize johannes' remarks - fix the float const bug - support high word fixed point multiply