Mon Apr 20 13:23:58 EDT 2020

danielg on #rust about async compiler

12:16 < doelie> Hi. Quick question: who implemented the gnarly details of the async->state machine conversion? I'm trying to gauge how hard it would be to write a stand-alone
                tool that could do the same for C.
12:17 < doelie> I'm guessing some of the rust code would be reusable.
12:17 < doelie> Or if not, at least some of the struct packing heuristics.

12:50 < danieldg> doelie: it's actually pretty simple in theory.  Make a struct that contains all local variables and use that, along with one "state" variable.  Yield is
                  "struct->state = N; return YIELDED"; the start of the function contains a switch/goto on state.
12:51 < danieldg> rust uses an (unnamed) enum to hold the local variables so that it can overlap variables that are never in scope at the same time, and uses the real stack for
                  variables not live across a yield. But those are just optimizations.
12:52 < danieldg> the result is that the struct packing is not specific to the async code at all

12:59 < danieldg> doelie: imo it would be easiest to do it as part of an LLVM pass instead of a standalone C->C tool

13:10 < doelie> danieldg:
13:10 < doelie> danieldg: the llvm pass might indeed be a good thing to consider
13:11 < doelie> however i want to see how much extra work i need to do to just do it in c.
13:11 < danieldg> look for existing tools to create generators in C, it's the same thing
13:12 < doelie> any pointers?
13:12 < danieldg> no, just giving you google keywords :D
13:12 < doelie> yeah i did some looking already :)
13:13 < danieldg> I've only dealt with higher-level tools (matlab) that generate state machines and then C code from it

13:13 < doelie> i've not found a tool that can just do a C->C translation
13:15 < doelie> anways this is just to see if i'm not missing anything obvious. i did some thinking about the struct overlap to encode the continuations, and it looks quite
                similar to what rust is doing.
13:15 < danieldg> one important thing to consider is that an async rust function is actually two real functions: one setup that creates the state struct, and one that is called
                  by poll() on the struct.  Splitting functions like that in C is not as trivial.
13:15 < danieldg> unless you want to do heap allocation and function pointers all the time
13:15 < danieldg> I think C++ has some support for generators in its drafts, btw
13:16 < danieldg> might look at that
13:18 < doelie> no i want to use static/stack allocation. i've used something similar in the past, but it's quite heavy on the C macros and also doesn't do any compilation to
                state machine, just dispatch: https://github.com/zwizwa/uc_tools/blob/master/sm.h

13:20 < doelie> danieldg: i think that the main issue is going to be to find a good C parsing/printing library.  i guess llvm can be made to spit out C code again.
13:20 < doelie> but i'd like to do this such that the resulting c code remains readable.
13:20 < j`ey> clang has rewriting tools
13:26 < doelie> j`ey: thanks. i found this post: https://eli.thegreenplace.net/2014/05/01/modern-source-to-source-transformation-with-clang-and-libtooling
13:26 < j`ey> > moder
13:26 < j`ey> n
13:26 < j`ey> >2014
13:26 < j`ey> :P
13:26 < doelie> yea good ideas dont age
13:27 < doelie> but it's enough to get unstuck for me. thanks you both.