Build Systems Entry: Initial post Date: Tue Feb 5 15:50:11 EST 2019 Posts maybe collected before this one, but this is the first buildsys.txt post. It is time to solve the problem. I have suffered enough. 'make' is not good enough. Some problems that have come up: - build systems should compose, i.e. export their dependencies, such that a partial rebuild can skip calling a "bulky" sub-build - a "bulky" build system is one that is slow at evaluating small increments. - a build system should be "invertible", meaning it should support a fast redeployment into a running system Entry: redo Date: Tue Feb 5 15:53:46 EST 2019 I want to first understand https://github.com/apenwarr/redo https://redo.readthedocs.io/en/latest/ I need an example to work on. Let's start with the temper driver, which I need to reshape into an Erlang port program. EDIT: It appears this redo version uses a sqlite3 db to save the dependencies. EDIT: Do this later. For simple projects the Makefile approach is fine. Don't change too much at once. tom@panda:~/exo/deps/temperv14$ cat default.elf.do #!/bin/bash redo-ifchange $2.c gcc -MD -MF $2.d -o $3 $2.c -Os -lusb -Wall -Werror read DEPS <$2.d redo-ifchange ${DEPS#*:} EDIT: The main property of "redo" is that it uses the first run to determine how to guide successive runs. Squinting a bit, this computes the output together with its derivative: I -> (O, dO/dI) Entry: redo install Date: Fri Feb 22 15:19:42 EST 2019 The incremental "push" idea becomes just install, with an .install target for each .elf file. This can then be parameterized at the top level as well. Entry: Encode binary type in the file Date: Sat Feb 23 07:52:02 EST 2019 E.g. instead of build/host/v4l.elf use v4l.host.elf This is the same as for uc_tools, and makes it harder to confuse the files once they are moved. EDIT: Works well. # Filename encodes which linker script to use. # E.g. for foo.x8.elf, the script is x8.ld # Variables below are annotated with the foo example: # foo.x8 APP_TYPED=$(basename $1 .elf) # x8 TYPE="${APP_TYPED##*.}" # foo APP=$(basename $APP_TYPED .$TYPE) # foo.ld LD=$TYPE.ld # foo.x8.map MAP=$APP_TYPED.map # foo.o O=$APP.o Entry: elf.list Date: Sat Feb 23 09:22:53 EST 2019 Instead of including an install target in each library, do this only at the top. But provide an elf.list file, and gather all those in the toplevel. EDIT: This thing is fucking amazing. Entry: time stamp vs. hashes Date: Sat Feb 23 10:33:41 EST 2019 Not everything can be easily expressed as time stamps, so is it possible to not update a file when its regenerated content would be the same? https://redo.readthedocs.io/en/latest/redo-stamp/ Entry: Put generated files in a different directory Date: Wed Apr 3 10:08:57 EDT 2019 Specific example: ~/exo/sql .load files for .proj files are stored next to the .proj file, which is in a different tree that I don't want to mess with. Why is this? Entry: redo bugs Date: Thu Apr 4 12:17:20 EDT 2019 tom@panda:~/exo/sql$ redo ../../../../priv/darcs-private/admin/us/tax/2018/tom_expenses_cat.proj.load Traceback (most recent call last): File "/usr/local/bin/redo", line 10, in redo.cmd_redo.main() File "/usr/local/bin/../lib/redo/cmd_redo.py", line 75, in main state.init(targets) File "/usr/local/bin/../lib/redo/state.py", line 119, in init db() File "/usr/local/bin/../lib/redo/state.py", line 48, in db os.mkdir(dbdir) OSError: [Errno 2] No such file or directory: '/home/tom/p/.redo' This is likely "mkdir -p" that is buggy. It doesn't seem possible to erase redo state without erasing the whole directory. Erasing state creates a problem with all the generated files that are suddenly seen as not generated, so don't do that. Anyways, the problem can probably be avoided by not generating files in remote directories, Entry: redo Date: Sat Apr 6 08:19:33 EDT 2019 It seems I have a tendency to factor this out into a lot of small files. Find a way to do this differently.