mkstaboot

Posted to the developer's mailing list on January 28th, 2005 by David Young.

I got fed up with waiting 10+ minutes for mkstaboot to run, even with -u -o, so I broke mkstaboot into several stages. Now I can do routine rebuilds---after I modify, say, hslsd---in just 10 seconds.

The new mkstaboot and, by extension, build{,upgrade,iso,flash}, take two new options:

% mkstaboot -h
usage: mkstaboot [ ... ] [ -L ] [ stage ]
-L List the mkstaboot stages and quit.
stage Restart mkstaboot at this build stage.

Here are the build stages, in the order they will be built:

% ./buildupgrade -s ~/radix-nbsd/src/ -L
tools
toolenv
makewrapperenv
iso-kernel
flash-kernel
distrib
metalog
patch
makewrapper
mv-root-home
extras
users
modules
cuwinize-metalog
install
postinstallenv
flash
upgrade
floppy
iso

For example, if I run "./buildupgrade -s ~/radix-nbsd/src -f disk.img install", mkstaboot will restart at the 'install' stage of the build, skipping all of the preceding stages on the -L list, UNLESS

• the stage ends in an 'env' suffix
• the stage did not run to completion during the last build
• some prior stage did not run to completion

If I do not tell mkstaboot to restart at any stage, then it starts at the beginning.

A few caveats:

• I've tried hard to make stages idempotent, but at least one of them ('patch') is not.
• I do not (yet) leave out any build stage based on the options. That is, the 'iso' stage will run (although it will be a null operation) even if I don't use an '-i' option.
• It is still a good idea to use -u and, if you know what you're doing, -o, as they are indicated, to speed up your build.

Just a few bullets about the architecture of the staged builds:

• In steps.d/ are the build stages. They get "sourced" (with '.') by mkstaboot. They are derived from the old mkstaboot sources.
• steps.d/deps is a two-column list of stage dependencies: for every row, the stage on the left-hand side cannot be run until the stage on the right-hand side has completed. If you add new stages, be sure to add them to steps.d/deps !
• In $BUILDDIR/, mkstaboot touch(1)es .<STAGE-NAME>.begin when a stage begins; it touches .<STAGE-NAME>.end if and when the stage finishes successfully. Next time you run a build, if .<STAGE-NAME>.end is newer than .<STAGE-NAME>.begin, then the stage ran successfully.
• POSIX touch(1) has a time resolution of 1 second. Therefore, the shell command "touch a ; touch b ; test b -nt a && echo yes" does not necessarily yield the string "yes". Sometimes it's necessary to sleep for a second before touching .<STAGE-NAME>.end. Sleeping synchronously will slow the build by a lot. So mkstaboot sleeps in the background. Hence the "joining pid xxx" messages at the end of a build.
• step.subr contains most of the logic for deciding which build stages to run, when.