Bug 40249

I have a rough implementation of parallel install checks that currently works on  mipsel (secondary) and riscv64 girar instances. It uses the same build node and user and runs install checks in parallel using several hasher instances (with different --number). For 4-core Loongson3A I run 5 install check threads, and it gives from 1.6 to 3 times speed up, depending on the task. Also, for trivial tasks where only one binary package is build parallel version may be slower by a second or so.

I'm not posting links to code here mainly because I believe that it will be more effective to write something from scratch then to forward-port & clean up my implementation onto current girar (I'm still using an ancient pre-ga version). Still, I can write smth up on what and how I was doing, if you think it'll be useful.

(In reply to Dmitry V. Levin from comment #0)
> When a task built more than just a few binary packages, it makes sense to
> parallelize gb-task-check-install-arch because the only thing that could be
> shared between install checks is the cache.

Since the cache is so important for install checks, it would make sense to generate the cache once and subsequently reuse it by parallelized install checks.

The next iteration of the parallelized install checks were deployed on loongarch64 girar instance last monday (2024-03-25). It works stably so far. We use 8 workers, as more install chroots would probably exhaust the free space at $TMPDIR. On heavy tasks with enough binary packages we see install checks became 3.5-6.5 times faster, the heavier the task is the better. On small tasks (e.g. updating one small noarch package), the install checks became slower for a few seconds, as some cache-related optimizations are now missing. Nevertheless, even now this seems like an good trade-off, and the optimization can be re-implemented in the future.