[PATCH] CI: Add gitlab tags to builds which can use all CPUs

[Tom Rini]

On Thu, Nov 28, 2024 at 09:00:10AM -0700, Simon Glass wrote:

> Most gitlab runners are mostly idle at present and could happily run
> several jobs at once.
> 
> There are a few exceptions, such as the world builds.
> 
> Add a 'single' tag for the exceptions so that they are dealt with
> separately, with no concurrency. This is not quite perfect, since it
> means that a 'single' job can be combined with a number of normal jobs,
> but in practice this doesn't seem to matter.
> 
> To use this, for each existing machine, add a new, separate runner, with
> the tag 'single'. Set 'limit = 1' in its config.toml file and set
> 'concurrent = 10' (for example) in the global section. The machine will
> then pick up one 'single' job and up to 9 other jobs at once.
> 
> In my testing this reduces the time to complete a CI pipeline.
> 
> My choice of which jobs to mark as single is mostly down to how much CPU
> they need. For the 'docs' job, it runs for a short time, so doesn't seem
> worth marking it as a 'single' job.
> 
> Signed-off-by: Simon Glass <sjg at chromium.org>

This is an interesting approach. What will work better long term is
adding more runners with the "all" tag because as part of the rework of
the Gitlab job I've done and you can see here:
https://source.denx.de/u-boot/u-boot/-/pipelines/23588
(but isn't ready to post as Ilias wants to look in to the sandbox+arm64
issue) means that every runner needs some tags and for runners which can
handle multiple "small" jobs at once (as the first two stages are
building a board and then testing, or similar) we can make use of that,
and then restrict the world builds to the fastest nodes only.
This also means that things like the Oracle "always free" tier cloud
instances could be used for example as an "all" jobs node, I suspect.
And the same for any other cloud provider with an equivalent tier.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20241128/26006874/attachment.sig>