Two jobs at once on denx-vulcan?

[Harald Seiler]

Hi Simon,

On Mon, 2021-09-20 at 08:06 -0600, Simon Glass wrote:
> Hi Harald,
> 
> On Mon, 20 Sept 2021 at 02:12, Harald Seiler <hws at denx.de> wrote:
> > 
> > Hi,
> > 
> > On Sat, 2021-09-18 at 10:37 -0600, Simon Glass wrote:
> > > Hi,
> > > 
> > > Is there something screwy with this? It seems that denx-vulcan does
> > > two builds at once?
> > > 
> > > https://source.denx.de/u-boot/custodians/u-boot-dm/-/jobs/323540
> > 
> > Hm, I did some changes to the vulcan runner which might have caused
> > this... But still, even if it is running multiple jobs in parallel, they
> > should still be isolated, so how does this lead to a build failure?
> 
> I'm not sure that it does, but I do see this at the above link:
> 
> Error: Unable to create
> '/builds/u-boot/custodians/u-boot-dm/.git/logs/HEAD.lock': File
> exists.

This is super strange... Each build should be running in its own
container so there should never be a way for such a race to occur.  No
clue what is going on here...

> Re doing multiple builds, have you set it up so it doesn't take on the
> very large builds? I would love to enable multiple builds for the qemu
> steps since they mostly use a single CPU, but am not sure how to do
> it.

Actually, this was more a mistake than an intentional change.  I updated
the runner on vulcan to also take jobs for some other repos and wanted
those jobs to run in parallel.  It looks like I just forgot setting the
`limit = 1` option for the U-Boot runner.

Now, I think doing what you suggest is possible.  We need to tag build
and "test" jobs differently and then define multiple runners with
different limits.  E.g. in `.gitlab-ci.yml`:

	build all 32bit ARM platforms:
	  stage: world build
	  tags:
	    - build

	cppcheck:
	  stage: testsuites
	  tags:
	    - test

And then define two runners in `/etc/gitlab-runner/config.toml`:

	concurrent = 4

	[[runners]]
	  name = "u-boot builder on vulcan"
	  limit = 1
	  ...

	[[runners]]
	  name = "u-boot tester on vulcan"
	  limit = 4
	  ...

and during registration they get the `build` and `test` tags
respectively.  This would allow running (in this example) up to 4 test
jobs concurrently, but only ever one large build job at once.

-- 
Harald

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws at denx.de