[PATCH 00/42] labgrid: Provide an integration with Labgrid

[Simon Glass]

Hi Tom,

On Wed, 12 Jun 2024 at 10:03, Tom Rini <trini at konsulko.com> wrote:
>
> On Wed, Jun 12, 2024 at 09:08:32AM +0300, Ilias Apalodimas wrote:
> > Hi Simon
> >
> > On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg at chromium.org> wrote:
> > >
> > > Labgrid provides access to a hardware lab in an automated way. It is
> > > possible to boot U-Boot on boards in the lab without physically touching
> > > them. It relies on relays, USB UARTs and SD muxes, among other things.
> > >
> > > By way of background, about 4 years ago I wrong a thing called Labman[1]
> > > which allowed my lab of about 30 devices to be operated remotely, using
> > > tbot for the console and build integration. While it worked OK and I
> > > used it for many bisects, I didn't take it any further.
> > >
> > > It turns out that there was already an existing program, called Labgrid,
> > > which I did not know about at time (thank you Tom for telling me). It is
> > > more rounded than Labman and has a number of advantages:
> > >
> > > - does not need udev rules, mostly
> > > - has several existing users who rely on it
> > > - supports multiple machines exporting their devices
> > >
> > > It lacks a 'lab check' feature and a few other things, but these can be
> > > remedied.
> > >
> > > On and off over the past several weeks I have been experimenting with
> > > Labgrid. I have managed to create an initial U-Boot integration (this
> > > series) by adding various features to Labgrid[2] and the U-Boot test
> > > hooks.
> > >
> > > I hope that this might inspire others to set up boards and run tests
> > > automatically, rather than relying on infrequent, manual test. Perhaps
> > > it may even be possible to have a number of labs available.
> > >
> > > Included in the integration are a number of simple scripts which make it
> > > easy to connect to boards and run tests:
> > >
> > > ub-int <target>
> > >     Build and boot on a target, starting an interactive session
> > >
> > > ub-cli <target>
> > >     Build and boot on a target, ensure U-Boot starts and provide an interactive
> > >     session from there
> > >
> > > ub-smoke <target>
> > >     Smoke test U-Boot to check that it boots to a prompt on a target
> > >
> > > ub-bisect
> > >     Bisect a git tree to locate a failure on a particular target
> > >
> > > ub-pyt <target> <testspec>
> > >     Run U-Boot pytests on a target
> > >
> > > Some of these help to provide the same tbot[4] workflow which I have
> > > relied on for several years, albeit much simpler versions.
> > >
> > > The goal here is to create some sort of script which can collect
> > > patches from the mailing list, apply them and test them on a selection
> > > of boards. I suspect that script already exists, so please let me know
> > > what you suggest.
> > >
> > > I hope you find this interesting and take a look!
> >
> > Thanks this is interesting!
> > I only got cc'ed on the cover letter and I'll slowly have a look on
> > the rest. A naive question -- I saw you did integrate this on gitlab
> > with your internal lab. How secure is this? Could we schedule weekly
> > builds that run on various remote labs and get results on actual
> > hardware? Or do we have to rely on users for that ?
>
> That's where this certainly gets a little tricky. Ideally, and part of
> why I've been hoping to get Labgrid going with our pytest suite, since
> it's good enough for kernelci (and people to be reasonably confident
> about allowing not-exactly-random-user-access), it should be good enough
> for us too. So my long term hope is that we can:
> - Have Labgrid based labs + something know to monitor trees+branches at
>   X location (like I know today some people poll and test my WIP
>   branches).
> - Have it be reasonably clear that if you maintain a lab for kernelci,
>   you can add testing U-Boot in too.
>
> In the medium term, I am hopeful we can at least expose this to all
> custodian trees on source.denx.de. And I'm saying "we" here as I have
> two much smaller labs also going, one with Labgrid, and both a much
> smaller set of targets currently.

That sounds good to me.

Regards,
Simon