[PATCH 00/42] labgrid: Provide an integration with Labgrid

[Tom Rini]

On Wed, Jun 12, 2024 at 09:08:32AM +0300, Ilias Apalodimas wrote:
> Hi Simon
> 
> On Tue, 11 Jun 2024 at 23:02, Simon Glass <sjg at chromium.org> wrote:
> >
> > Labgrid provides access to a hardware lab in an automated way. It is
> > possible to boot U-Boot on boards in the lab without physically touching
> > them. It relies on relays, USB UARTs and SD muxes, among other things.
> >
> > By way of background, about 4 years ago I wrong a thing called Labman[1]
> > which allowed my lab of about 30 devices to be operated remotely, using
> > tbot for the console and build integration. While it worked OK and I
> > used it for many bisects, I didn't take it any further.
> >
> > It turns out that there was already an existing program, called Labgrid,
> > which I did not know about at time (thank you Tom for telling me). It is
> > more rounded than Labman and has a number of advantages:
> >
> > - does not need udev rules, mostly
> > - has several existing users who rely on it
> > - supports multiple machines exporting their devices
> >
> > It lacks a 'lab check' feature and a few other things, but these can be
> > remedied.
> >
> > On and off over the past several weeks I have been experimenting with
> > Labgrid. I have managed to create an initial U-Boot integration (this
> > series) by adding various features to Labgrid[2] and the U-Boot test
> > hooks.
> >
> > I hope that this might inspire others to set up boards and run tests
> > automatically, rather than relying on infrequent, manual test. Perhaps
> > it may even be possible to have a number of labs available.
> >
> > Included in the integration are a number of simple scripts which make it
> > easy to connect to boards and run tests:
> >
> > ub-int <target>
> >     Build and boot on a target, starting an interactive session
> >
> > ub-cli <target>
> >     Build and boot on a target, ensure U-Boot starts and provide an interactive
> >     session from there
> >
> > ub-smoke <target>
> >     Smoke test U-Boot to check that it boots to a prompt on a target
> >
> > ub-bisect
> >     Bisect a git tree to locate a failure on a particular target
> >
> > ub-pyt <target> <testspec>
> >     Run U-Boot pytests on a target
> >
> > Some of these help to provide the same tbot[4] workflow which I have
> > relied on for several years, albeit much simpler versions.
> >
> > The goal here is to create some sort of script which can collect
> > patches from the mailing list, apply them and test them on a selection
> > of boards. I suspect that script already exists, so please let me know
> > what you suggest.
> >
> > I hope you find this interesting and take a look!
> 
> Thanks this is interesting!
> I only got cc'ed on the cover letter and I'll slowly have a look on
> the rest. A naive question -- I saw you did integrate this on gitlab
> with your internal lab. How secure is this? Could we schedule weekly
> builds that run on various remote labs and get results on actual
> hardware? Or do we have to rely on users for that ?

That's where this certainly gets a little tricky. Ideally, and part of
why I've been hoping to get Labgrid going with our pytest suite, since
it's good enough for kernelci (and people to be reasonably confident
about allowing not-exactly-random-user-access), it should be good enough
for us too. So my long term hope is that we can:
- Have Labgrid based labs + something know to monitor trees+branches at
  X location (like I know today some people poll and test my WIP
  branches).
- Have it be reasonably clear that if you maintain a lab for kernelci,
  you can add testing U-Boot in too.

In the medium term, I am hopeful we can at least expose this to all
custodian trees on source.denx.de. And I'm saying "we" here as I have
two much smaller labs also going, one with Labgrid, and both a much
smaller set of targets currently.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20240612/cd5bc78d/attachment.sig>