[PATCH v7 00/11] labgrid: Provide an integration with Labgrid

[Simon Glass]

Hi Tom,

On Mon, 21 Oct 2024 at 20:07, Tom Rini <trini at konsulko.com> wrote:
>
> On Mon, Oct 21, 2024 at 01:12:49PM +0200, Simon Glass wrote:
> > Hi Tom,
> >
> > On Thu, 10 Oct 2024 at 03:51, Simon Glass <sjg at chromium.org> wrote:
> > >
> > > Labgrid provides access to a hardware lab in an automated way. It is
> > > possible to boot U-Boot on boards in the lab without physically touching
> > > them. It relies on relays, USB UARTs and SD muxes, among other things.
> > >
> > > By way of background, about 4 years ago I wrong a thing called Labman[1]
> > > which allowed my lab of about 30 devices to be operated remotely, using
> > > tbot for the console and build integration. While it worked OK and I
> > > used it for many bisects, I didn't take it any further.
> > >
> > > It turns out that there was already an existing program, called Labgrid,
> > > which I did not know about at time (thank you Tom for telling me). It is
> > > more rounded than Labman and has a number of advantages:
> > >
> > > - does not need udev rules, mostly
> > > - has several existing users who rely on it
> > > - supports multiple machines exporting their devices
> > >
> > > It lacks a 'lab check' feature and a few other things, but these can be
> > > remedied.
> > >
> > > On and off over the past several weeks I have been experimenting with
> > > Labgrid. I have managed to create an initial U-Boot integration (this
> > > series) by adding various features to Labgrid[2] and the U-Boot test
> > > hooks.
> > >
> > > I hope that this might inspire others to set up boards and run tests
> > > automatically, rather than relying on infrequent, manual test. Perhaps
> > > it may even be possible to have a number of labs available.
> > >
> > > Included in the integration are a number of simple scripts which make it
> > > easy to connect to boards and run tests:
> > >
> > > ub-int <target>
> > >     Build and boot on a target, starting an interactive session
> > >
> > > ub-cli <target>
> > >     Build and boot on a target, ensure U-Boot starts and provide an interactive
> > >     session from there
> > >
> > > ub-smoke <target>
> > >     Smoke test U-Boot to check that it boots to a prompt on a target
> > >
> > > ub-bisect <target>
> > >     Bisect a git tree to locate a failure on a particular target
> > >
> > > ub-pyt <target> <testspec>
> > >     Run U-Boot pytests on a target
> > >
> > > Some of these help to provide the same tbot[4] workflow which I have
> > > relied on for several years, albeit much simpler versions.
> > >
> > > The goal here is to create some sort of script which can collect
> > > patches from the mailing list, apply them and test them on a selection
> > > of boards. I suspect that script already exists, so please let me know
> > > what you suggest.
> > >
> > > I hope you find this interesting and take a look!
> > >
> > > [1] https://github.com/sjg20/u-boot/tree/lab6a
> > > [2] https://github.com/labgrid-project/labgrid/pull/1411
> > > [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
> > > [4] https://tbot.tools/index.html
> > >
> > > Changes in v7:
> > > - Rebase on testb series
> > > - Expand commit message
> > > - Rename option to --use-running-system
> > > - Rebase on broken-out testb series
> > >
> > > Changes in v6:
> > > - Rebase without an earlier patch
> > > - Avoid doing the special shutdown unless USE_LABGRID is enabled
> > > - Drop patch 'test: Pass stderr to stdout'
> > >
> > > Changes in v5:
> > > - Add a few more comments
> > > - Comment out the debugging, which might be useful later
> > > - Add a patch to support testing with two board-builds
> > >
> > > Changes in v3:
> > > - Split out most patches into two new series and update cover letter
> > >
> > > Changes in v2:
> > > - Only disable echo if a terminal is detected
> > > - Avoid running a docker image for skipped lab tests
> > >
> > > Simon Glass (11):
> > >   test: Allow signaling that U-Boot is ready
> > >   test: Release board after tests complete
> > >   test: Allow connecting to a running board
> > >   test: Introduce the concept of a role
> > >   test: Introduce lab mode
> > >   test: Improve handling of sending commands
> > >   test: Avoid double echo when starting up
> > >   test: Try to shut down the lab console gracefully
> > >   test: Add a section for closing the connection
> > >   test: Support testing with two board-builds
> > >   CI: Allow running tests on sjg lab
> > >
> > >  .gitlab-ci.yml                        | 153 ++++++++++++++++++++++++++
> > >  test/py/conftest.py                   |  73 ++++++++++--
> > >  test/py/u_boot_console_base.py        | 123 +++++++++++++++------
> > >  test/py/u_boot_console_exec_attach.py |  31 ++++--
> > >  test/py/u_boot_spawn.py               |  36 +++++-
> > >  5 files changed, 363 insertions(+), 53 deletions(-)
> > >
> > > --
> > > 2.34.1
> > >
> >
> > Is there any news on these patches? I would dearly love to enable my
> > lab and this is what is needed...
>
> As our approaches are radically different I continue to be looking for
> feedback from a third person to set things up and say what does /
> doesn't work for them and their lab. I assume you're just pulling this
> in locally for use at least.

No, I cannot pull this in locally. I would like to set up gitlab so I
(and others) can push things to it for running on the lab. Without my
patches the lab only works on a few boards.

Could you perhaps try them out on your lab, to check they don't break anything?

Regards,
Simon