[PATCH v5 00/20] labgrid: Provide an integration with Labgrid

[Tom Rini]

On Wed, Aug 28, 2024 at 04:08:26PM -0600, Simon Glass wrote:

> Labgrid provides access to a hardware lab in an automated way. It is
> possible to boot U-Boot on boards in the lab without physically touching
> them. It relies on relays, USB UARTs and SD muxes, among other things.
> 
> By way of background, about 4 years ago I wrong a thing called Labman[1]
> which allowed my lab of about 30 devices to be operated remotely, using
> tbot for the console and build integration. While it worked OK and I
> used it for many bisects, I didn't take it any further.
> 
> It turns out that there was already an existing program, called Labgrid,
> which I did not know about at time (thank you Tom for telling me). It is
> more rounded than Labman and has a number of advantages:
> 
> - does not need udev rules, mostly
> - has several existing users who rely on it
> - supports multiple machines exporting their devices
> 
> It lacks a 'lab check' feature and a few other things, but these can be
> remedied.
> 
> On and off over the past several weeks I have been experimenting with
> Labgrid. I have managed to create an initial U-Boot integration (this
> series) by adding various features to Labgrid[2] and the U-Boot test
> hooks.
> 
> I hope that this might inspire others to set up boards and run tests
> automatically, rather than relying on infrequent, manual test. Perhaps
> it may even be possible to have a number of labs available.
> 
> Included in the integration are a number of simple scripts which make it
> easy to connect to boards and run tests:
> 
> ub-int <target>
>     Build and boot on a target, starting an interactive session
> 
> ub-cli <target>
>     Build and boot on a target, ensure U-Boot starts and provide an interactive
>     session from there
> 
> ub-smoke <target>
>     Smoke test U-Boot to check that it boots to a prompt on a target
> 
> ub-bisect <target>
>     Bisect a git tree to locate a failure on a particular target
> 
> ub-pyt <target> <testspec>
>     Run U-Boot pytests on a target
> 
> Some of these help to provide the same tbot[4] workflow which I have
> relied on for several years, albeit much simpler versions.
> 
> The goal here is to create some sort of script which can collect
> patches from the mailing list, apply them and test them on a selection
> of boards. I suspect that script already exists, so please let me know
> what you suggest.
> 
> I hope you find this interesting and take a look!
> 
> [1] https://github.com/sjg20/u-boot/tree/lab6a
> [2] https://github.com/labgrid-project/labgrid/pull/1411
> [3] https://github.com/sjg20/uboot-test-hooks/tree/labgrid
> [4] https://tbot.tools/index.html

Part of my concern here is still that this series contains:

- Generic test improvements and fixes:

>   test: Use a constant for the test timeout
>   test: Pass stderr to stdout
>   test: Avoid failing skipped tests
>   test: Move the receive code into a function
>   test: Separate out the exception handling
>   test: Detect dead connections
>   test: Tidy up remaining exceptions
>   test: Improve handling of sending commands
>   test: Fix mulptiplex_log typo
>   test: Avoid double echo when starting up

- Labgrid specific listed as generic changes:

>   test: Release board after tests complete
>   test: Try to shut down the lab console gracefully

- Your labgrid+builman implementation specific changes:

>   test: Allow signaling that U-Boot is ready
>   test: Allow connecting to a running board
>   test: Create a common function to get the config
>   test: Introduce the concept of a role
>   test: Introduce lab mode
>   test: Add a section for closing the connection
>   test: Support testing with two board-builds

Which makes this harder to review and clearly merge. For example, we
already have hooks for power on / power off, and we should just call
them as wow that would make everyones lab easier (I used to have a
pre-script to turn on everything).

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20240829/e220c062/attachment.sig>