[u-boot-test-hooks PATCH v4 3/3] Provide some basic scripts for Labgrid integration

[Simon Glass]

Hi Peter,

On Thu, 29 Aug 2024 at 04:43, Peter Robinson <pbrobinson at gmail.com> wrote:
>
> On Wed, 28 Aug 2024 at 22:25, Simon Glass <sjg at chromium.org> wrote:
> >
> > Hi Peter,
> >
> > On Wed, 28 Aug 2024 at 12:14, Peter Robinson <pbrobinson at gmail.com> wrote:
> > >
> > > Hi Simon,
> > >
> > > > With Labgrid we don't need to specify the various methods, except for
> > > > the console, which simply calls labgrid-client.
> > > >
> > > > This allows supporting any boards in your lab, without adding per-board
> > > > configuration to these hooks.
> > > >
> > > > Provide ellesmere files as an example.
> > >
> > > What's ellesmere?
> >
> > It is a lake but also the name of a computer.
> >
> > >
> > > > Signed-off-by: Simon Glass <sjg at chromium.org>
> > > > ---
> > > >
> > > > Changes in v4:
> > > > - Support pytest fully with dual-build boards like Beagleplay
> > > >
> > > > Changes in v3:
> > > > - Update scripts for latest version of Labgrid integration
> > > > - Add poweroff.none and poweron.none
> > > > - Provide -n flag when querying board info
> > > > - Target the grpc version of Labgrid which is now in -master
> > > > - Update README to cover the changes
> > > >
> > > > Changes in v2:
> > > > - Make use of the common script (only) to set bin_dir
> > > >
> > > >  README.md                    | 50 ++++++++++++++++++++++++++++++++++++
> > >
> > > Maybe that should be in a separate labsgrid readme?
> >
> > My hope is that Labgrid becomes the normal way of running these tests,
>
> Generally I agree with automated testing platforms, I think <insert
> you're preferred platform here> makes sense, there's a bunch like
> Linaro and a bunch in the arm ecosystem use LAVA [1] and  there's a
> bunch more that I'm aware of.

I am somewhat familiar with LAVA and I believe it can be used to test
U-Boot, although I need to learn how. Looking at a test run [2] for
beaglebone black I see that it is using a recent kernel but the U-Boot
seems to be older.

>
> Does it make sense, of course at some point in the future post this
> being merged, make sense to look at a general way of making it easier
> to plugin these sort of HW test platforms using this as a basis? I ask
> mostly because putting a bunch of my devices into some sort of
> platform can auto test things and of course everyone has an opinion to
> which is the best one :-P

Yes. I had heard from Tom that Labgrid is the new hotness for now.
Having dug into it I believe it is a good solution, although it can
certainly be improved to handle scale better.

Anyway, IMO the current test hooks are not a great solution, just
because the configuration is spread all over the place and it relies
on lots of little shell scripts. So I believe that the Labgrid
integration is a closer to where we want to be with others that come
along.

I would love for you to add a board into Labgrid and see how you go.

>
> [1] https://validation.linaro.org/
>
> > so having it in the main place (and perhaps eventually removing the
> > old way) is my goal.
> >
> > >
> > > >  bin/console.labgrid          | 42 ++++++++++++++++++++++++++++++
> > > >  bin/ellesmere/common-labgrid | 46 +++++++++++++++++++++++++++++++++
> > > >  bin/ellesmere/conf.all       | 24 +++++++++++++++++
> > > >  bin/getrole.labgrid          | 25 ++++++++++++++++++
> > > >  bin/release.labgrid          | 22 ++++++++++++++++
> > > >  bin/release.none             | 22 ++++++++++++++++
> > > >  bin/u-boot-test-getrole      | 38 +++++++++++++++++++++++++++
> > > >  bin/u-boot-test-release      | 26 +++++++++++++++++++
> > > >  9 files changed, 295 insertions(+)
> > > >  create mode 100644 bin/console.labgrid
> > > >  create mode 100755 bin/ellesmere/common-labgrid
> > > >  create mode 100644 bin/ellesmere/conf.all
> > > >  create mode 100644 bin/getrole.labgrid
> > > >  create mode 100644 bin/release.labgrid
> > > >  create mode 100644 bin/release.none
> > > >  create mode 100755 bin/u-boot-test-getrole
> > > >  create mode 100755 bin/u-boot-test-release
> > > >
> >
> > [..]
> >
> > Regards,
> > Simon

Regards,
Simon

[2] https://validation.linaro.org/scheduler/job/4089871#results_482796468