[PATCH v2 26/28] test/py: Allow tests to be filtered by role

[Simon Glass]

Hi Tom,

On Tue, 18 Feb 2025 at 08:12, Tom Rini <trini at konsulko.com> wrote:
>
> On Tue, Feb 18, 2025 at 05:09:06AM -0700, Simon Glass wrote:
> > Hi Tom,
> >
> > On Mon, 17 Feb 2025 at 10:48, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Sun, Feb 16, 2025 at 01:44:11PM -0700, Simon Glass wrote:
> > >
> > > > Some test can only be run by a particular board in a lab, e.g. because
> > > > they are loaded with an OS image used by the test. Add a way to specify
> > > > this in tests.
> > > >
> > > > Signed-off-by: Simon Glass <sjg at chromium.org>
> > > > ---
> > > >
> > > > Changes in v2:
> > > > - Add new patch to allow tests to be filtered by role
> > > >
> > > >  test/py/conftest.py | 22 ++++++++++++++++++++++
> > > >  test/py/pytest.ini  |  1 +
> > > >  2 files changed, 23 insertions(+)
> > >
> > > This seems duplicative of the existing functionality we have, all
> > > hardware tests can only be run on platforms which configure what the
> > > test requires, in the boardenv file.
> >
> > I was afraid you would say that, and yes you are right, it is duplicative.
> >
> > With my lab I am trying to use Labgrid's configuration and not need
> > anything else. For example, the params on how to launch QEMU and where
> > to find the OS file are there, rather than in u-boot-test-hooks. It
> > allows me to run a session interactively just as easily (and with the
> > same flow) as I run the pytests and gitlab.
> >
> > In my lab, I actually don't have any Python things, apart from what I
> > added for that tftp test.
> >
> > I wonder if I could create a way to pass the env__ stuff in from Labgrid?
>
> Well, I was also afraid that was going to be part of your answer. A
> challenge I see with the way you're using labgrid is that details are
> kept there, instead. Is there some way to publish your configs so
> they're also available for reference? I indeed have been lax in doing so
> for the way I use it and so posted:
> https://patchwork.ozlabs.org/project/uboot/patch/20250217200638.356583-1-trini@konsulko.com/
> yesterday.

Yes, that's great. It is a big help to have all the lab stuff in one
place and not being able to get things into that repo was one reason I
gave up with my local lab that used the hook scripts. Just too many
local branches and hacks, and too hard.

Anyway, yes I'd like to get it in u-boot-test-hooks. I last pushesd it
to Labgrid 7 months ago [1]. I thought I did it at the end of November
but must have imagined it. Perhaps I could put the config files, etc.
into u-boot-test somewhere, like in a labgrid/sjg directory? Could you
give me access to that repo?

>
> But then also yes, I think you need to figure out how to have boardenv
> files so that you can have pytest tests that read from the config that
> they expect.

I can have pytest read the config from Labgrid, like it does today.
Needs some thought though.

Regards,
Simon

[1] https://github.com/labgrid-project/labgrid/pull/1411