[u-boot-test-hooks PATCH v4 3/3] Provide some basic scripts for Labgrid integration

[Michal Simek]

+Kevin,

On 8/29/24 00:27, Simon Glass wrote:
> Hi Tom,
> 
> On Wed, 28 Aug 2024 at 15:33, Tom Rini <trini at konsulko.com> wrote:
>>
>> On Wed, Aug 28, 2024 at 03:25:00PM -0600, Simon Glass wrote:
>>> Hi Tom,
>>>
>>> On Wed, 28 Aug 2024 at 14:19, Tom Rini <trini at konsulko.com> wrote:
>>>>
>>>> On Wed, Aug 28, 2024 at 10:45:23AM -0600, Simon Glass wrote:
>>>>
>>>>> With Labgrid we don't need to specify the various methods, except for
>>>>> the console, which simply calls labgrid-client.
>>>>>
>>>>> This allows supporting any boards in your lab, without adding per-board
>>>>> configuration to these hooks.
>>>>>
>>>>> Provide ellesmere files as an example.
>>>>>
>>>>> Signed-off-by: Simon Glass <sjg at chromium.org>
>>>>
>>>> I think maybe we call this version "labgrid-buildman" and make sure the
>>>> documentation emphasizes using buildman to provide U-Boot. This will
>>>> make it easier for me to provide a follow-up of say "labgrid-external"
>>>> that requires the binary to be provided, similar to our other hooks.
>>>
>>> It can be used in this way. The 'ub-pyt -B' does a bootstrap of the
>>> board without doing a build, if that helps to show the path through
>>> the code.
>>>
>>> If you don't pass --build to test.py then it won't do a build.
>>>
>>> If you like, you can pass --build-dir (and with the testb series
>>> --build-dir-extra) as you do now. The binary will be written to the
>>> board as is.
>>
>> Right, but this gets back to what I'm most confused about (and need to
>> talk with Kevin Hillman about maybe, but I missed my last meeting where
>> he was as well), you have a bunch of logic for integration of buildman
>> too, and I have a few wrappers around exiting labgrid-client commands.
>> And I don't know what direction upstream is going to want to take.
> 
> Hmm, who is Kevin?

Kevin was starting with kernel testing long time ago which become kernelci.


> 
> At the moment we are a long way from upstream. I have only one
> reviewed patch and it is pretty trivial. I just updated the stack
> again. I am hoping to meet some of the people in a few weeks, but they
> are pretty tied up on other projects, from my understanding.
> 
> Anyway, we could have a chat about it if you like. So far I'm pretty
> comfortable with the approach taken here. At this point it replaces my
> Labman as well as tbot for my use cases. Without integrated building,
> everything will be a PITA for me.
> 
> I am not sure this much matters on your side though, since you don't
> intend to use the building features here? How exactly are you building
> U-Boot? Is it using the --build option with pytest, or something else?
> 
> In the interim, one way of threading the needle would be to have
> test.py get the info from Labgrid using the 'query' and then run in
> the build in pytest. Or, even worse, have a separate yaml file with
> the required info and parse that in pytest and run buildman there...
> but as I say we are a long way from getting this integrated into
> Labgrid, so it is somewhat academic at present.
> 
> My immediate goal is to get all these things merged in U-Boot, so I
> can get my lab running properly with gitlab, etc. and ideally allow
> other custodians to push to it. Getting the integration landed in
> labgrid will likely come a lot later.

We are using these hooks script for u-boot testing for a while in AMD boardfarm.
It is not perfect but we find a way how to get it work with AMD boardfarm.

As is visible there is never going to be one boardfarm solution because it is 
hard to do switch from existing system to something else and move all 
infrastructure around.

Recently Linaro announced onelab [1] where the part of it is LAA. Pretty much 
ready to go box which you can connect your local HW to it and start to schedule 
jobs to it.

And I think this is a step which I am missing here. I have bunch of HW here and 
I know how to configure it, boot it, etc but I want to use it and I don't really 
want to take care about lab SW, maintain it, test it, etc.

It means I have no issue with Labgrind but I don't want to waste my time on 
making sure that Labgrind itself is working. I just want to configure my board 
connection once and then do the work not really updating Labgrind instances, 
deal with dependencies, etc.

It means having generic box, sw with connection to gitlab should get us to 
situation that u-boot will be tested on more HW and make it available for others.

Thanks,
Michal

[1] https://www.linaro.org/blog/onelab-launch-at-linaro-connect-madrid-2024/