[PATCH v7 01/11] test: Allow signaling that U-Boot is ready

[Tom Rini]

On Thu, Oct 31, 2024 at 07:00:10PM +0100, Simon Glass wrote:
> Hi Tom,
> 
> On Wed, 30 Oct 2024 at 00:53, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Wed, Oct 09, 2024 at 07:51:35PM -0600, Simon Glass wrote:
> >
> > > When Labgrid is used, it can get U-Boot ready for running tests. It
> > > prints a message when it has done so.
> > >
> > > Add logic to detect this message and accept it.
> > >
> > > Note that this does not change pytest, which still (also) looks for the
> > > U-Boot banner. This change merely makes it possible for pytest to
> > > believe Labgrid when it says that the board is ready for use.
> > >
> > > In several cases, the board starts up and Labgrid receives some initial
> > > output, then pytest starts and misses some of that output, because it
> > > came in while Labgrid had the console open. Then pytest fails because
> > > it doesn't see the expected banners.
> > >
> > > With this change, Labgrid handles getting U-Boot to a prompt, in a
> > > fully reliable manner. Then pytest starts up and can simply start
> > > running its tests.
> > >
> > > But, again, this does not prevent pytest from handling a banner if one
> > > is provided (e.g. if not using the Labgrid integration).
> > >
> > > Signed-off-by: Simon Glass <sjg at chromium.org>
> >
> > I _may_ see this myself sometimes, but I'm not sure.
> 
> OK, but it isn't really necessary for you to see it yourself. I've
> described above the repeatable problems I have with this approach.

Well, with what I just now said in reply to v6 of this series:

Reviewed-by: Tom Rini <trini at konsulko.com>

So long as v8 also includes a much longer commit message.

> > But I'm not sure
> > where the bug would be between us and labgrid. Looking at
> > test/py/u_boot_console_exec_attach.py we grab the console and then reset
> > the target. If the spawn hasn't completed, especially before the power
> > cycle has completed (which in labgrid defaults to 2 seconds) we perhaps
> > need our own delay. But as I'll point out further on, NOT counting the
> > banners means we aren't able to verify other functionality. Even if they
> > are tests that I think you don't like because they verify the
> > functionality of resetting the device.
> 
> Yes, that's a use case that needs special support. If a board is
> booted from USB, then resets, it will run whatever it can find,
> perhaps just hang. Labgrid doesn't know the board is resetting and
> cannot therefore ensure that it sends the firmware over USB a second
> time (or sets the recovery line on the board). Resetting a board works
> OK using SDwire and some other cases, but will fail on quite a few of
> my boards.

It is indeed tricky matching up and catching everything given the number
of platforms where it's technically possible to do CI on the firmware
but practically a large complicated pain.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20241031/e85db8ff/attachment.sig>