[PATCH] test/py: efi_capsule: Handle expected reset after capsule on disk

[Masami Hiramatsu]

Hi Heinrich,

2022年2月18日(金) 22:59 Heinrich Schuchardt <xypron.glpk at gmx.de>:
>
> On 2/17/22 18:55, Simon Glass wrote:
> > Hi Masami,
> >
> > On Wed, 16 Feb 2022 at 18:11, Masami Hiramatsu
> > <masami.hiramatsu at linaro.org> wrote:
> >>
> >> Hi Simon,
> >>
> >> Let me confirm your point.
> >> So are you concerning the 'real' reset for the capsule update test
> >> case itself or this patch?
> >>
> >> I'm actually learning how the test is working, so please help me to
> >> understand how I can solve it.
> >>
> >> There are 3 environments to run the test, sandbox, Qemu, and a real board.
> >> If we reset a sandbox, it will continue to run (just restart itself),
> >
> > Here you should be able to avoid doing a reset. See
> > dm_test_sysreset_base() which tests sysreset drivers on sandbox.
> >
> >> but Qemu and real board will cause a real reset and it will terminate
> >> the qemu or stop the board (depends on how it is implemented). Thus,
> >> if a command or boot process will cause a reset, it will need a
> >> special care (maybe respawn?).
> >
> > Here you need to worry about the surrounding automation logic which
> > could be tbot of the U-Boot pytest hooks. I suggest you avoid this and
> > handle it some other way, without reset.
> >
> >>
> >> Since the capsule update testcase only runs on sandbox, it will not
> >> cause real reset. But maybe it is possible to support running on Qemu.
> >
> > Maybe, but I don't think you should worry about that, at least for
> > now. The sandbox test is enough.
> >
> >>
> >> Current my test patch (and capsule update testcase itself) doesn't
> >> handle the real reset case correctly even on Qemu. The Qemu needs
> >> spawn a new instance and re-connect the console when the reset
> >> happens.
>
> Respawning of QEMU instances after a reset is handled by the scripts in
> https://source.denx.de/u-boot/u-boot-test-hooks.git on Gitlab CI. For my
> local tests I have used:
> https://github.com/xypron/u-boot-build/tree/qemu-arm64/u-boot-test
>
> If you want to set up an environment for a real board, you have to write
> your own python scripts and make them available over PYTHONPATH. For an
> example see
> https://github.com/xypron/u-boot-build/tree/orangepi-pc/u-boot-test

Great! thank you for the information.
I actually would like to confirm that my change on ConsoleBase class
will work on non-sandbox, or it should be ConsoleSandbox class
method.

>
> >
> > Indeed.
> >
> >>
> >> If so, I think there are 2 issues to be solved.
> >> 1. change the capsule update testcase runable on Qemu
> >> 2. change my patch to handle the real reset correctly (not only
> >> waiting for the next boot, but also respawn it again)
> >>
> >> Do I understand correctly?
> >
> > I think the best approach is to get your test running on sandbox, with
> > the faked reset. Don't worry about the other cases as we don't support
> > them.
>
> Why fake a reset? Just call the normal reset code. Avoid sandbox quirks.
> We want the sandbox to just run through the same code as any other board
> to get meaningful test results.

OK. I got it.

Thank you,

>
> Best regards
>
> Heinrich
>
> >
> > Regards,
> > Simon
> >
> >
> >>
> >> Thank you,
> >>
> >> 2022年2月17日(木) 2:53 Simon Glass <sjg at chromium.org>:
> >>>
> >>> Hi Heinrich,
> >>>
> >>> On Wed, 16 Feb 2022 at 10:50, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> >>>>
> >>>> On 2/16/22 16:46, Tom Rini wrote:
> >>>>> On Wed, Feb 16, 2022 at 04:32:40PM +0100, Heinrich Schuchardt wrote:
> >>>>>> On 2/16/22 16:26, Simon Glass wrote:
> >>>>>>> Hi Masami,
> >>>>>>>
> >>>>>>> On Tue, 15 Feb 2022 at 02:05, Masami Hiramatsu
> >>>>>>> <masami.hiramatsu at linaro.org> wrote:
> >>>>>>>>
> >>>>>>>> Since now the capsule_on_disk will restart the u-boot sandbox right
> >>>>>>>> after the capsule update, if CONFIG_EFI_CAPSULE_ON_DISK_EARLY=y, the
> >>>>>>>> boot with a new capsule file will repeat reboot sequence. On the
> >>>>>>>> other hand, if CONFIG_EFI_CAPSULE_ON_DISK_EARLY=n, the 'env print -e'
> >>>>>>>> command will execute the capsule update on disk and reboot.
> >>>>>>>>
> >>>>>>>> Thus this update the uboot_console for those 2 cases;
> >>>>>>>>
> >>>>>>>>     - restart_uboot(): Add expect_earlyreset optional parameter so that
> >>>>>>>>       it can handle the reboot while booting.
> >>>>>>>>     - run_command(): Add wait_for_reboot optional parameter so that it
> >>>>>>>>       can handle the reboot after executing a command.
> >>>>>>>>
> >>>>>>>> And enable those options in the test_capsule_firmware.py test cases.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Masami Hiramatsu <masami.hiramatsu at linaro.org>
> >>>>>>>> ---
> >>>>>>>>     .../test_efi_capsule/test_capsule_firmware.py      |   39 ++++++--
> >>>>>>>>     test/py/u_boot_console_base.py                     |   95 +++++++++++++++-----
> >>>>>>>>     test/py/u_boot_console_sandbox.py                  |    6 +
> >>>>>>>>     3 files changed, 102 insertions(+), 38 deletions(-)
> >>>>>>>
> >>>>>>> We have a means to avoid actually doing the reset, see the reset driver.
> >>>>>>
> >>>>>> The UEFI specification requires a cold reset after a capsule is updated
> >>>>>> and before the console is reached. How could the reset driver help to
> >>>>>> fix the Python tests?
> >>>>>
> >>>>> Is this test going to be able to run on qemu, sandbox, real hardware, or
> >>>>> all 3?  The tests may well end up having to know a bit more, sadly,
> >>>>> about the type of system they're testing.
> >>>>>
> >>>> Currently the test will only run on the sandbox in Gitlab (see usage of
> >>>> @pytest.mark.boardspec('sandbox') in test/py/tests/test_efi_capsule/).
> >>>
> >>> Let me know if you need help reworking this patch to operate on
> >>> sandbox without a 'real' reset.
> >>>
> >>> Regards,
> >>> Simon
> >>
> >>
> >>
> >> --
> >> Masami Hiramatsu
>


-- 
Masami Hiramatsu