[PATCH] test/py: efi_capsule: Handle expected reset after capsule on disk

[Heinrich Schuchardt]

On 2/17/22 18:55, Simon Glass wrote:
> Hi Masami,
>
> On Wed, 16 Feb 2022 at 18:11, Masami Hiramatsu
> <masami.hiramatsu at linaro.org> wrote:
>>
>> Hi Simon,
>>
>> Let me confirm your point.
>> So are you concerning the 'real' reset for the capsule update test
>> case itself or this patch?
>>
>> I'm actually learning how the test is working, so please help me to
>> understand how I can solve it.
>>
>> There are 3 environments to run the test, sandbox, Qemu, and a real board.
>> If we reset a sandbox, it will continue to run (just restart itself),
>
> Here you should be able to avoid doing a reset. See
> dm_test_sysreset_base() which tests sysreset drivers on sandbox.
>
>> but Qemu and real board will cause a real reset and it will terminate
>> the qemu or stop the board (depends on how it is implemented). Thus,
>> if a command or boot process will cause a reset, it will need a
>> special care (maybe respawn?).
>
> Here you need to worry about the surrounding automation logic which
> could be tbot of the U-Boot pytest hooks. I suggest you avoid this and
> handle it some other way, without reset.
>
>>
>> Since the capsule update testcase only runs on sandbox, it will not
>> cause real reset. But maybe it is possible to support running on Qemu.
>
> Maybe, but I don't think you should worry about that, at least for
> now. The sandbox test is enough.
>
>>
>> Current my test patch (and capsule update testcase itself) doesn't
>> handle the real reset case correctly even on Qemu. The Qemu needs
>> spawn a new instance and re-connect the console when the reset
>> happens.

Respawning of QEMU instances after a reset is handled by the scripts in
https://source.denx.de/u-boot/u-boot-test-hooks.git on Gitlab CI. For my
local tests I have used:
https://github.com/xypron/u-boot-build/tree/qemu-arm64/u-boot-test

If you want to set up an environment for a real board, you have to write
your own python scripts and make them available over PYTHONPATH. For an
example see
https://github.com/xypron/u-boot-build/tree/orangepi-pc/u-boot-test

>
> Indeed.
>
>>
>> If so, I think there are 2 issues to be solved.
>> 1. change the capsule update testcase runable on Qemu
>> 2. change my patch to handle the real reset correctly (not only
>> waiting for the next boot, but also respawn it again)
>>
>> Do I understand correctly?
>
> I think the best approach is to get your test running on sandbox, with
> the faked reset. Don't worry about the other cases as we don't support
> them.

Why fake a reset? Just call the normal reset code. Avoid sandbox quirks.
We want the sandbox to just run through the same code as any other board
to get meaningful test results.

Best regards

Heinrich

>
> Regards,
> Simon
>
>
>>
>> Thank you,
>>
>> 2022年2月17日(木) 2:53 Simon Glass <sjg at chromium.org>:
>>>
>>> Hi Heinrich,
>>>
>>> On Wed, 16 Feb 2022 at 10:50, Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>>>>
>>>> On 2/16/22 16:46, Tom Rini wrote:
>>>>> On Wed, Feb 16, 2022 at 04:32:40PM +0100, Heinrich Schuchardt wrote:
>>>>>> On 2/16/22 16:26, Simon Glass wrote:
>>>>>>> Hi Masami,
>>>>>>>
>>>>>>> On Tue, 15 Feb 2022 at 02:05, Masami Hiramatsu
>>>>>>> <masami.hiramatsu at linaro.org> wrote:
>>>>>>>>
>>>>>>>> Since now the capsule_on_disk will restart the u-boot sandbox right
>>>>>>>> after the capsule update, if CONFIG_EFI_CAPSULE_ON_DISK_EARLY=y, the
>>>>>>>> boot with a new capsule file will repeat reboot sequence. On the
>>>>>>>> other hand, if CONFIG_EFI_CAPSULE_ON_DISK_EARLY=n, the 'env print -e'
>>>>>>>> command will execute the capsule update on disk and reboot.
>>>>>>>>
>>>>>>>> Thus this update the uboot_console for those 2 cases;
>>>>>>>>
>>>>>>>>     - restart_uboot(): Add expect_earlyreset optional parameter so that
>>>>>>>>       it can handle the reboot while booting.
>>>>>>>>     - run_command(): Add wait_for_reboot optional parameter so that it
>>>>>>>>       can handle the reboot after executing a command.
>>>>>>>>
>>>>>>>> And enable those options in the test_capsule_firmware.py test cases.
>>>>>>>>
>>>>>>>> Signed-off-by: Masami Hiramatsu <masami.hiramatsu at linaro.org>
>>>>>>>> ---
>>>>>>>>     .../test_efi_capsule/test_capsule_firmware.py      |   39 ++++++--
>>>>>>>>     test/py/u_boot_console_base.py                     |   95 +++++++++++++++-----
>>>>>>>>     test/py/u_boot_console_sandbox.py                  |    6 +
>>>>>>>>     3 files changed, 102 insertions(+), 38 deletions(-)
>>>>>>>
>>>>>>> We have a means to avoid actually doing the reset, see the reset driver.
>>>>>>
>>>>>> The UEFI specification requires a cold reset after a capsule is updated
>>>>>> and before the console is reached. How could the reset driver help to
>>>>>> fix the Python tests?
>>>>>
>>>>> Is this test going to be able to run on qemu, sandbox, real hardware, or
>>>>> all 3?  The tests may well end up having to know a bit more, sadly,
>>>>> about the type of system they're testing.
>>>>>
>>>> Currently the test will only run on the sandbox in Gitlab (see usage of
>>>> @pytest.mark.boardspec('sandbox') in test/py/tests/test_efi_capsule/).
>>>
>>> Let me know if you need help reworking this patch to operate on
>>> sandbox without a 'real' reset.
>>>
>>> Regards,
>>> Simon
>>
>>
>>
>> --
>> Masami Hiramatsu