[PATCH 15/17] pytest: Collect SPL unit tests

[Stephen Warren]

On 10/5/20 1:51 PM, Simon Glass wrote:
> Hi Stephen,
> 
> On Mon, 5 Oct 2020 at 13:39, Stephen Warren <swarren at wwwdotorg.org> wrote:
>>
>> On 10/3/20 9:25 AM, Simon Glass wrote:
>>> Add a new test_spl fixture to handle running SPL unit tests.
>>
>>> diff --git a/test/py/conftest.py b/test/py/conftest.py
>>
>>> @@ -317,10 +318,13 @@ def pytest_generate_tests(metafunc):
>>>      Returns:
>>>          Nothing.
>>>      """
>>> -
>>> +    #print('name', metafunc.fixturenames)
>>
>> Revert that debug change?
> 
> Will do.
> 
>>
>>> diff --git a/test/py/tests/test_spl.py b/test/py/tests/test_spl.py
>>
>>> +    cons.restart_uboot_with_flags(['-u', ut_spl_subtest])
>>
>> How is that change reverted when the test runs, so that subsequent tests
>> are run on the main U-Boot rather than this restarted U-Boot?
> 
> Well actually at the moment it just continues into U-Boot. It will
> mostly pass the tests, but probably not all of them.

Looking at existing tests, there are quite a few that do this:

try:
    test body which might do something nasty to U-Boot
finally:
    u_boot_console.restart_uboot()

... so that might be applicable.

>> It feels like it'd be better to start a separate top-level test run for
>> this purpose, rather than swap out the U-Boot process in the middle of a
>> test run.
> 
> I was hoping that the fixture stuff would take care of that. How would
> I do a separate top-level test run?

That'd simply be just running test/py/test.py and passing it the
relevant U-Boot binary/args rather than the main binary. I assume you'd
want to pass relevant -k option to restrict the set of tests run to
something relevant, and an appropriate --build-dir option to point at
the binary.