[U-Boot] [PATCH] test/py: make each unit test a pytest

[Stephen Warren]

On 01/28/2016 08:52 PM, Simon Glass wrote:
> Hi Stephen,
> 
> On 28 January 2016 at 16:45, Stephen Warren <swarren at wwwdotorg.org> wrote:
>> From: Stephen Warren <swarren at nvidia.com>
>>
>> A custom fixture named ut_subtest is implemented which is parametrized
>> with the names of all unit tests that the U-Boot binary supports. This
>> causes each U-Boot unit test to be exposes as a separate pytest. In turn,
>> this allows more fine-grained pass/fail counts and test selection, e.g.:
>>
>> test.py --bd sandbox -k ut_dm_usb
>>
>> ... will run about 8 tests at present.
>>
>> Signed-off-by: Stephen Warren <swarren at nvidia.com>
>> ---
>> This depends on at least my recently sent "test/py: run C-based unit tests".
>>
>>  test/py/conftest.py      | 105 ++++++++++++++++++++++++++++++++++++-----------
>>  test/py/tests/test_ut.py |  14 +++----
>>  2 files changed, 86 insertions(+), 33 deletions(-)
> 
> This seems a bit extreme. It might be better to move the remaining
> three commands under the 'ut' subcommand. Then all unit tests would be
> visible from the 'ut' help...

I'm not sure what you mean by "extreme"? Do you mean you don't want each
unit test exposed as a separate pytest? I thought based on our previous
conversation that was exactly what you wanted. If not, I'm not sure what
the deficiency in the current code is; either all the dm subtests are
executed at once by a single pytest with a single overall status, or
they're each a separate pytest with individual status. Any grouping
that's in between those seems like it would be entirely arbitrary?