[U-Boot] [PATCH 0/3] add inital SF tests

[Stephen Warren]

On 02/27/2018 04:52 AM, Michal Simek wrote:
> On 27.2.2018 05:17, Liam Beguin wrote:
>> Hi all,
>>
>> This is the inital step to adding tests for the SF subsystem plus very
>> minor fixes. It is based on work I found on the mailing list[1].
>> For now, it doesn't do much but I plan on adding code to reset the flash
>> to its initial state (based on an env flag) and more code to test the
>> `sf protect` subcommand (which is the main goal of this series).
>> I'm sending it now to make sure it's headed in the right direction.
>>
>> Thanks,
>> Liam Beguin
>>
>> [ 1 ] https://patchwork.ozlabs.org/patch/623061/
>>
>> Liam Beguin (3):
>>    test/py: README: fix typo
>>    test/py: README: add HOSTNAME to PYTHONPATH
>>    test/py: add spi_flash tests
>>
>>   test/py/README.md        |   6 +-
>>   test/py/tests/test_sf.py | 233 +++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 236 insertions(+), 3 deletions(-)
>>   create mode 100644 test/py/tests/test_sf.py
>>
>> base-commit: 4bafceff0e9e5a36908031e41c69a6b37e82da58
>> Published-As: https://github.com/Liambeguin/u-boot/releases/tag/test_sf_v1
>>
> 
> one more question about these tests in general.
> I have looked at randomization of these tests and all tests should be
> written in a way that you can run just single one. That's why every test
> should be self contained without dependencies on each other.
> 
> I just found that running order in hush matters.
> test_hush_if_test_setup
> test_hush_if_test
> test_hush_if_test_teardown
> 
> Stephen: Is this expected or this simply just a bug?

This is expected given the way the test is currently written. You can 
easily run all 3 tests together by specifying "-k test_hush_if_test"; 
that string is a substring of all the 3 tests above. I guess we could 
rework test_hush_if_test_setup/test_hush_if_test_teardown to be a pytest 
module-level fixture in order to squash those 3 functions into a single 
test though.