[PATCH v3 00/41] test: Refactor tests to have a single test runner

[Tom Rini]

On Wed, Mar 03, 2021 at 02:37:51PM -0500, Tom Rini wrote:
> On Wed, Feb 03, 2021 at 05:44:06AM -0700, Simon Glass wrote:
> 
> > At present U-Boot has two broad sets of tests in the C code: driver model
> > tests which do a lot of pre-/post-init and command tests which do not.
> > 
> > This separation makes it slightly harder to write a test, since there are
> > two different test-state structures and different rules for running the
> > two different test types. At present these rules are determined by where
> > the test is (actually its prefix).
> > 
> > All unit tests can be run from the command line with the 'ut' command.
> > Since SPL does not have commands, it currently calls the test runner
> > directly and offers no control of which tests are run.
> > 
> > This seems like a good time to refactor the tests into a unified test
> > runner, allowing U-Boot proper and SPL to use the same path, perhaps with
> > some different conditions along the way.
> > 
> > This series sets up a unified runner called ut_run_list(), which runs a
> > set of tests from a linker_list. Driver model tests are distinguished by
> > a new UT_TESTF_DM flag so that the necessary init and cleanup can still
> > be done.
> > 
> > The runner is modified to support running SPL tests that are not solely
> > for driver model. An example test for FIT loading is added as a
> > demonstration.
> > 
> > In addition, some documentation is added to explain how to write tests.
> > 
> > This series is available at u-boot-dm/test-working
> > 
> > Changes in v3:
> > - Add new patch to re-enable test_ofplatdata
> > - Reword the SPL tests section for clarity
> > - Use test_set_state() throughout test-main.c instead of direct assignment
> > - Update the pytest collector as well
> > 
> > Changes in v2:
> > - Use correct rst format for 'Ad-hoc tests' section
> > - Expand docs on how each type of test is marked
> > - Put the docs in tests_sandbox since it is more related to sandbox
> > - Put in a mention of tests_sandbox in the main testing docs
> > - Add a note that SPL tests can in fact be run individualy
> > - Document how to run all C tests with 'ut all'
> > - Fix 'get list' typo
> > - Fix conditions so non-DM SPL tests are actually run
> > - Allow for prefix to be NULL, to match function comment
> > - Add new patches to cover running an SPL test
> 
> There was a small problem applying one or two of these patches that I
> fixed up.  But then when testing locally (Ubuntu 18.04, but virtualenv
> setup and populated with test/py/requirements.txt) like 200 tests failed
> to run for Sandbox.  Please check this and that CI runs.

Er, something has gone odd for me locally, please disregard.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20210303/06d8956b/attachment.sig>