[PATCH 04/30] doc: Explain how to run tests without pytest

[Simon Glass]

Add details about how to run a sandbox test directly, without using
pytest. This is more convenient for rapid development, since it is faster
and allows easier use of a debugger. Also mention sandbox_flattree as an
example of the different sandbox builds available.

Signed-off-by: Simon Glass <sjg at chromium.org>
---

 doc/develop/index.rst       |  1 +
 doc/develop/testing.rst     |  6 +++
 doc/develop/tests_under.rst | 79 +++++++++++++++++++++++++++++++++++++
 3 files changed, 86 insertions(+)
 create mode 100644 doc/develop/tests_under.rst

diff --git a/doc/develop/index.rst b/doc/develop/index.rst
index beaa64d8d90..cc930c9add4 100644
--- a/doc/develop/index.rst
+++ b/doc/develop/index.rst
@@ -32,3 +32,4 @@ Testing
    coccinelle
    testing
    py_testing
+   tests_under
diff --git a/doc/develop/testing.rst b/doc/develop/testing.rst
index a2a2c276374..593ca50de36 100644
--- a/doc/develop/testing.rst
+++ b/doc/develop/testing.rst
@@ -47,6 +47,12 @@ You can run the tests on sandbox with::
 
 This will produce HTML output in build-sandbox/test-log.html
 
+Some tests run with other versions of sandbox. For example sandbox_flattree
+runs the tests with livetree (the hierachical devicetree) disabled. You can
+also select particular tests with -k::
+
+   ./test/py/test.py --bd sandbox_flattree --build -k hello
+
 See test/py/README.md for more information about the pytest suite.
 
 
diff --git a/doc/develop/tests_under.rst b/doc/develop/tests_under.rst
new file mode 100644
index 00000000000..85bbd4f6734
--- /dev/null
+++ b/doc/develop/tests_under.rst
@@ -0,0 +1,79 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+Tests Under the Hood
+====================
+
+Running sandbox tests directly
+------------------------------
+
+Typically tests are run using the pytest suite. This is easy and always gets
+things right.
+
+But it is also possible to run some sandbox tests directly. For example, this
+runs the dm_test_gpio() test which you can find in test/dm/gpio.c::
+
+   $ ./u-boot -T -c "ut dm gpio"
+
+
+   U-Boot 2021.01
+
+   Model: sandbox
+   DRAM:  128 MiB
+   WDT:   Started with servicing (60s timeout)
+   MMC:   mmc2: 2 (SD), mmc1: 1 (SD), mmc0: 0 (SD)
+   In:    serial
+   Out:   vidconsole
+   Err:   vidconsole
+   Model: sandbox
+   SCSI:
+   Net:   eth0: eth at 10002000, eth5: eth at 10003000, eth3: sbe5, eth6: eth at 10004000
+   Test: dm_test_gpio: gpio.c
+   Test: dm_test_gpio: gpio.c (flat tree)
+   Failures: 0
+
+The -T option tells U-Boot to run with the 'test' devicetree (test.dts) instead
+of -D which selects the normal sandbox.dts - this is necessary because many
+tests rely on things in the test devicetree. If you try running tests without
+-T then you may see failures, like::
+
+   $ ./u-boot -c "ut dm gpio"
+
+
+   U-Boot 2021.01
+
+   DRAM:  128 MiB
+   WDT:   Not found!
+   MMC:
+   In:    serial
+   Out:   serial
+   Err:   serial
+   SCSI:
+   Net:   No ethernet found.
+   Please run with test device tree:
+       ./u-boot -d arch/sandbox/dts/test.dtb
+   Test: dm_test_gpio: gpio.c
+   test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
+   Test: dm_test_gpio: gpio.c (flat tree)
+   test/dm/gpio.c:37, dm_test_gpio(): 0 == gpio_lookup_name("b4", &dev, &offset, &gpio): Expected 0x0 (0), got 0xffffffea (-22)
+   Failures: 2
+
+The message above should provide a hint if you forget. Even running with -D
+will produce different results.
+
+You can easily use gdb on these tests, without needing --gdbserver::
+
+   $ gdb u-boot --args -T -c "ut dm gpio"
+   ...
+   (gdb) break dm_test_gpio
+   Breakpoint 1 at 0x1415bd: file test/dm/gpio.c, line 37.
+   (gdb) run -T -c "ut dm gpio"
+   Starting program: u-boot -T -c "ut dm gpio"
+   Test: dm_test_gpio: gpio.c
+
+   Breakpoint 1, dm_test_gpio (uts=0x5555558029a0 <global_dm_test_state>)
+       at files/test/dm/gpio.c:37
+   37		ut_assertok(gpio_lookup_name("b4", &dev, &offset, &gpio));
+   (gdb)
+
+You can then single-step and look at variables as needed.
+
-- 
2.30.0.280.ga3ce27912f-goog