[PATCH 10/12] test: Add documentation for the test framework

[Simon Glass]

From: Simon Glass <simon.glass at canonical.com>

Add documentation for test assertions, the private buffer, and test
parameters. This covers the available ut_assert*() macros, console
output checks, memory helpers, and how to pass typed arguments from
Python to C tests.

Signed-off-by: Simon Glass <simon.glass at canonical.com>
Signed-off-by: Simon Glass <sjg at chromium.org>
---

 doc/develop/tests_writing.rst | 169 ++++++++++++++++++++++++++++++++++
 doc/usage/cmd/ut.rst          |  11 ++-
 2 files changed, 178 insertions(+), 2 deletions(-)

diff --git a/doc/develop/tests_writing.rst b/doc/develop/tests_writing.rst
index 1a020caa411..40192f9db30 100644
--- a/doc/develop/tests_writing.rst
+++ b/doc/develop/tests_writing.rst
@@ -101,6 +101,47 @@ constructs, in this case to check that the expected things happened in the
 Python test.
 
 
+Passing arguments to C tests
+----------------------------
+
+Sometimes a C test needs parameters from Python, such as filenames or expected
+values that are generated at runtime. The test-argument feature allows this.
+
+Use the `UNIT_TEST_ARGS` macro to declare a test with arguments::
+
+   static int my_test_norun(struct unit_test_state *uts)
+   {
+      const char *filename = ut_str(0);
+      int count = ut_int(1);
+
+      /* test code using filename and count */
+
+      return 0;
+   }
+   UNIT_TEST_ARGS(my_test_norun, UTF_CONSOLE | UTF_MANUAL, my_suite,
+                  { "filename", UT_ARG_STR },
+                  { "count", UT_ARG_INT });
+
+Each argument definition specifies a name and type:
+
+- `UT_ARG_STR` - string argument, accessed via `ut_str(n)`
+- `UT_ARG_INT` - integer argument, accessed via `ut_int(n)`
+- `UT_ARG_BOOL` - boolean argument, accessed via `ut_bool(n)`
+  (use `1` for true, any other value for false)
+
+Arguments are passed on the command line in `name=value` format::
+
+   ut -f my_suite my_test_norun filename=/path/to/file count=42
+
+From Python, you can call the test like this::
+
+   cmd = f'ut -f my_suite my_test_norun filename={filepath} count={count}'
+   ubman.run_command(cmd)
+
+This approach combines Python's flexibility for setup (creating files,
+generating values) with C's speed and debuggability for the actual test logic.
+
+
 How slow are Python tests?
 --------------------------
 
@@ -384,6 +425,134 @@ existing suite or creating a new one.
 An example SPL test is spl_test_load().
 
 
+.. _tests_writing_assertions:
+
+Assertions
+----------
+
+The test framework provides various assertion macros, defined in
+``include/test/ut.h``. All of these return from the test function on failure,
+unless ``uts->soft_fail`` is set.
+
+Basic assertions
+~~~~~~~~~~~~~~~~
+
+ut_assert(cond)
+    Assert that a condition is non-zero (true)
+
+ut_assertf(cond, fmt, args...)
+    Assert that a condition is non-zero, with a printf() message on failure
+
+ut_assertok(cond)
+    Assert that an operation succeeds (returns 0)
+
+ut_reportf(fmt, args...)
+    Report a failure with a printf() message (always fails)
+
+Value comparisons
+~~~~~~~~~~~~~~~~~
+
+ut_asserteq(expr1, expr2)
+    Assert that two int expressions are equal
+
+ut_asserteq_64(expr1, expr2)
+    Assert that two 64-bit expressions are equal
+
+ut_asserteq_str(expr1, expr2)
+    Assert that two strings are equal
+
+ut_asserteq_strn(expr1, expr2)
+    Assert that two strings are equal, up to the length of the first
+
+ut_asserteq_mem(expr1, expr2, len)
+    Assert that two memory areas are equal
+
+ut_asserteq_ptr(expr1, expr2)
+    Assert that two pointers are equal
+
+ut_asserteq_addr(expr1, expr2)
+    Assert that two addresses (converted from pointers via map_to_sysmem())
+    are equal
+
+ut_asserteq_regex(pattern, str)
+    Assert that a string matches a regular expression pattern. Uses the SLRE
+    library for regex matching. Useful when exact matching is fragile, e.g.
+    when output contains line numbers or variable content.
+
+Pointer assertions
+~~~~~~~~~~~~~~~~~~
+
+ut_assertnull(expr)
+    Assert that a pointer is NULL
+
+ut_assertnonnull(expr)
+    Assert that a pointer is not NULL
+
+ut_assertok_ptr(expr)
+    Assert that a pointer is not an error pointer (checked with IS_ERR())
+
+Console output assertions
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These are used to check console output when ``UTF_CONSOLE`` flag is set.
+
+ut_assert_nextline(fmt, args...)
+    Assert that the next console output line matches the format string
+
+ut_assert_nextlinen(fmt, args...)
+    Assert that the next console output line matches up to the format
+    string length
+
+ut_assert_nextline_regex(pattern)
+    Assert that the next console output line matches a regex pattern
+
+ut_assert_nextline_empty()
+    Assert that the next console output line is empty
+
+ut_assert_skipline()
+    Assert that there is a next console output line, and skip it
+
+ut_assert_skip_to_line(fmt, args...)
+    Skip console output until a matching line is found
+
+ut_assert_skip_to_linen(fmt, args...)
+    Skip console output until a partial match is found (compares up to the
+    format-string length)
+
+ut_assert_console_end()
+    Assert that there is no more console output
+
+ut_assert_nextlines_are_dump(total_bytes)
+    Assert that the next lines are a print_buffer() hex dump of the specified
+    size
+
+Memory helpers
+~~~~~~~~~~~~~~
+
+These help check for memory leaks:
+
+ut_check_free()
+    Return the number of bytes free in the malloc() pool
+
+ut_check_delta(last)
+    Return the change in free memory since ``last`` was obtained from
+    ``ut_check_free()``. A positive value means more memory has been allocated.
+
+Private buffer
+~~~~~~~~~~~~~~
+
+Each test has access to a private buffer ``uts->priv`` (256 bytes) for temporary
+data. This avoids the need to allocate memory or use global variables::
+
+   static int my_test(struct unit_test_state *uts)
+   {
+      snprintf(uts->priv, sizeof(uts->priv), "/%s", filename);
+      /* use uts->priv as a path string */
+
+      return 0;
+   }
+
+
 Writing Python tests
 --------------------
 
diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst
index d8c3cbf496c..a26ee6ad7de 100644
--- a/doc/usage/cmd/ut.rst
+++ b/doc/usage/cmd/ut.rst
@@ -11,7 +11,7 @@ Synopsis
 
 ::
 
-    ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
+    ut [-r<runs>] [-f] [-R] [-I<n>:<one_test>] [<suite> | all [<test>]] [<args>...]
     ut [-s] info
 
 Description
@@ -37,10 +37,17 @@ test
     causes another test to fail. If the one test fails, testing stops
     immediately.
 
+-R
+    Preserve console recording on test failure. Normally when a test fails,
+    console recording is disabled so error messages go directly to output.
+    This flag keeps recording enabled, which is useful when testing the test
+    framework itself.
+
 args
     Optional arguments to pass to the test, in `name=value` format. These are
     used by tests declared with `UNIT_TEST_ARGS()` which define expected
-    argument names and types.
+    argument names and types. See :ref:`develop/tests_writing:passing arguments
+    to c tests` for details.
 
 Typically the command is run on :ref:`arch/sandbox/sandbox:sandbox` since it
 includes a near-complete set of emulators, no code-size limits, many CONFIG
-- 
2.43.0