[PATCH v3 01/10] binman: support running multiple tests

[Simon Glass]

Hi Sam,

On 2026-06-10T01:27:41, Sam Day via B4 Relay
<devnull+me.samcday.com at kernel.org> wrote:
> binman: support running multiple tests
>
> The helptext for binman test suggests that this was already supported,
> but it wasn't properly wired up.
>
> tools/binman/main.py            |  8 ++++----
>  tools/u_boot_pylib/test_util.py | 23 ++++++++++++++++-------
>  2 files changed, 20 insertions(+), 11 deletions(-)

This looks good.

> diff --git a/tools/u_boot_pylib/test_util.py b/tools/u_boot_pylib/test_util.py
> @@ -157,7 +157,7 @@ def run_test_suites(toolname, debug, verbosity, no_capture, test_preserve_dirs,
>              the output directory for this test. Both directories are displayed
>              on the command line.
>          processes: Number of processes to use to run tests (None=same as #CPUs)
> -        test_name: Name of test to run, or None for all
> +        test_name: Name of test, or list of test names, to run; None for all
>          toolpath: List of paths to use for tools

Since the parameter now accepts a list, please rename it to test_names
and update the four other callers (tools/patman/__main__.py,
tools/dtoc/main.py, tools/buildman/main.py,
tools/u_boot_pylib/__main__.py, tools/dtoc/test_fdt.py) at the same
time. The isinstance() branch then goes away too.

> diff --git a/tools/u_boot_pylib/test_util.py b/tools/u_boot_pylib/test_util.py
> @@ -182,12 +182,19 @@ def run_test_suites(toolname, debug, verbosity, no_capture, test_preserve_dirs,
>          resultclass=FullTextTestResult,
>      )
>
> -    if use_concurrent and processes != 1 and not test_name:
> +    if isinstance(test_name, str):
> +        test_names = [test_name]
> +    else:
> +        test_names = list(test_name or [])
> +    test_name_set = set(test_names)

test_name_set is dead by the end of patch 2, which replaces 'module in
test_name_set' with _match_name(module). Please fold the two patches
so this set is never introduced, or skip building it here and just use
'in test_names' until patch 2 replaces the check.

> diff --git a/tools/binman/main.py b/tools/binman/main.py
> @@ -58,8 +58,8 @@ def RunTests(debug, verbosity, processes, test_preserve_dirs, args, toolpath):
>              the output directory for this test. Both directories are displayed
>              on the command line.
>          processes: Number of processes to use to run tests (None=same as #CPUs)
> -        args: List of positional args provided to binman. This can hold a test
> -            name to execute (as in 'binman test testSections', for example)
> +        args: List of positional args provided to binman. This can hold test
> +            names to execute (as in 'binman test testSections', for example)

While you are here, the line above ("If a single test is selected (in
args[0]) it also preserves the output directory for this test") is now
stale - preserve_outdirs fires for any number of selected tests. Same
comment for the matching docstring in run_test_suites(). Please tidy
both up.

Regards,
Simon