[PATCH v2 00/12] test: Add support for passing arguments to C unit tests

[Simon Glass]

Hi Tom,

On Thu, 16 Apr 2026 at 11:42, Simon Glass <sjg at chromium.org> wrote:
>
> Hi Tom,
>
> On Mon, 13 Apr 2026 at 14:13, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Sun, Apr 12, 2026 at 05:19:37AM -0600, Simon Glass wrote:
> > > Some tests have two parts: a Python test which handles setup, such as
> > > creating filesystem images or configuring the environment, and a C
> > > test which does the bulk of the work including assertions. At present
> > > there is no way to pass information between these two parts, so
> > > things like paths and device names must be hard-coded in the C test.
> > >
> > > This tends to make people write tests in Python, but this is not ideal
> > > for several reasons:
> >
> > You still need to work on explaining your changes in a positive manner.
> >
> > > - Python lacks access to things like return codes and has to scan the
> > >   console output to figure out what happened
> >
> > Pro: Python exercises things the way users exercise them and so exposes
> > deficiencies in our user interface as well, when applicable.
> >
> > > - Python lacks access to internal state, so it cannot directly check
> > >   the result of an operation
> >
> > It's unclear when that is, or is not a problem and when tests that do
> > check intermediate state are providing value.
> >
> > > - Python is ~100x slower due to console interaction, etc.
> >
> > Again, maybe you should go back to your suggestion of several years ago
> > to see about addressing that, as the tests themselves are not slow, it's
> > the pipe? Our brave new AI powered world runs on Python handing stuff
> > off and getting it back super quickly.
> >
> > > - C tests can be stepped through in gdb, which is much harder with
> > >   Python driving the console
> > > - C tests can exercise internal APIs directly (e.g. fs_read(),
> > >   fs_size()) rather than going through the command layer, so they
> > >   test more precisely what they intend to
> >
> > All things that may or may not be problems for other people, or are
> > intentional choices as they again test the system the way it's used.
> > Using gdb attached to u-boot in the other terminal where you're running
> > breaking things sounds like a reasonable debug path to me. And having
> > unit tests calling in to functions is why today we have a lot of things
> > not marked as "static" and so not being optimized as well as they could
> > by the compiler, which is its own set of trade-offs and not an automatic
> > good thing and design win.
> >
> > [snip]
> > >  16 files changed, 1437 insertions(+), 279 deletions(-)
> >
> > All of which is to say, you need to explain yourself in a positive, not
> > negative or oppositional way. Otherwise this reads like "I don't like
> > what we do now, here's a bunch of change to what I like instead".
>
> Just to check, is this a discussion and you would like me to reply
> with more details about the benefits of C over Python in particular
> circumstances? Or are you wanting the cover-letter rewritten. Or is
> this just a NAK?
>
> I did some work on using a socket to talk to sandbox (it is faster and
> also avoids some of the confusion as to what state sandbox is in
> without having to rely on looking at console output). But I never
> quite finished it. The other option (which we briefly discussed years
> ago) would be to build U-Boot as a Python module. I haven't tried that
> at all, but it's an idea.
>
> I am well aware of the tradeoffs between Python and C, so I am not
> advocating one over the other in all cases. I use gdb a lot with the
> pytests - e.g. hunting down memory leaks recent. Really I just want
> them to coexist better - i.e. improving the sandbox pattern where
> Python sets something up and then C runs it.
>

I see that this series is marked 'deferred'. Does that mean rejected,
or is there something you are waiting on. I believe this (or something
like it) is an important new feature for testing.

Regards,
SImon