[PATCH 0/3] doc: Document remaining test_b* tests

[Tom Rini]

On Sat, May 10, 2025 at 01:27:23PM +0200, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 8 May 2025 at 23:42, Tom Rini <trini at konsulko.com> wrote:
> >
> > Hey all,
> >
> > This short series is a follow-up to my last series. It could have been
> > longer, but I think this is a good point to get some feedback on a few
> > points.
> >
> > As background, something I wondered about and the answer seems to be No
> > is, can we automatically take things like:
> > @pytest.mark.buildconfigspec('cmd_button')
> > and get some nice output? Since we can't that limits the value,
> > possibly, of generating some documentation.
> >
> > First, as this series shows, is there value in documenting tests which
> > do not require additional configuration? We have some tests with no
> > comments and some that are fully commented, but at the end of the day
> > when trying to run these tests, pytest will already say helpful things
> > like not being supported on a given platform, or not run because
> > something isn't enabled in the build. Spelling that out in generated
> > documentation would require duplication of work.
> 
> Yes, I agree. I think the code is the best place for most of the docs.

> 
> >
> > Second, is there value in documenting arguments? Especially ones like
> > "ubman" that are just required and always present?
> 
> I like documenting arguments, since when people see the docs they
> continue the trend. Python suffers terribly from a 'def
> my_function(*args, **kwargs)' disease where you have to spend hours
> spelunking around to figure out what is actually valid as an argument.
> Highly Pythonic, but dreadful.

Maybe I wasn't clear in these two points. Let us use
test/py/tests/test_cat.py as a concrete example. Today, most of the file
is:
# SPDX-License-Identifier:      GPL-2.0+

""" Unit test for cat command
"""

import pytest
from subprocess import call, check_call, CalledProcessError
from tests import fs_helper

@pytest.mark.boardspec('sandbox')
@pytest.mark.buildconfigspec('cmd_cat')
def test_cat(ubman):
    """ Unit test for cat

    Args:
        ubman -- U-Boot console
    """
    try:
    ....

My point and question is that "@pytest.mark.buildconfigspec('cmd_cat')"
will not be translated to generated documentation, but also that if you
try and run this test outside of sandbox, pytest already tells you it's
not supported, and a similar error if sandbox but not cmd_cat.

So is there any value in adding doc/develop/pytest/test_cat.rst that
will consume the above and generate valid documentation that I'm not
sure would ever be read?

> > Third, should we document internal functions too, or just the tests? I
> > would be inclined to say no unless they're expected to be re-used. The
> > test_net functions to bring up the networking are an example that of
> > reuse that should be documented for example.
> 
> Sounds right. But I am likely to continue to document all Python
> functions, since I never know when I'm going to want to export
> something for use elsewhere.

I'm certainly not saying we shouldn't comment the code, but perhaps
a follow-up to document test/py/tests/test_distro.py would clarify some
of what I'm asking about in your mind.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20250512/87c9f769/attachment.sig>