[U-Boot] [PATCH V3 1/7] test/py: Implement pytest infrastructure

[Simon Glass]

On 5 January 2016 at 15:58, Stephen Warren <swarren at wwwdotorg.org> wrote:
> This tool aims to test U-Boot by executing U-Boot shell commands using the
> console interface. A single top-level script exists to execute or attach
> to the U-Boot console, run the entire script of tests against it, and
> summarize the results. Advantages of this approach are:
>
> - Testing is performed in the same way a user or script would interact
>   with U-Boot; there can be no disconnect.
> - There is no need to write or embed test-related code into U-Boot itself.
>   It is asserted that writing test-related code in Python is simpler and
>   more flexible that writing it all in C.
> - It is reasonably simple to interact with U-Boot in this way.
>
> A few simple tests are provided as examples. Soon, we should convert as
> many as possible of the other tests in test/* and test/cmd_ut.c too.
>
> The hook scripts, relay control utilities, and udev rules I use for my
> own HW setup are published at https://github.com/swarren/uboot-test-hooks.
>
> See README.md for more details!
>
> Signed-off-by: Stephen Warren <swarren at wwwdotorg.org>
> Signed-off-by: Stephen Warren <swarren at nvidia.com>
> Tested-by: Michal Simek <michal.simek at xilinx.com>
> Tested-by: Simon Glass <sjg at chromium.org>
> ---
> v3:
> - Rework HTML log generation so that TAB characters render as expected.
>   Suggested by Michal Simek.
> - Move test scripts into a sub-directory.
>   Suggested by Michal Simek.
> - s/uboot/u[-_]boot/g. Suggested by Simon Glass.
> - s/"/'/g. Suggested by Simon Glass.
> - Typo fixes.
> - Add more documentation. Suggested by Simon Glass.
> - Make "notes" in the log file be <pre> so that their formatting is
>   preserved. This is useful for large notes such as exception dumps.
>
> v2:
> - Many fixes and tweaks have been squashed in. Separated out some of
>   the tests into separate commits, and added some more tests.
> ---
>  test/py/.gitignore                    |   1 +
>  test/py/README.md                     | 300 ++++++++++++++++++++++++++++++
>  test/py/conftest.py                   | 335 ++++++++++++++++++++++++++++++++++
>  test/py/multiplexed_log.css           |  88 +++++++++
>  test/py/multiplexed_log.py            | 335 ++++++++++++++++++++++++++++++++++
>  test/py/pytest.ini                    |  11 ++
>  test/py/test.py                       |  32 ++++
>  test/py/tests/test_000_version.py     |  20 ++
>  test/py/tests/test_help.py            |   9 +
>  test/py/tests/test_unknown_cmd.py     |  14 ++
>  test/py/u_boot_console_base.py        | 260 ++++++++++++++++++++++++++
>  test/py/u_boot_console_exec_attach.py |  51 ++++++
>  test/py/u_boot_console_sandbox.py     |  48 +++++
>  test/py/u_boot_spawn.py               | 123 +++++++++++++
>  14 files changed, 1627 insertions(+)
>  create mode 100644 test/py/.gitignore
>  create mode 100644 test/py/README.md
>  create mode 100644 test/py/conftest.py
>  create mode 100644 test/py/multiplexed_log.css
>  create mode 100644 test/py/multiplexed_log.py
>  create mode 100644 test/py/pytest.ini
>  create mode 100755 test/py/test.py
>  create mode 100644 test/py/tests/test_000_version.py
>  create mode 100644 test/py/tests/test_help.py
>  create mode 100644 test/py/tests/test_unknown_cmd.py
>  create mode 100644 test/py/u_boot_console_base.py
>  create mode 100644 test/py/u_boot_console_exec_attach.py
>  create mode 100644 test/py/u_boot_console_sandbox.py
>  create mode 100644 test/py/u_boot_spawn.py

Looks good!

Acked-by: Simon Glass <sjg at chromium.org>

One comment about function / class comments. If you like at
buildman/patman they use this sort of format:

    def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build,
                  force_build_failures):
        """Build a particular commit.

        If the build is already done, and we are not forcing a build, we skip
        the build and just return the previously-saved results.

        Args:
            commit_upto: Commit number to build (0...n-1)
            brd: Board object to build
            work_dir: Directory to which the source will be checked out
            do_config: True to run a make <board>_defconfig on the source
            force_build: Force a build even if one was previously done
            force_build_failures: Force a bulid if the previous result showed
                failure

        Returns:
            tuple containing:
                - CommandResult object containing the results of the build
                - boolean indicating whether 'make config' is still needed
        """

So double quotes, and the first line describes the function in a very
simple way, a bit like a git commit subject. The arguments and return
value are specifically called out, as we do in the C code.

For consistency I'd suggest doing the same.

Regards,
Simon