[U-Boot] [PATCH V4 1/8] test/py: Implement pytest infrastructure

[Simon Glass]

On 15 January 2016 at 11:15, Stephen Warren <swarren at wwwdotorg.org> wrote:
> This tool aims to test U-Boot by executing U-Boot shell commands using the
> console interface. A single top-level script exists to execute or attach
> to the U-Boot console, run the entire script of tests against it, and
> summarize the results. Advantages of this approach are:
>
> - Testing is performed in the same way a user or script would interact
>   with U-Boot; there can be no disconnect.
> - There is no need to write or embed test-related code into U-Boot itself.
>   It is asserted that writing test-related code in Python is simpler and
>   more flexible that writing it all in C.
> - It is reasonably simple to interact with U-Boot in this way.
>
> A few simple tests are provided as examples. Soon, we should convert as
> many as possible of the other tests in test/* and test/cmd_ut.c too.
>
> The hook scripts, relay control utilities, and udev rules I use for my
> own HW setup are published at https://github.com/swarren/uboot-test-hooks.
>
> See README.md for more details!
>
> Signed-off-by: Stephen Warren <swarren at wwwdotorg.org>
> Signed-off-by: Stephen Warren <swarren at nvidia.com>
> Tested-by: Michal Simek <michal.simek at xilinx.com>
> Tested-by: Simon Glass <sjg at chromium.org>
> Acked-by: Simon Glass <sjg at chromium.org> #v3
> ---
> Apologies for the resend; I forgot to CC the list last time.
>
> v4:
> - Fix all users after ubspawn -> u_boot_spawn module rename.
> - Don't escape ^ in the HTML log file.
> - Fix u_boot_spawn.py Spawn.__init__ to set SIGHUP handler is set to
>   SIG_DFL. This ensures that sandbox responds to SIGHUP when run non-
>   interactively (e.g. under a daemon continuous integration server).
> - Fix logging of all child processes to make sure that all stdout/err,
>   exit code, and exception data from all processes makes its way to both
>   test stdout (for pytest to log to its stdout in case of error) and the
>   HTML test log. This changed multiplexed_log.py RunAndLog.run() (a fair
>   bit of change) and u_boot_console_exec_attach.py ConseolExecAttach.*
>   (just passing sys.stdout to log.get_runner() everywhere).
> - Also mention kermit as a possible app to attach to a serial console.
> - Fix test_version to work correctly when BUILD_TAG is set, which causes
>   U-Boot to print both the version string and BUILD_TAG value at startup,
>   whereas the "version" shell command only prints the version string.
> - Reformat function-level documentation.
>
> v3:
> - Rework HTML log generation so that TAB characters render as expected.
>   Suggested by Michal Simek.
> - Move test scripts into a sub-directory.
>   Suggested by Michal Simek.
> - s/uboot/u[-_]boot/g. Suggested by Simon Glass.
> - s/"/'/g. Suggested by Simon Glass.
> - Typo fixes.
> - Add more documentation. Suggested by Simon Glass.
> - Make "notes" in the log file be <pre> so that their formatting is
>   preserved. This is useful for large notes such as exception dumps.
>
> v2:
> - Many fixes and tweaks have been squashed in. Separated out some of
>   the tests into separate commits, and added some more tests.
> ---
>  test/py/.gitignore                    |   1 +
>  test/py/README.md                     | 300 ++++++++++++++++++++
>  test/py/conftest.py                   | 422 ++++++++++++++++++++++++++++
>  test/py/multiplexed_log.css           |  88 ++++++
>  test/py/multiplexed_log.py            | 515 ++++++++++++++++++++++++++++++++++
>  test/py/pytest.ini                    |  11 +
>  test/py/test.py                       |  32 +++
>  test/py/tests/test_000_version.py     |  20 ++
>  test/py/tests/test_help.py            |   9 +
>  test/py/tests/test_unknown_cmd.py     |  14 +
>  test/py/u_boot_console_base.py        | 360 ++++++++++++++++++++++++
>  test/py/u_boot_console_exec_attach.py |  65 +++++
>  test/py/u_boot_console_sandbox.py     |  79 ++++++
>  test/py/u_boot_spawn.py               | 174 ++++++++++++
>  14 files changed, 2090 insertions(+)
>  create mode 100644 test/py/.gitignore
>  create mode 100644 test/py/README.md
>  create mode 100644 test/py/conftest.py
>  create mode 100644 test/py/multiplexed_log.css
>  create mode 100644 test/py/multiplexed_log.py
>  create mode 100644 test/py/pytest.ini
>  create mode 100755 test/py/test.py
>  create mode 100644 test/py/tests/test_000_version.py
>  create mode 100644 test/py/tests/test_help.py
>  create mode 100644 test/py/tests/test_unknown_cmd.py
>  create mode 100644 test/py/u_boot_console_base.py
>  create mode 100644 test/py/u_boot_console_exec_attach.py
>  create mode 100644 test/py/u_boot_console_sandbox.py
>  create mode 100644 test/py/u_boot_spawn.py

Applied to u-boot-dm, thanks!