[U-Boot] [PATCH] Implement pytest-based test infrastructure

[Tom Rini]

On Thu, Nov 19, 2015 at 10:00:32AM -0700, Stephen Warren wrote:
> On 11/19/2015 07:45 AM, Simon Glass wrote:
> >Hi Stephen,
> >
> >On 14 November 2015 at 23:53, Stephen Warren <swarren at wwwdotorg.org> wrote:
> >>This tool aims to test U-Boot by executing U-Boot shell commands using the
> >>console interface. A single top-level script exists to execute or attach
> >>to the U-Boot console, run the entire script of tests against it, and
> >>summarize the results. Advantages of this approach are:
> >>
> >>- Testing is performed in the same way a user or script would interact
> >>   with U-Boot; there can be no disconnect.
> >>- There is no need to write or embed test-related code into U-Boot itself.
> >>   It is asserted that writing test-related code in Python is simpler and
> >>   more flexible that writing it all in C.
> >>- It is reasonably simple to interact with U-Boot in this way.
> >>
> >>A few simple tests are provided as examples. Soon, we should convert as
> >>many as possible of the other tests in test/* and test/cmd_ut.c too.
> >
> >It's great to see this and thank you for putting in the effort!
> >
> >It looks like a good way of doing functional tests. I still see a role
> >for unit tests and things like test/dm. But if we can arrange to call
> >all U-Boot tests (unit and functional) from one 'test.py' command that
> >would be a win.
> >
> >I'll look more when I can get it to work - see below.
> ...
> >I get this on my Ubuntu 64-bit machine (14.04.3)
> >
> >$ ./test/py/test.py --bd sandbox --buildTraceback (most recent call last):
> >   File "./test/py/test.py", line 12, in <module>
> >     os.execvp("py.test", args)
> >   File "/usr/lib/python2.7/os.py", line 344, in execvp
> >     _execvpe(file, args)
> >   File "/usr/lib/python2.7/os.py", line 380, in _execvpe
> >     func(fullname, *argrest)
> >OSError: [Errno 2] No such file or directory
> 
> "py.test" isn't in your $PATH. Did you install it? See the following
> in test/py/README.md:
> 
> >## Requirements
> >
> >The test suite is implemented using pytest. Interaction with the U-Boot
> >console uses pexpect. Interaction with real hardware uses the tools of your
> >choice; you get to implement various "hook" scripts that are called by the
> >test suite at the appropriate time.
> >
> >On Debian or Debian-like distributions, the following packages are required.
> >Similar package names should exist in other distributions.
> >
> >| Package        | Version tested (Ubuntu 14.04) |
> >| -------------- | ----------------------------- |
> >| python         | 2.7.5-5ubuntu3                |
> >| python-pytest  | 2.5.1-1                       |
> >| python-pexpect | 3.1-1ubuntu0.1                |
> 
> In the main Python code, I trapped at least one exception location
> and made it print a message about checking the docs for missing
> requirements. I can probably patch the top-level test.py to do the
> same.

Isn't there some way to inject the local to U-Boot copy of the libraries
in?  I swear I've done something like that before in python..


-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20151123/2dcdd578/attachment.sig>