[U-Boot] [PATCHv2 10/13] test/py: Rework test.py to be a different kind of wrapper

[Tom Rini]

On Wed, Oct 23, 2019 at 12:04:30PM -0600, Stephen Warren wrote:
> On 10/23/19 11:29 AM, Stephen Warren wrote:
> > On 10/23/19 11:12 AM, Tom Rini wrote:
> > > On Wed, Oct 23, 2019 at 11:03:38AM -0600, Stephen Warren wrote:
> > > > On 10/23/19 10:58 AM, Tom Rini wrote:
> > > > > On Wed, Oct 23, 2019 at 10:55:57AM -0600, Stephen Warren wrote:
> > > > > > On 10/22/19 9:20 PM, Tom Rini wrote:
> > > > > > > Now that we have moved to being based on pytest for
> > > > > > > python3 we need to
> > > > > > > make our test.py wrapper more robust in terms of only calling python3
> > > > > > > rather than possibly finding and using python2.  To
> > > > > > > do this, change from
> > > > > > > execvp()'ing pytest to invoking the package itself
> > > > > > > via python.  In the
> > > > > > > event that pytest is unavailable we still get a user-friendly error:
> > > > > > 
> > > > > > I guess if this works, it's fine. But I think it'd work
> > > > > > just as well to
> > > > > > simply s/py.test/py.test-3/ in the existing code?
> > > > > 
> > > > > We can't because pip only provides "py.test" / "pytest" binaries.
> > > > 
> > > > Hmm. It sucks that virtualenv does things differently to Ubuntu
> > > > packaging
> > > > then. That means this series will probably break my test system, which
> > > > doesn't use virtualenv. I guess I'll apply it and see.
> > > 
> > > This should be what works everywhere tho.  Instead of hoping that
> > > "py.test" will be the right python version we tell python3 to use the
> > > pytest module it has.  So if you have system python3-pytest, that works,
> > > if you (python3) pip install pytest locally, that works and if you
> > > virtualenv + python3 + pip install pytest, that works.
> > 
> > On Ubuntu 16.04, with everything installed from Ubuntu packages rather
> > than using virtualenv, I get this:
> > 
> > > + ./src/u-boot/test/py/test.py --bd sandbox --build-dir /var/lib/jenkins/workspace/u-boot-github_swarren_u-boot-tegra_dev-test-py/U_BOOT_BOARD/sandbox/build/u-boot/sandbox
> > > -k 'not sleep'
> > > Traceback (most recent call last):
> > >   File "./src/u-boot/test/py/test.py", line 20, in <module>
> > >     sys.exit(load_entry_point('pytest', 'console_scripts',
> > > 'pytest')(args))
> > >   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py",
> > > line 542, in load_entry_point
> > >     return get_distribution(dist).load_entry_point(group, name)
> > >   File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py",
> > > line 2568, in load_entry_point
> > >     raise ImportError("Entry point %r not found" % ((group, name),))
> > > ImportError: Entry point ('console_scripts', 'pytest') not found
> > 
> > > [swarren at swarren-lx1 u-boot]$ dpkg -l|grep py|grep test
> > > ii  python-pytest    2.8.7-4
> > > ii  python3-pytest   2.8.7-4
> > 
> > I guess that's much older than the pytest in requirements.txt (5.2.1!!).
> > While I think we can require a newish version of Python (3.5) since
> > that's widely available on supported OSs, I don't think we want to force
> > people to have a very new version of all the packages (which supported
> > distros don't package yet), or change they way they run pytest (by
> > requiring use of a virtualenv and requirements.txt) since that mechanism
> > isn't supported on old branches of U-Boot, which will then require all
> > kinds of special cases in scripts that invoke the U-Boot test system.
> 
> Ah. The issue is the name of the entry point; the last parameter to
> load_entry_point(). "py.test" is supported everywhere, but only newer
> versions of pytesst have a "pytest" entry point. Changing that parameter to
> "py.test" makes the wrapper script work. Now every other test fails; I will
> see if there's an easy fix for that or not.

We're going to have to make a cut-off point somewhere it looks like.
pytest 2.8.7 (which we had pinned to) is incompatible with pytest 4.x
and throws a lot of warnings I believe on pytest 3.x..

What I really want to avoid is the problem we have today where we've
actively pinned ourselves to a now quite out of date version and have a
lot of problems showing up.

So we aren't going to have one environment that works for history and
future but I am hoping / trying to plan such that if we have
requirements.txt we can say from here until the next new best practices
(which today would be using pipenv, but that starts out with pip install
pipenv, sigh...) we can say to use requirements.txt and for history,
python2 + pytest 2.8.7 and local setups could even dump that in to a
requirements.txt file if that makes scripting easier.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191023/7a81a956/attachment.sig>