[U-Boot] [PATCHv2 10/13] test/py: Rework test.py to be a different kind of wrapper

Wed Oct 23 21:37:46 UTC 2019

On Wed, Oct 23, 2019 at 03:11:57PM -0600, Stephen Warren wrote:
> On 10/23/19 12:17 PM, Tom Rini wrote:
> > On Wed, Oct 23, 2019 at 12:04:30PM -0600, Stephen Warren wrote:
> > > On 10/23/19 11:29 AM, Stephen Warren wrote:
> > > > On 10/23/19 11:12 AM, Tom Rini wrote:
> > > > > On Wed, Oct 23, 2019 at 11:03:38AM -0600, Stephen Warren wrote:
> > > > > > On 10/23/19 10:58 AM, Tom Rini wrote:
> > > > > > > On Wed, Oct 23, 2019 at 10:55:57AM -0600, Stephen Warren wrote:
> > > > > > > > On 10/22/19 9:20 PM, Tom Rini wrote:
> > > > > > > > > Now that we have moved to being based on pytest for
> > > > > > > > > python3 we need to
> > > > > > > > > make our test.py wrapper more robust in terms of only calling python3
> > > > > > > > > rather than possibly finding and using python2.  To
> > > > > > > > > do this, change from
> > > > > > > > > execvp()'ing pytest to invoking the package itself
> > > > > > > > > via python.  In the
> > > > > > > > > event that pytest is unavailable we still get a user-friendly error:
> > > > > > > > 
> > > > > > > > I guess if this works, it's fine. But I think it'd work
> > > > > > > > just as well to
> > > > > > > > simply s/py.test/py.test-3/ in the existing code?
> > > > > > > 
> > > > > > > We can't because pip only provides "py.test" / "pytest" binaries.
> > > > > > 
> > > > > > Hmm. It sucks that virtualenv does things differently to Ubuntu
> > > > > > packaging
> > > > > > then. That means this series will probably break my test system, which
> > > > > > doesn't use virtualenv. I guess I'll apply it and see.
> > > > > 
> > > > > This should be what works everywhere tho.  Instead of hoping that
> > > > > "py.test" will be the right python version we tell python3 to use the
> > > > > pytest module it has.  So if you have system python3-pytest, that works,
> > > > > if you (python3) pip install pytest locally, that works and if you
> > > > > virtualenv + python3 + pip install pytest, that works.
> > > > 
> > > > On Ubuntu 16.04, with everything installed from Ubuntu packages rather
> > > > than using virtualenv, I get this:
> > > > 
> > > > > + ./src/u-boot/test/py/test.py --bd sandbox --build-dir /var/lib/jenkins/workspace/u-boot-github_swarren_u-boot-tegra_dev-test-py/U_BOOT_BOARD/sandbox/build/u-boot/sandbox
> > > > > -k 'not sleep'
> > > > > Traceback (most recent call last):
> > > > >    File "./src/u-boot/test/py/test.py", line 20, in <module>
> > > > >      sys.exit(load_entry_point('pytest', 'console_scripts',
> > > > > 'pytest')(args))
> > > > >    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py",
> > > > > line 542, in load_entry_point
> > > > >      return get_distribution(dist).load_entry_point(group, name)
> > > > >    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py",
> > > > > line 2568, in load_entry_point
> > > > >      raise ImportError("Entry point %r not found" % ((group, name),))
> > > > > ImportError: Entry point ('console_scripts', 'pytest') not found
> > > > 
> > > > > [swarren at swarren-lx1 u-boot]$ dpkg -l|grep py|grep test
> > > > > ii  python-pytest    2.8.7-4
> > > > > ii  python3-pytest   2.8.7-4
> > > > 
> > > > I guess that's much older than the pytest in requirements.txt (5.2.1!!).
> > > > While I think we can require a newish version of Python (3.5) since
> > > > that's widely available on supported OSs, I don't think we want to force
> > > > people to have a very new version of all the packages (which supported
> > > > distros don't package yet), or change they way they run pytest (by
> > > > requiring use of a virtualenv and requirements.txt) since that mechanism
> > > > isn't supported on old branches of U-Boot, which will then require all
> > > > kinds of special cases in scripts that invoke the U-Boot test system.
> > > 
> > > Ah. The issue is the name of the entry point; the last parameter to
> > > load_entry_point(). "py.test" is supported everywhere, but only newer
> > > versions of pytesst have a "pytest" entry point. Changing that parameter to
> > > "py.test" makes the wrapper script work. Now every other test fails; I will
> > > see if there's an easy fix for that or not.
> > 
> > We're going to have to make a cut-off point somewhere it looks like.
> > pytest 2.8.7 (which we had pinned to) is incompatible with pytest 4.x
> > and throws a lot of warnings I believe on pytest 3.x..
> > 
> > What I really want to avoid is the problem we have today where we've
> > actively pinned ourselves to a now quite out of date version and have a
> > lot of problems showing up.
> > 
> > So we aren't going to have one environment that works for history and
> > future but I am hoping / trying to plan such that if we have
> > requirements.txt we can say from here until the next new best practices
> > (which today would be using pipenv, but that starts out with pip install
> > pipenv, sigh...) we can say to use requirements.txt and for history,
> > python2 + pytest 2.8.7 and local setups could even dump that in to a
> > requirements.txt file if that makes scripting easier.
> 
> Indeed the pytest 4 changes were a lot of the problem, but there was plenty
> more. It turned out to be too painful to fix it all.
> 
> After a lot of pain, I've switched my system over to using a virtualenv when
> there's a test/py/requirements.txt file in the source. Using a virtualenv
> broke some of the HW remote control scripts, which were written for Python2
> but just said "#!/usr/bin/env python". Fixing that to request Python 2
> explicitly fixed that issue. Another pain was that the ftdi1 module (used by
> the HW remote control scripts) isn't available from pip so can't be
> installed in a virtualenv. Luckily explicitly using python2 for those
> scripts makes this irrelevant, since system libraries are used, since the
> python2 interpreter doesn't come from the virtualenv in that case. However,
> if that didn't fix the issue, then I'd have needed extra options to set up
> the virtualenv in order to rely on Ubuntu-provided package for the module.
> I'm quoting them below just in case someone searching the archives finds
> this later in the future!
> 
> > +if [ -f ./src/u-boot/test/py/requirements.txt ]; then
> > +    rm -rf "${venv_dir}"
> > +    # --system-site-packages: Need ftdi1 module which is not available via pip,
> > +    #   but is installed on the system
> > +    virtualenv --system-site-packages -p /usr/bin/python3 "${venv_dir}"
> > +    . "${venv_dir}/bin/activate"
> > +    # -U: upgrade: Ensure the package is installed in the venv, at the version
> > +    #   requested in requirements.txt, even if already available on the host.
> > +    pip install -U -r ./src/u-boot/test/py/requirements.txt
> > +fi
> 
> So in summary, with the Python 3.5 fix I already sent, then this series,
> Tested-by: Stephen Warren <swarren at nvidia.com>

I appreciate it, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191023/4b65a1f2/attachment.sig>