[U-Boot] [PATCH 04/10] test/py: Automated conversion to Python 3

[Tom Rini]

On Sat, Oct 19, 2019 at 07:25:37AM +0200, Heinrich Schuchardt wrote:
> On 10/18/19 10:53 PM, Tom Rini wrote:
> > Use the 2to3 tool to perform numerous automatic conversions from Python
> > 2 syntax to Python 3.  Also fix whitespace problems that Python 3
> > catches that Python 2 did not.
> > 
> > Signed-off-by: Tom Rini <trini at konsulko.com>
> 
> This patch breaks
> 
> ./test/py/test.py --bd=qemu-arm64 --build-dir=. -k=test_efi_
> 
> For remediation see below.
> 
> > ---
> >   test/py/conftest.py               |  9 ++--
> >   test/py/multiplexed_log.py        |  4 +-
> >   test/py/test.py                   |  4 +-
> >   test/py/tests/test_fit.py         |  2 -
> >   test/py/tests/test_fpga.py        | 46 +++++++++----------
> >   test/py/tests/test_fs/conftest.py |  4 +-
> >   test/py/tests/test_log.py         |  8 ++--
> >   test/py/tests/test_mmc_wr.py      | 74 +++++++++++++++----------------
> >   8 files changed, 71 insertions(+), 80 deletions(-)
> > 
> > diff --git a/test/py/conftest.py b/test/py/conftest.py
> > index 30c898b40a0d..5c19af1d5034 100644
> > --- a/test/py/conftest.py
> > +++ b/test/py/conftest.py
> > @@ -19,13 +19,10 @@ import os.path
> >   import pytest
> >   from _pytest.runner import runtestprotocol
> >   import re
> > -import StringIO
> > +import io
> >   import sys
> > 
> > -try:
> > -    import configparser
> > -except:
> > -    import ConfigParser as configparser
> > +import configparser
> > 
> >   # Globals: The HTML log file, and the connection to the U-Boot console.
> >   log = None
> > @@ -169,7 +166,7 @@ def pytest_configure(config):
> > 
> >           with open(dot_config, 'rt') as f:
> >               ini_str = '[root]\n' + f.read()
> > -            ini_sio = StringIO.StringIO(ini_str)
> > +            ini_sio = io.StringIO(ini_str)
> 
> This results in an error
> 
> INTERNALERROR>   File "/home/user/u-boot/test/py/conftest.py", line 169,
> in pytest_configure
> INTERNALERROR>     ini_sio = io.StringIO(ini_str)
> INTERNALERROR> TypeError: initial_value must be unicode or None, not str
> 
> -            ini_sio = io.StringIO(ini_str)
> +            ini_sio = io.StringIO(ini_str.decode())

With the whole series applied:
INTERNALERROR> AttributeError: 'str' object has no attribute 'decode'

> >               parser = configparser.RawConfigParser()
> >               parser.readfp(ini_sio)
> >               ubconfig.buildconfig.update(parser.items('root'))
> > diff --git a/test/py/multiplexed_log.py b/test/py/multiplexed_log.py
> > index 637a3bd257ba..de0aacc659b8 100644
> > --- a/test/py/multiplexed_log.py
> > +++ b/test/py/multiplexed_log.py
> > @@ -5,8 +5,8 @@
> >   # Generate an HTML-formatted log file containing multiple streams of data,
> >   # each represented in a well-delineated/-structured fashion.
> > 
> > -import cgi
> >   import datetime
> > +import html
> >   import os.path
> >   import shutil
> >   import subprocess
> > @@ -334,7 +334,7 @@ $(document).ready(function () {
> >           data = data.replace(chr(13), '')
> >           data = ''.join((ord(c) in self._nonprint) and ('%%%02x' % ord(c)) or
> >                          c for c in data)
> > -        data = cgi.escape(data)
> > +        data = html.escape(data)
> >           return data
> > 
> >       def _terminate_stream(self):
> > diff --git a/test/py/test.py b/test/py/test.py
> > index a5140945d4b9..0ce1838833f6 100755
> > --- a/test/py/test.py
> > +++ b/test/py/test.py
> > @@ -1,4 +1,4 @@
> > -#!/usr/bin/env python2
> > +#!/usr/bin/env python3
> >   # SPDX-License-Identifier: GPL-2.0
> > 
> >   # Copyright (c) 2015 Stephen Warren
> > @@ -7,8 +7,6 @@
> >   # Wrapper script to invoke pytest with the directory name that contains the
> >   # U-Boot tests.
> > 
> > -from __future__ import print_function
> > -
> >   import os
> >   import os.path
> >   import sys
> > diff --git a/test/py/tests/test_fit.py b/test/py/tests/test_fit.py
> > index e3210ed43fa4..4922b9dcc664 100755
> > --- a/test/py/tests/test_fit.py
> > +++ b/test/py/tests/test_fit.py
> 
> Please, replace all occurrences of
> 
> -            print(base_fdt, file=fd)
> +            fd.write(base_fdt)
> 
> in this file.
> 
> Together with the correction in test/py/conftest.py I now can run
> 
> ./test/py/test.py --bd=qemu-arm64 --build-dir=. -k=test_efi_
> 
> using origin/master + patches 1-4.

Wait, sorry.  You need to have everything applied.  A partial conversion
to python3 doesn't work, there's other changes needed and then show the
EFI problem I hit.

That said, if you re-work what I do in my patch #7 to get our input
generally encoded to fix the "doing str stuff on byte object" problems
and EFI is still happy in the end and we only have the pytest warnings
to fix, I'm fine with that.  I'm not a python expert and am happy to
take the fixes in another direction.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191019/bc20a059/attachment.sig>