[U-Boot] python3 support for pylibfdt

[Simon Glass]

Hi Tom,

On Fri, 28 Jun 2019 at 06:51, Tom Rini <trini at konsulko.com> wrote:
>
> On Fri, Jun 28, 2019 at 01:38:01PM +0100, Peter Robinson wrote:
> > > > > > With the EOL of python2 soon I've been looking at the Fedora U-Boot
> > > > > > builds to see what it would take to move over to python3. There's a
> > > > > > couple of issues building the bundled pylibfdt, the first is the
> > > > > > Makefile hard codes python2, the second is that the generated
> > > > > > libfdt_wrap.c doesn't seem to find the python3 version of Python.h
> > > > > > (errors below).
> > > > > >
> > > > > > It seems upstream now supports building pylibfdt with dtc 1.5.0 but I
> > > > > > couldn't quite work out how this fits into the U-Boot bundled version.
> > > > > > Is there plans to be able to support pylibfdt with python3?
> > > > >
> > > > > Sounds like we need to run the normal kernel script to re-sync with
> > > > > upstream?  Thanks!
> > > >
> > > > Seems reasonable, I'll keep an eye out for a patch series to test,
> > > > it's quite straight forward to test from my PoV.
> > >
> > > It won't be any time soon, sadly.  Updating to the same dtc in the
> > > kernel (so just v1.4.7+) causes both massive amount of new device tree
> > > warnings as well as several fail to link due to size growth problems.
> >
> > I'm guessing the size problem is due to an increase in size of where
> > libfdt is linked in and not due to pylibfdt. Would it make sense to
> > have a feature branch with the rebase to make it easier to test/fix
> > the issues? And rebasing some DT to the current kernel versions should
> > fix up a bunch of the DT problems?
>
> The size growth is due to the C side growing, yes.  Some of it is the
> safety checking I warned about back at the time and the rest of it is
> just general growth (U-Boot is guilty of that all the time, so I can't
> really complain about some other project growing slightly).
>
> Rebasing to the current kernel version adds a bunch of DT problems, some
> of which I hope would be fixed by re-syncing the base DT at least.
>
> > Fedora, as are many other distros, is actively retiring python2 due to
> > it's upcoming EOL ~ 6 months from now. I've had to already rescue a
> > couple of python2 packages to keep U-Boot building, there's an
> > intention to actively remove python2 in Fedora 32 (scheduled for May
> > 2020) which means anything post U-Boot 2019.10 (the version we're
> > aiming for in F-31) will start to cause me big problems.
> >
> > Maybe we could add this to migration plans like any of the DM subsytems?
>
> So, looking again, the problem is that upstream, pylibfdt/setup.py is
> still using python2.  So while a resync with upstream dtc might be good,
> it won't solve this issue.  I'm not even sure why it doesn't work with
> Python3, other than that in January we added a patch to ensure we used
> Python2 and not 3.
>
> Simon, I don't see your series that updated a bunch of stuff to work
> with Python 3 in master, but it's listed as Accepted in patchwork, do
> you know what happened?  Thanks!

There's a patchwork bug that breaks my script for accepting bundles.
So when I applied these patches to u-boot-dm/next I just replied
'accepted' to the cover letter emails.

Unfortunately libfdt added various security checks which blow out the
code side. I saw it at the time and tried to suggest that David could
make them optional, as did Tom, but he was not willing to do that. So
for now we have a libfdt which we cannot use in U-Boot. It needs some
work, unfortunately.

Re Python, pylibfdt supports Python 3. At least with upstream dtc I
can build for both.

I've converted patman, binman, etc. to Python 3 and should be able to
pull these patches in when the release finally goes out.

If no one screams I'd like to apply a patch to make these tools use
Python 3 instead, in time for the next release (October?!). Given that
the release after that might not be until next year, I might try to do
that change in the same release. There is good test coverage for most
of it.

Re the other Python scripts in U-Boot, there are a few that are used
by builds. The make_fit_atf.py should move to binman, I think. Not
sure about the synopsys ones.

Regades,
Simon