[ANN] U-Boot v2025.07-rc3 released

[Tom Rini]

On Mon, May 26, 2025 at 04:03:11PM -0600, Tom Rini wrote:
`
> Hey all,
> 
> So it's release day and I have tagged and pushed things out. I will be
> updating the next branch shortly. There's still a few things that need
> to come in for the release, as fixes, but otherwise I think the next
> branch is where most changes should end up.
> 
> We're continuing with a community meeting following the release and is
> the same time as the previous meeting. The meeting details itself are:
> https://meet.google.com/btj-wgcg-euw
> May 27th, 2025. 9am (GMT -06:00)
> 
> To join by phone:
> https://meet.google.com/tel/btj-wgcg-euw?pin=1307528552322&hs=1

For today's meeting it was suggested to try and use Google's
transcription service to take notes. What follows is a lightly edited
copy of the transcript. I will say it took a good amount of time to read
and re-edit. I will follow-up with a lightly edited Gemini summary, just
to finish evaluating this idea.

Attendees:
Andre Przywara, Andrew Goodbody, Bryan Brattlof, Gopinath sekar, Ilias
Apalodimas, Jesse T, Mattijs Korpershoek, MV, Rogan Dawes, Simon Glass,
Tom Rini, Yao Zi

Transcript
Tom Rini: So yesterday was release day. I put out RC3. I believe overall
things are in a good state. the one thing I know I need to take in next
is that the revert I posted for the ext4 overflow issue as unfortunately
it leads to different problems. That's on me for taking it so quickly.
Sorry about that. looking forward into the next window next branch
rather.
Tom Rini: so I see that kernel release v6.15 is now in the device tree
rebasing repository as well. So I plan to sync that up and throw it at
CI today. I already did the first half of it. There's a couple of real
quick merge conflicts I need to resolve with copy. let's see.  So we are
what little over a month away from the July release. So I would
encourage everyone to test the current state of master on whatever
hardware they can easily test things on.
Tom Rini: if there are regression fixes that people are aware of, please
reach out to me with a link either in chat here or IRC or email so I can
make sure it doesn't get lost and I will make sure it doesn't get lost.
with that, are there specific things people would like to talk about?
Ilias Apalodimas: I can start the discussion on your preference of the
makefiles because it's a mess anyway the Kbuild system.  I mean, I've
sent the first round which was I don't know basically me porting patches
back typing make looking at the patches and getting further than that is
a bit problematic right the number of patches is huge because the last
thing was on v4.20 and it's not a full thing anyway so there's things
missing to move forward from v4.16
Ilias Apalodimas: I've pasted you some of the patches yesterday, So
there's two options. I either do it incrementally so we can read the
commit messages and review them, but this is going to take a long time
or I just do a diff and I try to bring it up to speed and the commit
message is going to be unreadable. the parts, right? So I don't know
what everyone prefers. I have been moving it forward. I'm trying to sync
it up to v5.1, but again, if I have to do patch by patch, it's going to
take ages.
Jesse T: You're updating the Kbuild system, I take it.
Ilias Apalodimas: Yeah. Yeah.
Jesse T: Okay.
Simon Glass: From my point of view important thing is to update it not
to have a commit history. And you're talking about going back years,
right?
Ilias Apalodimas: Yeah. It's more than four years. Simon Exactly.
Jesse T: I was wondering when I was going to get updated.
Ilias Apalodimas: I can makefiles. that's the easy part. if no one
minds, I can just send the parts I have once I make it pass Power PC or
whatever is failing, right? Because the Partial linking has gone away.
There's a few things that changed over the years. Now, I'm not sure
about partial linking because I was planning on using it on EFI, but
that's fine. And I can plug it on the EFI only. I can send the be if Tom
how confident are you the CI. if that thing passes it mean it means our
CI build is fine.
Tom Rini: So, first, when we've done things up, historically, it hasn't
really been a reviewable diff anyways because it's been grabbing
everything that appears to have been missing from one kernel release
over, the next one we're syncing up with. So, I think it's all right if
we have just several large patches so long as they're at least
biseectable for builds.
Ilias Apalodimas: Yeah. Yeah. Yeah.
Tom Rini: And then when it does come to so what do we do with the
output? There's definitely going to be a few gaps just because there are
some platforms that don't make use of the flag to say hey here's what my
actual final build target is and instead the build instructions for them
are first configure run make all then run make you do whatever to get
the actual resulting binary.  I'm not overly concerned with those just
because I believe we'll just need to do a little bit of kind of manual
spot checking and then ask people to go ahead and test them out. The
biggest offenders there, I believe, are the IMX family. So, it's also
something that's fairly straightforward to spot check at least.
Ilias Apalodimas: Yeah. So the plan I had is this I have the patches I
sent you which work fine and I've tested them enough and I think it's
fine to pull them into next. Then I have another path that basically
brings the cable bill to 5.1. The CI is not passing on that. I have to
fix the remaining CI issues. But my plan once I reach 5.1 is to split
off all the make files to autmake file that we can include and then just
do the dig 6.15 and that should be substantially easier. That's the
plan. Now if this is going to work
Tom Rini: Okay. Yeah.
Ilias Apalodimas: I have no idea. but that's the best thing I found, The
only downside is that you're going to get a makefile patch of 300 lines
that's going to be unreadable.
Tom Rini: If we can do things such that it will be easier to do future
reyncs that would also be good and breaking out as much of our specific
logic as possible into a standalone file will aid with that.
Ilias Apalodimas: Yeah. Yeah.
Tom Rini: I definitely agree. We should be able to get what we've done
so far and then what you have into the current next window and out for
the October release and then sticking up on top of that. Definitely see
how long it takes to get v6.15 or so working.
Ilias Apalodimas: It shouldn't be too far away, If I spend a couple weeks
Tom Rini: Okay.
Ilias Apalodimas: I think I'll be fine. But it's just painful. you add
something, you look at object files, try to figure out what's going on.
It's a bit painful, but overall, I have it passing for most of the ARM
boards.  Sandbox is working. some DTS are blowing up for Xilinx boards,
and I haven't figured that I probably saved off a rule I shouldn't have.
and then it's the Power PC, but I'll leave that for last. but overall I
mean overall I changed the targets to v5.1 I changed everything to v5.1
once I reach v5.1 I'll do a refactoring so we can split off the hub
stuff
Tom Rini: Okay.
Ilias Apalodimas: then if my theory is correct we should be able to jump
to v6.15 a relatively painless move
Tom Rini: Yeah, I definitely look forward to you being able to evaluate
your theory in practice. Yeah, thank you for digging in on all the
Kbuild stuff.  It's greatly appreciated and it really will be nice that
there's light at the end of the tunnel and we'll be able to just go with
hey colonel release let's reync again and not get years out of date.
Ilias Apalodimas: So my whole point is to fix LVM on that, because I'd
like to compile with LVM because it's catching some bugs that are not at
least on the GCC does the same job, But does it's a bit stricter on the
default flags. and the kernel has this nice it says LLVM equals one and
then it use all of LLVMs and flags assemblers linkers and everything
while the only thing we do is use the compiler right now with our make
files I want to use the whole thing but yeah I'll get Yeah.
Tom Rini: Yeah, that definitely be nice.
Tom Rini: And then another thing for people to integrate into their
labs. I know it's been a good number of years since I was able to check
for Pi, but mostly built with clang and see how it worked because
there's some bugs.
Ilias Apalodimas: It does pop errors about unaligned struct members that
are indeed errors. and nothing is blowing up because most of the CPUs
today do unaligned access is fine.
Tom Rini: All right.
Ilias Apalodimas: But yeah I will get there once I fix the build system
first.
Tom Rini: Anything else to talk about on the Kbuild side of things? If
not, that does remind me that I wanted to ask if you had any chance to
further delve into just what we need to do about that QEMU issue on
vexpress_ca9x4
Ilias Apalodimas: No. I'll have a look. I really don't know what's going
on there, right? Because in theory, unaligned accesses should be up and
running at that point. and from what I've looked, it's not EFI that's
crashing because that's what I figured out from the bug report. It's the
TFTP that's crashing.  So my local reproduction, I didn't even manage to
launch the application. The moment I TFTP it just blew up.
Tom Rini: Which is strange for a lot of reasons.  I just gonna say I'm
still not entirely unconvinced that we don't go back to QEMU and say we
dug into this more and know really why is this happening the way it
does? Just for more context for everybody else trying to upgrade CI from
QEMU 9.0.0 to a more current release introduced two regressions or
there's two problems in One of which is specific to a sifive platform in
the SD card emulation
(https://gitlab.com/qemu-project/qemu/-/issues/2945) The other one is
that the V Express Cortex A9 platform fails on a specific test and boot
blows up (https://gitlab.com/qemu-project/qemu/-/issues/2944). Peter
looked into it and one of his comments was that QEMU aims to I don't
think I best phrase it, but it's allowed to do things that real hardware
may not have done but still be correct. one of his thoughts was that
this test should actually fail on real hardware as well, but as far as
I've been able to test and get other people to try out and test as well
on similar real hardware that particular Cortex A9, it's fine on real
hardware. So, it really is a funky thing that QEMU is doing. Maybe it's
really a funky thing we're doing. It's not getting there.
Ilias Apalodimas: So Tom, I talked to Peter in connect and I also talked
to Richard Henderson for QEMU. they believe this could also blow up in
A15 not's due to the architectural things that are enabled. on the other
hand Peter directly went and said this function calls Peter assumed that
this is because of an aligned accesses and he also assumed that the MMU
is not enabled. right now. I don't know if I have the reproduction
wrong, but I didn't even get to initializing the FI subsystem. The thing
crushed when I TFTP something. So, I don't think it's that. And I think
that his assumption by reading the code because the guy didn't run it,
He just read the code and tried to figure out what's going on. I think
we just started from the wrong assumption. I'll try to find some time
and debug it by the end of the week.
Tom Rini: Okay. Yeah, for both of the issues I provided, here are the
steps to reproduce a crash in hopes that someone fire up GDB and Q side
of things and see what's going on. But yeah, it's only, inspecting some
aspects of the code and not rerunning it. It's entirely impossible to do
an assumption. Okay. Thank you. are there new topics people would like
to bring up?  So from the chat, does that also happen on sandbox running
on a armv7a? this particular blowup. I don't know that we've actually
run sandbox on an armv7a host just because I know we don't do that in
CI. Part of that would be because the sandbox tests take so long to run
on anything that's not really really fast. And the next problem is we
don't have armv7a hosts in CI just because again platform be rather
slow.  So in chat (from Marek Vasut) the comment was that would make it
easy to test on sandbox and do we want to sync up off list to talk about
this a bit more I can show you the reproducers? OK yes, so we'll talk
about this a bit more on IRC, thank you. Are there other topics people
want to bring up?
Jesse T: I have a quick question. so for CI, is there a way to make
reproducible builds how buildroot kind of does where it compiles the
compiler and a bunch of other stuff like that or no?
Tom Rini: We don't go to that level of reproducibility with CI and part
of the pluses or minus there is that we do have the docker file which in
turn says we're going to grab some existing well-known pre-built tool
chains so that we can not have to worry about hey did something go wrong
with building a compiler I guess I will as an aside mention that's much
less of a concern these days than it was 20 years ago, but yeah in terms
of reproducibility build support we do have things like being able to
say here is the timestamp to use for any sort of build and we kind of
rely on whatever external build system is building us for handling the
more general build.
Jesse T: And another question, if I was going to set up a CI bot to test
on boards, is there has someone made a build bot for that? I know LDM, I
think LV has their own build bot thing. But that's a little complicated
to set up. Has anyone done that for you?
Ilias Apalodimas: I've done build in the past.  It's not that
complicated, but it's just annoying because the configuration is mostly
in Python.  Other people find it it's fine. I just find it annoying. But
it's pretty reliably.
Jesse T: Can you share that with me? Is that public or ?
Ilias Apalodimas: Yeah, I can share it. I mean, it's not public, but
it's a file. I can send it over.
Jesse T: Okay, thank you. that would be great.
Tom Rini: I was just going to note that I don't know if anyone else has
done buildbot. I know that Heiko has done tbot and has a bunch of that
stuff configured and couple months ago at this point also posted some
patches for here's how to plug bot stuff into the GitLab CI pipeline we
have
Jesse T: Okay. I'm specifically using it to test boards, so I don't know
if it's easy to transfer to runtime testing versus buildtime testing,
especially because runtime testing you don't get a lot of verbosity. It
kind of just fails silently typically.  Tom Rini: So, this was just to
be clear how to have, various boards in his hardware lab get a build,
throw it on, run some tests, and come back.  Jesse T: Okay.  Ilias
Apalodimas: Send me an email because I'm ound to Send me an email and
I'll respond with the billboard confluence.
[edit: Removed some confusion about copying out Ilias' email address].
Jesse T: Also, one of my professors at school found a bug in, DE1 SOC. I
can't reproduce it, which is unfortunate because it's a compiler issue,
which is why I asked about the reproducibility thing. so this was
actually like last year he found this and it's still broken for him. and
another student actually compiled it the same way and also broke but
basically he compiled the compiler himself. He compiled yes that the
Altera thing this is on one person did it on Fedora but also compiled
the compiler themselves but my professor did it on Arch and it's a
runtime thing but if you build root to compile it with the latest deboot
it works fine and on De using the Debian's built-in compiler it works
fine but using that compiler doesn't work. But if I try and compile the
compiler on Debian, it works.
[edit: For clarity, Marek was making some comments in chat here about
the problem].
Tom Rini: Yeah. Any chance you can get the failure log post and post it
somewhere.
Jesse T: Yeah, it does the SPL and then nothing after that.
Tom Rini: And also I guess maybe a log of the building you with that
compiler just in case there's some warnings being thrown up and not as
visible as they should be.
Jesse T: As far as I'm aware, no. I can't reproduce it on my end. If I'm
on their computer, I can reproduce it. two people have been able to
reproduce it. Except I can't. I've been trying to fix it. I have the
board, but I can't. even in an Arch Docker container, it doesn't work.
And I'm like, what? Mhm.
Tom Rini: Yeah, this definitely reminds me of some of the user induced
errors I've seen over the years, but it would be interesting to see what
the logs are.
Jesse T: Where it failed on try to build but the D1 SOC that's a
different thing. yeah. I was just bringing that up??? because compiler
issues. if anyone has the board, maybe they can try to figure it out. I
can send the build script.
Tom Rini: Any chance you also have the script for how everyone is
building the compiler?
[edit: Jesse T has access to the scripts being used]
Tom Rini: Yeah, that'd definitely be an interesting to look at. And I
feel like the end result here might be filing another issue on the GCC
Bugzilla, seeing what happens.
Tom Rini: is there anything else to handle on the call or should we
finish it up afterwards? Okay. Are there other topics people would like
to bring up?
Ilias Apalodimas: There's a CI discussion that Simon asked for yesterday
about running an OS. that's doable. there's one module missing from the
kernel config. I've only worked with Fedora to add that. I tried asking
on Alpine, no one responded and then I just ignored them. I'm pretty
sure that the distro cases involving the one for the phones which is
based on Alpine supports it. If anyone is interested I can send him the
commit message you need a very specific kernel module enabled but if you
enable that kernel module then we can basically boot Alpine on the fly
in DRAM. and it's really easy and really fast to test.  We're working
with Yocto to do the same thing. I lost track of where this is. but if
you don't care about Alpine specifically, in a few months or weeks or
whatever, we'll probably be able to test with a generic Yocto image as
well.
Simon Glass: Yeah. the image I was looking at was it Alpine?
Ilias Apalodimas: you said Alpine on the email and it makes sense
because it's very small but yeah it's fine
Simon Glass: Yeah. 250 megs or so. and I think Tom's response was that
that was sort of okay.
Ilias Apalodimas: but it depends what you want to test right because If
you boot with EFI, what happens is that you create the ISO image in
Uboot, you jump there, but once you jump into the kernel calls exit boot
services and the image you just create, it just disappears. So, You can
boot the kernel, but you won't reach a login.
Simon Glass: I see.
Ilias Apalodimas: So, has all the patches it needs. I mean, I think I
sent a pull request to Tom for that a month ago or something like that.
I think it's in master the obvious problem is that in order to do that
you need the kernel to support the same feature which is basically
preserve a memory node. I'm not sure how many dual support it does
because I added it. I tried to talk to Alpine and they never responded.
So I was like all right I'm going to fix it somewhere else.
Simon Glass: So there is the only test I have at the moment and Tom's
pointed out that there are the network tests that run in the QEMU in CI
but the one not QEMU in some boards or something. the one I have at the
moment is a Ubuntu 24.04 desktop which is 5 GB and that's running on my
lab. so it doesn't matter how big it is because it's just sitting on a
local disc somewhere.  And basically all that does is it boots up. This
is running the installer by the way, not the real OS, just the
installer. And then at some point it says welcome to Ubuntu 24.04 and
then the test passes. is you I don't care after that kind of thing.
Ilias Apalodimas: So Tom, I can add instructions somewhere on how to do
that over HTTP. I mean, it's literally a single command. you just need
the right OS. I can even send you an OS to test with for QEMU and I can
some somewhere in IRC on in an email explain what modules need to be
enabled.
Simon Glass: Is it?
Ilias Apalodimas: I think someone added it on Debian. I haven't tested.
I fixed you. That's all I could do. Right. If this can follow up that
would be great. If they can't that's unfortunate.
Tom Rini: So I guess kind of the high level question is what is it we
are wanting to be testing here? And what I've long wanted and we do have
some ways of doing now is U-Boot able to start an OS successfully? And
for that, it's a much more on the one hand, it's a much simpler test
because we really just need to make sure that we get some known line out
of whatever OS kernel we're testing. It says, "Hey, I've gotten far
enough that the bootloader is likely not going to be an issue. We're not
trying to have an entire compliance suite run every time, especially
since runtime services are their own kind of special challenge." Kind of
the next part of the I know it would certainly be good for other labs to
do more intensive testing infrequently but still at a regular interval
but just kind of for every time CI you definitely also have to worry
about how long is that going to take and what value are we going to get
out of that too and then kind of the next set of challenge is also if
we're going to start including a distribution image and kernel. what's
it actually like to support hardware wise?  Because on one hand we have
two virtualized platforms. On the other hand everybody has their own
hardware lab and everyone is aware of the levels of pain involved with
hey is this particular board actually going to be functionally usable
with this particular distribution.
Simon Glass: I mean, my question was really just what can I put in CI?
and maybe if you could send an email about the problem you're talking
about because I don't really get it.  But, from my point of view, just
being able to boot and see welcome to Ubuntu or welcome to Alpine would
be good enough on the serial console.
Ilias Apalodimas: So the question is do we want to do it now?
Because I have one of my team members fixing exactly that adding the
modules in. Once this is merged you'll get a nightly build for Yocto
that works. Right. If that's good enough fine. If not, I can send an
email of what's needed and someone can enable it in Alpine or Ubuntu or
whatever and then we can test with that.  Right? But the whole point is
that from Yubot's point of view, we did exactly what ACPI does to solve
the problem. booting from a USB disc.
Simon Glass: But sorry, I actually don't understand the problem because
I have a test that boots. So, what is the problem you're talking about?
Ilias Apalodimas: [edit: Booting from USB is] fine, If you're booting
from network or an ISO, that doesn't work because the ISO gets mounted
in RAM, gets abstracted as a disk, but once exit boot services is
called, that disc goes away. If it's a USB disc, Linux just rescans it
and happily continues, but the problem is that on a CI you can't expect
to, write the OS all the time or write a USB disc all the time. your
best course of action on a CI is to be able to boot everything from
DRAM.
Simon Glass: I see.
Ilias Apalodimas: And that's what this fixes.
Simon Glass: I was assuming we could just if it's in the Docker image,
just run QEMU with the ISO.
Ilias Apalodimas: That will work as well. Yeah.
Simon Glass: So, if we're okay with having it in the Docker image, that
was really what I was getting at, then I think I could just do that with
Alpine because if it's small enough. I mean,
Tom Rini: that's worth doing just because it gets down to the next
problem of okay, so we can run that on one image. if we're going to
include an image, we want to make sure it's as small as possible. We
want to make sure we can get something as widely usable as possible and
a network, if it's also loaded over the network, that's a lot easier to
deal with rather than, okay, now we also need to make sure we're passing
the right things to QEMU. That's one more little thing to configure.
That's what I was trying to get at with, hey, let's look at network
testing rather than hard coding passing some device to each QEMU
instance we're going to try and test this on.
Ilias Apalodimas: Simon, the network one is probably going to be even
easier than the disc one. It's literally a single command. So, I don't
mind.
Tom Rini: We're still going to have to include the image inside the
Docker image so that it's not something that we have to spend some
amount of time fetching every time we're running a test.  It's just a
file that already exists local to CI.
Simon Glass: But I mean, it sounds like you're already working on this,
Ilias. So, if that's all right, I might just do nothing and wait for you
to
Ilias Apalodimas: I was so by Simon. I don't mind. If you want to do it,
I can help. I'll wait until the ARM 64 image is fixed and I'm probably
going to plug that in or maybe not me someone from my
Simon Glass: Yeah. Yeah. let's just do that. I'm in no rush. I was just
thinking, what to do. I've just logged into that ARM 64 machine to see
Tom Rini: Sounds good. Thank you both.
[edit: Simon and Ilias now start talking about the U-Boot running on
aarch64 host with the -kvm flag to QEMU]
Ilias Apalodimas: I haven't found time to look at it. Sorry.
Simon Glass: if I could repeat that thing you were talking about. The hang
Ilias Apalodimas: It just hangs right and it just hangs only when you
have KVM enabled. I think it's something running out of memory in KVM
with blob list.
Simon Glass: Yeah. Yeah.
Ilias Apalodimas: I couldn't find time to look at it.
Simon Glass: I've got the bug, so might look at it at some point. I've
never tried running QEMU on an ARM 64 machine.
Ilias Apalodimas: Yeah, I ARM sent me a server I don't know four years
ago or something. Ampere sent me one. So it's easy for me to run that
stuff. I just have it locally and I just, SSH and run an instance.
Jesse T: By the way, it was actually the D10 standard, not D1 SOC. We
were using both boards for different things. So I also sent all the pkg
builds and on IRC I used pastebin. I don't know if I should have used
something else or
Tom Rini: Pastebin is fine. Thank you.

And that was the end of the meeting.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot-custodians/attachments/20250527/c467c780/attachment-0001.sig>