[PATCH v3 30/31] bootstd: doc: Add documentation

[Tom Rini]

On Fri, Jan 21, 2022 at 02:15:24PM -0700, Simon Glass wrote:
> Hi Tom,
> 
> On Fri, 21 Jan 2022 at 12:23, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Fri, Jan 21, 2022 at 12:14:22PM -0700, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Fri, 21 Jan 2022 at 11:09, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Fri, Jan 21, 2022 at 09:02:13AM -0700, Simon Glass wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Fri, 21 Jan 2022 at 08:31, Tom Rini <trini at konsulko.com> wrote:
> > > > > >
> > > > > > On Fri, Jan 21, 2022 at 08:20:17AM -0700, Simon Glass wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > On Fri, 21 Jan 2022 at 08:08, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Wed, Jan 19, 2022 at 12:39:03PM +0100, Heinrich Schuchardt wrote:
> > > > > > > > > On 1/19/22 02:43, Simon Glass wrote:
> > > > > > [snip]
> > > > > > > > > > +Introduction
> > > > > > > > > > +------------
> > > > > > > > > > +
> > > > > > > > > > +Standard boot provides a built-in way for U-Boot to automatically boot
> > > > > > > > > > +an Operating System without custom scripting and other customisation. It
> > > > > > > > > > +introduces the following concepts:
> > > > > > > > > > +
> > > > > > > > > > +   - bootdev  - a device which can hold or access a distro (e.g. MMC, Ethernet)
> > > > > > > > > > +   - bootmeth - a method to scan a bootdev to find bootflows (e.g. distro boot)
> > > > > > > > > > +   - bootflow - a description of how to boot (provided by the distro)
> > > > > > > > > > +
> > > > > > > > > > +For Linux, the distro (Linux distribution, e.g. Debian, Fedora) is responsible
> > > > > > > > > > +for creating a bootflow for each kernel combination that it wants to offer.
> > > > > > > > >
> > > > > > > > > This gets it completely wrong. There is one standardized boot flow: UEFI.
> > > > > > > > > All major distros support this. U-Boot has to offer UEFI booting out of the
> > > > > > > > > box.
> > > > > > > >
> > > > > > > > I want to jump up and down and emphasize this part as well.  While I
> > > > > > > > believe our UEFI bootmgr is still missing the normal scan code, that's
> > > > > > > > something that has been promised to be implemented.  And that turns the
> > > > > > > > bootcmd for platforms that just want to support modern off the shelf
> > > > > > > > distros in to something fairly small.
> > > > > > >
> > > > > > > Sigh...
> > > > > > >
> > > > > > > UEFI is a bootflow in this model, one of many. If we don't support the
> > > > > > > others, then U-Boot is not U-Boot anymore, it is just EFI Boot.
> > > > > >
> > > > > > No one is talking about removing anything else.  But a major part of
> > > > > > your motivation here seems to be "discovering what to boot where is a
> > > > > > pain" and that's solved (or at least defined, I'm poking Ilias about the
> > > > > > status of that off-list).  And I want to emphasize discover.
> > > > >
> > > > > But only if you use EFI boot manager, right? Even then I'm not sure we
> > > > > have a deterministic way of listing the available bootdevs, which is
> > > > > something that this series provides.
> > > >
> > > > I'll let someone else that knows the EFI boot manager code / design
> > > > speak to this.  But yes, for the discoverable case I'm not seeing where
> > > > "use EFI boot manager" isn't the reasonable answer.
> > >
> > > With bootdev you have a proper driver and device tree node where
> > > configuration can be provided. The default ordering for bootdevs can
> > > be provided. We can deal with the quirks of particular subsystems,
> > > like MMC. I think there will be other benefits apparent as this all
> > > matures.
> > >
> > > But let's see. Perhaps it doesn't really matter anyway, since it seems
> > > that EFI boot manager is doing its own thing for now.
> > >
> > > >
> > > > > > > If we get EFI bootmgr going, then are you saying you want to disable
> > > > > > > everything else?
> > > > > >
> > > > > > Not at all.
> > > > >
> > > > > OK, good.
> > > > >
> > > > > >
> > > > > > > You say 'major distros' but there are many that don't use it,
> > > > > > > particularly in the embedded space. I'll go out on a limb and say that
> > > > > > > the vast majority of embedded devices in the world don't use it. Are
> > > > > > > you really saying we should drop support for everything else? Even the
> > > > > > > distro stuff supports other options.
> > > > > >
> > > > > > I don't know about buildroot off-hand, but I've had OpenEmbedded spit
> > > > > > out UEFI-compatible aarch64 images no problem.  If you're talking about
> > > > > > embedded Debian/Ubuntu/Fedora, that goes back up to "wants UEFI boot
> > > > > > flow".  Armbian is the biggest distro I know of off-hand that doesn't
> > > > > > do UEFI boot for U-Boot targets and I would love to talk with someone
> > > > > > there and find out why (but I guess it's all the 32bit platforms).
> > > > > >
> > > > > > But I'd also say the vast majority of embedded devices don't need the
> > > > > > complexity you're adding here, but DO need the ability to implement A/B
> > > > > > things as easily in U-Boot as they can in grub.  And that in turn is
> > > > > > because it's a pain to modify the default environment in U-Boot and easy
> > > > > > to drop in another script for grub.
> > > > >
> > > > > My feeling is the complexity is already there, just in scripts, which
> > > > > are harder to understand (from personal experience trying to
> > > > > understand what they do) and don't have tests. They are also very hard
> > > > > to build on top of (e.g. verified boot).
> > > >
> > > > Yes, the scripts that live in the environment based boot flow is
> > > > complex.  It's also been a huge step forward from what we had before and
> > > > has been a great help.
> > > >
> > > > > I can't really say that this series is more complex than EFI bootmgr,
> > > > > if that is what you are suggesting. I think the bootdev uclass is
> > > > > well-motivated and will prove useful even for EFI.
> > > >
> > > > No, what I'm suggesting is that I see this as "current boot script stuff
> > > > is too complex, lets replace it".  And I also see the big part of the
> > > > complexity there being the discover where to boot from side of things.
> > >
> > > OK, so shall we replace the scripts, or not?
> >
> > I'm still looking to be convinced that something is better than the
> > scripts, or that the problem is the scripts and not the pain involved
> > with modifying the U-Boot environment.
> 
> Well luckily, someone has gone to all the trouble of providing an
> alternative that we can try :-)
> 
> >
> > > > > Also A/B/recovery is a lot easier to implement in code than in
> > > > > scripts. I have linked to the proposed design there.
> > > >
> > > > Show me what implementing Mender or RAUC or swupdate looks like with
> > > > your proposal.  Those are some of the real A/B use cases today and have
> > > > had to take various approaches to dealing with our environment, and also
> > > > supporting x86 and so started dealing with grub.
> > >
> > > That sounds like an interesting project. For mendor I found this:
> > >
> > > https://github.com/mendersoftware/meta-mender/blob/master/meta-mender-core/recipes-bsp/u-boot/patches/0002-Generic-boot-code-for-Mender.patch
> > >
> > > More scripts...
> >
> > Yes.  Mender, RAUC and swupdate (iirc) all have some form of environment
> > based bootcmd to Do The Right Thing and boot A or B or detect failure
> > and fall back.  Show me how what you're doing improves integration for
> > that case.  That's a case that's not covered by "use UEFI boot manager"
> > directly.  I know for Mender a huge pain point for integration is "drop
> > _this_ script in to the default environment".  RAUC is similar.  Show me
> > something that makes that much easier to integrate, like it is with
> > grub.
> 
> The obvious answer would be to create a boot method for Mender. In
> that you can do anything, including using the standard bootmethods to
> obtain things to boot, etc. Also perhaps a 'recovery' bootdev which is
> invoked last, when other things have failed, or first if recovery mode
> is selected. Both could be done without any scripting, so could be
> enabled on any board with just enabling CONFIG_MENDER, I suspect.

OK.  So please show that this will improve things for this case, rather
than speculate.

> > > > > Anyway if we can agree that we are not going to disable the non-EFI
> > > > > flows, then how about we focus on replacing the distro boot scripts,
> > > > > dropping the config.h files, and leave EFI bootmgr out of this
> > > > > discussion?
> > > >
> > > > The primary use case for the distro boot work is EFI boot, and the "make
> > > > this logic clearer" solution is "use EFI bootmgr".  That's where we get
> > > > stuck in a loop here.  There's no "the distro must create .." because
> > > > the distro is already creating what's needed.
> > >
> > > So let me ask again. Is EFI bootmgr the only thing U-Boot is going to
> > > support? I thought you said no.
> >
> > Maybe we're talking at cross purposes.  I'm not going to drop
> > booti/bootm/bootz/bootelf.  I'm not going to drop setting bootcmd to
> > whatever the user / board developer knows is best for them.
> 
> Maybe. My confusion is that EFI seems to be blocking progress in the
> general booting domain. When innovation is suggested, the answer is
> 'well EFI does this so why bother'. When the boot scripts are
> mentioned, we can apparently limp along with them and iterate there,
> since EFI is the only thing that matters anyway.
> 
> Perhaps the real difference here is that I am not focused on EFI as
> the solution to all things. I know that is where many distros are and
> I believe you feel I am wasting my time. But that is where I am. From
> my vantage point I see a lot of improvements we can make with booting,
> independent of EFI.

My high level concern is that yes, I don't see this as improving
anything yet.  This isn't an improvement over what EFI has for booting
generic distributions.  And this one of the areas where the boot scripts
complex, but can be simplified.  Another area where they could be
simplified is for any of the existing A/B style update mechanisms.
That's an interesting case that could show value here.

> > > Are you saying you want to keep the environment boot scripts, or not?
> >
> > As I said in the previous iteration on this series, I'm not convinced
> > this is a win over other clean-ups that can be done with what we have
> > today, or that it solves the problems that I often see popping up.
> 
> Previously you were worried about the size growth. I am pretty
> confident this will make a lot of boot-related things more structured
> and easier over the medium term. But we do need something to build on.
> I believe the concepts are sound, and applicable to various boot
> scenarios. In fact I think the lack of a 'bootdev' is something we
> should have thought to create a while back.
> 
> I certainly think it is better than building more and more on the boot scripts.

Alright, so rpi_3_32b grew as:
       arm: (for 1/1 boards) all +6908.0 data +920.0 rodata -2664.0 text +8652.0
            rpi_3_32b      : all +6908 data +920 rodata -2664 text +8652
               u-boot: add: 101/0, grow: 3/-2 bytes: 8662/-160 (8502)
with this series, and I corrected the minor problems due to '@return'
becoming "Return:".  That's a problem.  Environment space is cheap (it's
a file, or it needs to have some pretty big these days alignment
requirements to have physical redundancy, if you need that).  The
biggest problem I see with boot scripts is that defining them in a
header file where we have to deal with escaping stuff in a header, and
you resolved that with the environment cleanup series you did before.
The series (that I keep failing to find the time to reply to, but was
happy to see!) that updates hush to a modern copy is another good step
in the right direction.  Being able to write the script but more as
plain text rather than escaped strings in C is I suspect why Armbian
does what it does.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20220121/718db391/attachment.sig>