[PATCH v4 00/33] Initial implementation of standard boot

Tue Mar 29 03:36:48 CEST 2022

On Sat, Mar 26, 2022 at 02:51:05PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Sat, 26 Mar 2022 at 13:58, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Sat, Mar 26, 2022 at 01:56:36PM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Fri, 25 Mar 2022 at 08:50, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Fri, Mar 25, 2022 at 03:36:24PM +0100, Michael Nazzareno Trimarchi wrote:
> > > > > Hi Tom
> > > > >
> > > > >
> > > > > On Wed, Mar 23, 2022 at 9:07 PM Tom Rini <trini at konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Mar 23, 2022 at 08:57:36PM +0100, Michael Nazzareno Trimarchi wrote:
> > > > > > > Hi Tom
> > > > > > >
> > > > > > > On Wed, Mar 23, 2022 at 8:30 PM Tom Rini <trini at konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Wed, Mar 23, 2022 at 08:21:22PM +0100, Michael Nazzareno Trimarchi wrote:
> > > > > > > > > Hi Tom
> > > > > > > > >
> > > > > > > > > On Wed, Mar 23, 2022 at 7:46 PM Simon Glass <sjg at chromium.org> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Tom,
> > > > > > > > > >
> > > > > > > > > > On Wed, 23 Mar 2022 at 08:05, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > On Sun, Mar 06, 2022 at 05:49:43AM -0700, Simon Glass wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > The bootflow feature provide a built-in way for U-Boot to automatically
> > > > > > > > > > > > boot an Operating System without custom scripting and other customisation.
> > > > > > > > > > > > This is called 'standard boot' since it provides a standard way for
> > > > > > > > > > > > U-Boot to boot a distro, without scripting.
> > > > > > > > > > > >
> > > > > > > > > > > > It introduces the following concepts:
> > > > > > > > > > > >
> > > > > > > > > > > >    - bootdev - a device which can hold a distro
> > > > > > > > > > > >    - bootmeth - a method to scan a bootdev to find bootflows (owned by
> > > > > > > > > > > >                 U-Boot)
> > > > > > > > > > > >    - bootflow - a description of how to boot (owned by the distro)
> > > > > > > > > > > >
> > > > > > > > > > > > This series provides an implementation of these, enabled to scan for
> > > > > > > > > > > > bootflows from MMC, USB and Ethernet. It supports the existing distro
> > > > > > > > > > > > boot as well as the EFI loader flow (bootefi/bootmgr). It works
> > > > > > > > > > > > similiarly to the existing script-based approach, but is native to
> > > > > > > > > > > > U-Boot.
> > > > > > > > > > > >
> > > > > > > > > > > > With this we can boot on a Raspberry Pi 3 with just one command:
> > > > > > > > > > > >
> > > > > > > > > > > >    bootflow scan -lb
> > > > > > > > > > > >
> > > > > > > > > > > > which means to scan, listing (-l) each bootflow and trying to boot each
> > > > > > > > > > > > one (-b). The final patch shows this.
> > > > > > > > > > > >
> > > > > > > > > > > > With a standard way to identify boot devices, booting become easier. It
> > > > > > > > > > > > also should be possible to support U-Boot scripts, for backwards
> > > > > > > > > > > > compatibility only.
> > > > > > > > > > > >
> > > > > > > > > > > > This series relies on the PXE clean-up series, posted here:
> > > > > > > > > > > >
> > > > > > > > > > > >    https://patchwork.ozlabs.org/project/uboot/list/?series=267078
> > > > > > > > > > > >
> > > > > > > > > > > > For documentation, see the 'doc' patch.
> > > > > > > > > > > >
> > > > > > > > > > > > For version 2, a new naming scheme is used as above:
> > > > > > > > > > > >
> > > > > > > > > > > >    - bootdev is used instead of bootdevice, because 'device' is overused,
> > > > > > > > > > > >        is everywhere in U-Boot, can be confused with udevice
> > > > > > > > > > > >    - bootmeth - because 'method' is too vanilla, appears 1300 times in
> > > > > > > > > > > >        U-Boot
> > > > > > > > > > > >
> > > > > > > > > > > > Also in version 2, drivers are introduced for the boot methods, to make
> > > > > > > > > > > > it more extensible. Booting a custom OS is simply a matter of creating a
> > > > > > > > > > > > bootmeth for it and implementing the read_file() and boot() methods.
> > > > > > > > > > > >
> > > > > > > > > > > > Version 4 makes some minor improvements and leaves out the RFC patch for
> > > > > > > > > > > > rpi conversion, in the hope of getting the base support applied sooner
> > > > > > > > > > > > rather than later.
> > > > > > > > > > > >
> > > > > > > > > > > > The design is described in these two documents:
> > > > > > > > > > > >
> > > > > > > > > > > > https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing
> > > > > > > > > > > >
> > > > > > > > > > > > https://drive.google.com/file/d/1kTrflO9vvGlKp-ZH_jlgb9TY3WYG6FF9/view?usp=sharing
> > > > > > > > > > >
> > > > > > > > > > > I keep putting off commenting more here, but, I still feel this is the
> > > > > > > > > > > wrong direction.  What problems do we have today with distro boot?
> > > > > > > > > > > Well, we haven't figured out how to move configuring it out of the board
> > > > > > > > > > > config.h file.  But that's just one of a half dozen or so examples of
> > > > > > > > > > > how we haven't figured out a good solution to configuring the default
> > > > > > > > > > > environment.  And only some of those other examples are boot related
> > > > > > > > > > > (the NXP chain of trust booting stuff is another boot example, ETHPRIME,
> > > > > > > > > > > HOSTNAME, etc, are non-boot examples).
> > > > > > > > > > >
> > > > > > > > > > > We also aren't improving testing of "can we boot" here, because what
> > > > > > > > > > > THAT needs is setting up LAVA and booting some installers on some
> > > > > > > > > > > hardware (and some QEMU).  That's testing that Linux boot works.  Today
> > > > > > > > > > > we have tests for hush parsing, and if distro boot makes use of
> > > > > > > > > > > something we don't have a test for, we need a test for it.  This adds
> > > > > > > > > > > tests for itself, which is good.
> > > > > > > > > > >
> > > > > > > > > > > And I still don't see an example of where this demonstrates that
> > > > > > > > > > > existing non-UEFI boot cases are now easier to handle or cleaner to
> > > > > > > > > > > handle or otherwise better.
> > > > > > > > > > >
> > > > > > > > > > > In that this is an attempt to tackle one of the long standing needed
> > > > > > > > > > > migrations (be able to drop board config.h files), something here needs
> > > > > > > > > > > doing.  But I don't see this as the right direction, sorry.
> > > > > > > > > >
> > > > > > > > > > Does anyone have a better idea for all of this? This is a solid base
> > > > > > > > > > we can build on but we can't make any progress while this is just
> > > > > > > > > > patches. What not apply it and we can move forward?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I agree with Simon. Having a well documented flow, help to integrate
> > > > > > > > > products and have a standard
> > > > > > > > > way to handle the booting flow
> > > > > > > > >
> > > > > > > > > > - solves the env problem for distro boot in that we don't need the scripts
> > > > > > > > > > - gets rid of the scripts which are a confusing mess
> > > > > > > > > > - provides proper high-level concepts of boot device and boot method
> > > > > > > > > > - allows testing of the U-Boot part of 'can we boot' because we have
> > > > > > > > > > tests for all the cases - we can expand this over time
> > > > > > > > > > - allows non-UEFI boot cases like Chrome OS, which is currently just a
> > > > > > > > > > hack for one board[1]
> > > > > > > > > > - provides a programmatic base for A/B boot, etc.
> > > > > > > > > >
> > > > > > > > > > I feel the same way with Takahiro's series, which has been out-of-tree
> > > > > > > > > > for too long.
> > > > > > > > >
> > > > > > > > > I don't see the problem in having it merged. I'm dealing every day
> > > > > > > > > with crazy script
> > > > > > > > > to handle situation like [1] and I think that company that integrates
> > > > > > > > > their product can
> > > > > > > > > benefits on those changes. They can be improved with other people
> > > > > > > > > wants to use it
> > > > > > > > > in their products.
> > > > > > > > >
> > > > > > > > > Michael
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Please reconsider this. What do we have to lose?
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Simon
> > > > > > > > > >
> > > > > > > > > > [1] CONFIG_BOOTCOMMAND="tpm init; tpm startup TPM2_SU_CLEAR; read mmc
> > > > > > > > > > 0:2 100000 0 80; setexpr loader *001004f0; setexpr size *00100518;
> > > > > > > > > > setexpr blocks $size / 200; read mmc 0:2 100000 80 $blocks; setexpr
> > > > > > > > > > setup $loader - 1000; setexpr cmdline_ptr $loader - 2000; setexpr.s
> > > > > > > > > > cmdline *$cmdline_ptr; setexpr cmdline gsub %U \\\\${uuid}; if part
> > > > > > > > > > uuid mmc 0:2 uuid; then zboot start 100000 0 0 0 $setup cmdline; zboot
> > > > > > > > > > load; zboot setup; zboot dump; zboot go;fi"
> > > > > > > >
> > > > > > > > OK, and what does your example here look like on top of Simon's series?
> > > > > > > > Or do you just mean ChromeOS boot?
> > > > > > >
> > > > > > > I can use some of our boards and move on to the Simon patchset. In
> > > > > > > that case, are you happy with it?
> > > > > >
> > > > > > No, I'm not saying I'll take this if someone uses it somewhere.  But
> > > > > > I've been asking for in previous iterations for showing that it makes
> > > > > > some existing use case easier.  And I don't see any implementations of
> > > > > > that in v4.  Adding UEFI boot to this isn't a good example since that's
> > > > > > already being re-done via the UEFI boot manager series that implements
> > > > > > what the spec says to do for that.
> > > > >
> > > > > I don't think that a lot of real use cases in embedded devices are
> > > > > using distro boot but
> > > > > they have proper customized boot flow and update, recovery. What you
> > > > > call A, B and C.
> > > > > Then we have a special recovery path that instead of using emmc , uses
> > > > > a usb pen drive. Having
> > > > > some more structure boot flow with documentation and some use cases
> > > > > will help to have in uboot what it's in
> > > > > private repositories.
> > > >
> > > > Exactly.  My concern is that this does, or will end up, spending a lot
> > > > of effort to replicate the "find an arbitrary bootable thing" logic the
> > > > UEFI boot manager stuff has to do to that spec rather than making it
> > > > easier to handle the common everything else cases where the developer
> > > > knows the valid cases for normal boot and recovery boot and needs to do
> > > > whatever validation is required there.  Maybe that's where some of the
> > > > hang up is.
> > >
> > > I'm actually not sure where the hang-up is. We seem to be enforcing
> > > UEFI everywhere in U-Boot. That must make some people very happy, but
> > > it is not the right approach to take for a general purpose,
> > > open-source, Universal Boot Loader.
> > >
> > > If U-Boot is to remain a boot loader for non-UEFI cases, then I think
> > > bootstd is an important step forward. There is more work to do, but it
> > > sets up the basic abstractions and is a strong base to work from.
> > >
> > > Or is U-Boot for UEFI only, with only 'boot manager' allowed to have a
> > > structured boot?
> > >
> > > I hope not, but I'm struggling to read much else from this thread.
> >
> > Perhaps you and I need to have a call at some point soon then?  It is
> > not my intention to make UEFI the only, or only well supported, way of
> > booting things.
> 
> Sure, I could do this time tomorrow or 2-3 hours earlier / later. I am
> around on irc.

Sorry, my time over the weekend got away from me.

Perhaps instructive of where I'm coming from is illustrated by what
happened to me, today, with my x86-64 build box and modern UEFI
firmware.  It doesn't reliably reboot.  Today I had to reboot, it didn't
come back up and I had a few minutes to dig.  The problem?  UEFI
heuristic booting logic decided the assorted USB drives attached to the
machine had a higher priority, picked some random ESP (with Aarch64
stuff in it only) and just stopped there.  Easily enough fixed, but not
at all what I expected.  Logic for "do what the user thinks they want"
fails more often than it should.  I don't like it, but it's a defined
and popular specification, so, OK.

Should it be easier to implemented defined, structured boot flows in
U-Boot, for things resembling a known and defined set of choices (even
allowing user input as a choice)?  Yes.  But I am really skeptical of
"do what the user thinks they want" implementations.  But I also think
where that's the right path to try is not the frequent case where U-Boot
is concerned.

But also, frankly, there's more than enough places in U-Boot where
you're the person that understands critical parts of the world the best,
so if the price is you're going to implemented a better heuristic boot
too, well, alright.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20220328/f5ef5d73/attachment.sig>