[PATCH v4 00/33] Initial implementation of standard boot

[Simon Glass]

Hi Tom,

On Fri, 25 Mar 2022 at 08:50, Tom Rini <trini at konsulko.com> wrote:
>
> On Fri, Mar 25, 2022 at 03:36:24PM +0100, Michael Nazzareno Trimarchi wrote:
> > Hi Tom
> >
> >
> > On Wed, Mar 23, 2022 at 9:07 PM Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Wed, Mar 23, 2022 at 08:57:36PM +0100, Michael Nazzareno Trimarchi wrote:
> > > > Hi Tom
> > > >
> > > > On Wed, Mar 23, 2022 at 8:30 PM Tom Rini <trini at konsulko.com> wrote:
> > > > >
> > > > > On Wed, Mar 23, 2022 at 08:21:22PM +0100, Michael Nazzareno Trimarchi wrote:
> > > > > > Hi Tom
> > > > > >
> > > > > > On Wed, Mar 23, 2022 at 7:46 PM Simon Glass <sjg at chromium.org> wrote:
> > > > > > >
> > > > > > > Hi Tom,
> > > > > > >
> > > > > > > On Wed, 23 Mar 2022 at 08:05, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Sun, Mar 06, 2022 at 05:49:43AM -0700, Simon Glass wrote:
> > > > > > > > >
> > > > > > > > > The bootflow feature provide a built-in way for U-Boot to automatically
> > > > > > > > > boot an Operating System without custom scripting and other customisation.
> > > > > > > > > This is called 'standard boot' since it provides a standard way for
> > > > > > > > > U-Boot to boot a distro, without scripting.
> > > > > > > > >
> > > > > > > > > It introduces the following concepts:
> > > > > > > > >
> > > > > > > > >    - bootdev - a device which can hold a distro
> > > > > > > > >    - bootmeth - a method to scan a bootdev to find bootflows (owned by
> > > > > > > > >                 U-Boot)
> > > > > > > > >    - bootflow - a description of how to boot (owned by the distro)
> > > > > > > > >
> > > > > > > > > This series provides an implementation of these, enabled to scan for
> > > > > > > > > bootflows from MMC, USB and Ethernet. It supports the existing distro
> > > > > > > > > boot as well as the EFI loader flow (bootefi/bootmgr). It works
> > > > > > > > > similiarly to the existing script-based approach, but is native to
> > > > > > > > > U-Boot.
> > > > > > > > >
> > > > > > > > > With this we can boot on a Raspberry Pi 3 with just one command:
> > > > > > > > >
> > > > > > > > >    bootflow scan -lb
> > > > > > > > >
> > > > > > > > > which means to scan, listing (-l) each bootflow and trying to boot each
> > > > > > > > > one (-b). The final patch shows this.
> > > > > > > > >
> > > > > > > > > With a standard way to identify boot devices, booting become easier. It
> > > > > > > > > also should be possible to support U-Boot scripts, for backwards
> > > > > > > > > compatibility only.
> > > > > > > > >
> > > > > > > > > This series relies on the PXE clean-up series, posted here:
> > > > > > > > >
> > > > > > > > >    https://patchwork.ozlabs.org/project/uboot/list/?series=267078
> > > > > > > > >
> > > > > > > > > For documentation, see the 'doc' patch.
> > > > > > > > >
> > > > > > > > > For version 2, a new naming scheme is used as above:
> > > > > > > > >
> > > > > > > > >    - bootdev is used instead of bootdevice, because 'device' is overused,
> > > > > > > > >        is everywhere in U-Boot, can be confused with udevice
> > > > > > > > >    - bootmeth - because 'method' is too vanilla, appears 1300 times in
> > > > > > > > >        U-Boot
> > > > > > > > >
> > > > > > > > > Also in version 2, drivers are introduced for the boot methods, to make
> > > > > > > > > it more extensible. Booting a custom OS is simply a matter of creating a
> > > > > > > > > bootmeth for it and implementing the read_file() and boot() methods.
> > > > > > > > >
> > > > > > > > > Version 4 makes some minor improvements and leaves out the RFC patch for
> > > > > > > > > rpi conversion, in the hope of getting the base support applied sooner
> > > > > > > > > rather than later.
> > > > > > > > >
> > > > > > > > > The design is described in these two documents:
> > > > > > > > >
> > > > > > > > > https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing
> > > > > > > > >
> > > > > > > > > https://drive.google.com/file/d/1kTrflO9vvGlKp-ZH_jlgb9TY3WYG6FF9/view?usp=sharing
> > > > > > > >
> > > > > > > > I keep putting off commenting more here, but, I still feel this is the
> > > > > > > > wrong direction.  What problems do we have today with distro boot?
> > > > > > > > Well, we haven't figured out how to move configuring it out of the board
> > > > > > > > config.h file.  But that's just one of a half dozen or so examples of
> > > > > > > > how we haven't figured out a good solution to configuring the default
> > > > > > > > environment.  And only some of those other examples are boot related
> > > > > > > > (the NXP chain of trust booting stuff is another boot example, ETHPRIME,
> > > > > > > > HOSTNAME, etc, are non-boot examples).
> > > > > > > >
> > > > > > > > We also aren't improving testing of "can we boot" here, because what
> > > > > > > > THAT needs is setting up LAVA and booting some installers on some
> > > > > > > > hardware (and some QEMU).  That's testing that Linux boot works.  Today
> > > > > > > > we have tests for hush parsing, and if distro boot makes use of
> > > > > > > > something we don't have a test for, we need a test for it.  This adds
> > > > > > > > tests for itself, which is good.
> > > > > > > >
> > > > > > > > And I still don't see an example of where this demonstrates that
> > > > > > > > existing non-UEFI boot cases are now easier to handle or cleaner to
> > > > > > > > handle or otherwise better.
> > > > > > > >
> > > > > > > > In that this is an attempt to tackle one of the long standing needed
> > > > > > > > migrations (be able to drop board config.h files), something here needs
> > > > > > > > doing.  But I don't see this as the right direction, sorry.
> > > > > > >
> > > > > > > Does anyone have a better idea for all of this? This is a solid base
> > > > > > > we can build on but we can't make any progress while this is just
> > > > > > > patches. What not apply it and we can move forward?
> > > > > > >
> > > > > >
> > > > > > I agree with Simon. Having a well documented flow, help to integrate
> > > > > > products and have a standard
> > > > > > way to handle the booting flow
> > > > > >
> > > > > > > - solves the env problem for distro boot in that we don't need the scripts
> > > > > > > - gets rid of the scripts which are a confusing mess
> > > > > > > - provides proper high-level concepts of boot device and boot method
> > > > > > > - allows testing of the U-Boot part of 'can we boot' because we have
> > > > > > > tests for all the cases - we can expand this over time
> > > > > > > - allows non-UEFI boot cases like Chrome OS, which is currently just a
> > > > > > > hack for one board[1]
> > > > > > > - provides a programmatic base for A/B boot, etc.
> > > > > > >
> > > > > > > I feel the same way with Takahiro's series, which has been out-of-tree
> > > > > > > for too long.
> > > > > >
> > > > > > I don't see the problem in having it merged. I'm dealing every day
> > > > > > with crazy script
> > > > > > to handle situation like [1] and I think that company that integrates
> > > > > > their product can
> > > > > > benefits on those changes. They can be improved with other people
> > > > > > wants to use it
> > > > > > in their products.
> > > > > >
> > > > > > Michael
> > > > > >
> > > > > > >
> > > > > > > Please reconsider this. What do we have to lose?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Simon
> > > > > > >
> > > > > > > [1] CONFIG_BOOTCOMMAND="tpm init; tpm startup TPM2_SU_CLEAR; read mmc
> > > > > > > 0:2 100000 0 80; setexpr loader *001004f0; setexpr size *00100518;
> > > > > > > setexpr blocks $size / 200; read mmc 0:2 100000 80 $blocks; setexpr
> > > > > > > setup $loader - 1000; setexpr cmdline_ptr $loader - 2000; setexpr.s
> > > > > > > cmdline *$cmdline_ptr; setexpr cmdline gsub %U \\\\${uuid}; if part
> > > > > > > uuid mmc 0:2 uuid; then zboot start 100000 0 0 0 $setup cmdline; zboot
> > > > > > > load; zboot setup; zboot dump; zboot go;fi"
> > > > >
> > > > > OK, and what does your example here look like on top of Simon's series?
> > > > > Or do you just mean ChromeOS boot?
> > > >
> > > > I can use some of our boards and move on to the Simon patchset. In
> > > > that case, are you happy with it?
> > >
> > > No, I'm not saying I'll take this if someone uses it somewhere.  But
> > > I've been asking for in previous iterations for showing that it makes
> > > some existing use case easier.  And I don't see any implementations of
> > > that in v4.  Adding UEFI boot to this isn't a good example since that's
> > > already being re-done via the UEFI boot manager series that implements
> > > what the spec says to do for that.
> >
> > I don't think that a lot of real use cases in embedded devices are
> > using distro boot but
> > they have proper customized boot flow and update, recovery. What you
> > call A, B and C.
> > Then we have a special recovery path that instead of using emmc , uses
> > a usb pen drive. Having
> > some more structure boot flow with documentation and some use cases
> > will help to have in uboot what it's in
> > private repositories.
>
> Exactly.  My concern is that this does, or will end up, spending a lot
> of effort to replicate the "find an arbitrary bootable thing" logic the
> UEFI boot manager stuff has to do to that spec rather than making it
> easier to handle the common everything else cases where the developer
> knows the valid cases for normal boot and recovery boot and needs to do
> whatever validation is required there.  Maybe that's where some of the
> hang up is.

I'm actually not sure where the hang-up is. We seem to be enforcing
UEFI everywhere in U-Boot. That must make some people very happy, but
it is not the right approach to take for a general purpose,
open-source, Universal Boot Loader.

If U-Boot is to remain a boot loader for non-UEFI cases, then I think
bootstd is an important step forward. There is more work to do, but it
sets up the basic abstractions and is a strong base to work from.

Or is U-Boot for UEFI only, with only 'boot manager' allowed to have a
structured boot?

I hope not, but I'm struggling to read much else from this thread.

Regards,
Simon