[PATCH 00/28] Initial implementation of bootmethod/bootflow

[AKASHI Takahiro]

Mark, Tom,

On Wed, Aug 25, 2021 at 06:06:05PM -0400, Tom Rini wrote:
> On Wed, Aug 25, 2021 at 11:54:58PM +0200, Mark Kettenis wrote:
> > > Date: Wed, 25 Aug 2021 10:56:35 -0400
> > > From: Tom Rini <trini at konsulko.com>
> > > 
> > > On Wed, Aug 25, 2021 at 11:42:51PM +0900, AKASHI Takahiro wrote:
> > > > Simon,
> > > > 
> > > > On Wed, Aug 25, 2021 at 07:11:44AM -0600, Simon Glass wrote:
> > > > > Hi,
> > > > > 
> > > > > On Wed, 25 Aug 2021 at 04:45, Emmanuel Vadot <manu at bidouilliste.com> wrote:
> > > > > >
> > > > > > On Tue, 24 Aug 2021 12:22:42 +0200 (CEST)
> > > > > > Mark Kettenis <mark.kettenis at xs4all.nl> wrote:
> > > > > >
> > > > > > > > Date: Mon, 23 Aug 2021 16:01:46 -0400
> > > > > > > > From: Tom Rini <trini at konsulko.com>
> > > > > > > >
> > > > > > > > On Mon, Aug 23, 2021 at 11:25:42AM -0600, Simon Glass wrote:
> > > > > > > > > Hi Mark,
> > > > > > > > >
> > > > > > > > > On Mon, 23 Aug 2021 at 05:54, Mark Kettenis <mark.kettenis at xs4all.nl> wrote:
> > > > > > > > > >
> > > > > > > > > > > From: Simon Glass <sjg at chromium.org>
> > > > > > > > > > > Date: Wed, 18 Aug 2021 21:45:33 -0600
> > > > > > > > > > >
> > > > > > > > > > > Bootmethod and bootflow provide a built-in way for U-Boot to automatically boot
> > > > > > > > > > > an Operating System without custom scripting and other customisation:
> > > > > > > > > > >
> > > > > > > > > > >   - bootmethod - a method to scan a device to find bootflows (owned by U-Boot)
> > > > > > > > > > >   - bootflow - a description of how to boot (owned by the distro)
> > > > > > > > > > >
> > > > > > > > > > > This series provides an initial implementation of these, enable to scan
> > > > > > > > > > > for bootflows from MMC and Ethernet. The only bootflow supported is
> > > > > > > > > > > distro boot, i.e. an extlinux.conf file included on a filesystem or
> > > > > > > > > > > tftp server. It works similiarly to the existing script-based approach,
> > > > > > > > > > > but is native to U-Boot.
> > > > > > > > > > >
> > > > > > > > > > > With this we can boot on a Raspberry Pi 3 with just one command:
> > > > > > > > > > >
> > > > > > > > > > >    bootflow scan -lb
> > > > > > > > > > >
> > > > > > > > > > > which means to scan, listing (-l) each bootflow and trying to boot each
> > > > > > > > > > > one (-b). The final patch shows this.
> > > > > > > > > > >
> > > > > > > > > > > It is intended that this approach be expanded to support mechanisms other
> > > > > > > > > > > than distro boot, including EFI-related ones. With a standard way to
> > > > > > > > > > > identify boot devices, these features become easier. It also should
> > > > > > > > > > > support U-Boot scripts, for backwards compatibility only.
> > > > > > > > > > >
> > > > > > > > > > > The first patch of this series moves boot-related code out of common/ and
> > > > > > > > > > > into a new boot/ directory. This helps to collect these related files
> > > > > > > > > > > in one place, as common/ is quite large.
> > > > > > > > > > >
> > > > > > > > > > > Like sysboot, this feature makes use of the existing PXE implementation.
> > > > > > > > > > > Much of this series consists of cleaning up that code and refactoring it
> > > > > > > > > > > into something closer to a module that can be called, teasing apart its
> > > > > > > > > > > reliance on the command-line interpreter to access filesystems and the
> > > > > > > > > > > like. Also it now uses function arguments and its own context struct
> > > > > > > > > > > internally rather than environment variables, which is very hard to
> > > > > > > > > > > follow. No core functional change is included in the included PXE patches.
> > > > > > > > > > >
> > > > > > > > > > > For documentation, see the 'doc' patch.
> > > > > > > > > > >
> > > > > > > > > > > There is quite a long list of future work included in the documentation.
> > > > > > > > > > > One question is the choice of naming. Since this is a bootloader, should
> > > > > > > > > > > we just call this a 'method' and a 'flow' ? The 'boot' prefix is already
> > > > > > > > > > > shared by other commands like bootm, booti, etc.
> > > > > > > > > > >
> > > > > > > > > > > The design is described here:
> > > > > > > > > > >
> > > > > > > > > > > https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing
> > > > > > > > > > >
> > > > > > > > > > > The series is available at u-boot-dm/bmea-working
> > > > > > > > > >
> > > > > > > > > > How does the user control the order in which devices are scanned/booted?
> > > > > > > > >
> > > > > > > > > That is not supported in distroboot at present, at least so far as I
> > > > > > > > > can see. For Fedora it seems to happen in grub. Do I have that right?
> > > > > > > >
> > > > > > > > Well, there's "find the next stage", which is boot_targets environment
> > > > > > > > variable, and then "where that next stage looks for stuff" which is
> > > > > > > > OS-dependent.  Sometimes the ESP grub.cfg file is just enough to tell
> > > > > > > > grub to find the full grub.cfg file elsewhere, and sometimes it's a full
> > > > > > > > grub.cfg file.  I think Mark is talking about the former, and you've
> > > > > > > > said it's not part of this series, yet, but on the TODO list.
> > > > > > >
> > > > > > > Right.  With the current distroboot code the order of the devices that
> > > > > > > appears in boot_targets is determined by per-board/SOC/machine config
> > > > > > > files and the order isn't the same for all of them.  Users can change
> > > > > > > the order if necessary by modifying the environment variable and
> > > > > > > saving the environment.  And for a one-off boot from a different
> > > > > > > device they can simply run an appropriate boot command.  The
> > > > > > > boot_targets variable in particular is documented in various install
> > > > > > > documents so it would probably be good of the new "bootmethod" code
> > > > > > > would respect this variable.
> > > > > > >
> > > > > > > For OpenBSD I'm not really interested in the bootflow part.  As I
> > > > > > > explained in the past, that part of the problem is solved in a
> > > > > > > (mostly) uniform way across platforms by the OpenBSD bootloader which
> > > > > > > can read an /etc/boot.conf that allows bootflow customization.  So as
> > > > > > > long as the default of the new code still results in
> > > > > > > \EFI\BOOT\BOOT{machine type short-name}.EFI being loaded and run if
> > > > > > > there is no U-Boot specific bootflow configured, I'm happy.
> > > > > >
> > > > > >  Mostly the same for FreeBSD, as long as the efi boot<arch>.efi is
> > > > > > loaded and run by default (respecting the boot_targets order) we will
> > > > > > be fine.
> > > > > 
> > > > > OK thanks for the info. My expectation is that bootmethod/bootflow can
> > > > > support this easily enough (it is actually simpler than distro boot).
> > > > > 
> > > > > >
> > > > > > > I can't speak for the other BSDs, but my impression is that they are
> > > > > > > pretty much in the same position.  The FreeBSD bootloader for example
> > > > > > > supports a high-degree of "bootflow" customization and I doubt that
> > > > > > > taking it out of the loop is a viable option for most users.
> > > > > 
> > > > > I think the same may happen with grub. E.g. with Ubuntu I see quite a
> > > > > bit of code in the grub.cfg file and it's not clear to me that it can
> > > > > be replaced with a 'data instead of code' approach. Still, a valid
> > > > > bootflow is simply to jump to an EFI app, which seems to be what is
> > > > > happening here. The bootflow side is really just about describing what
> > > > > to do, and this case is no different. For now I see three types of
> > > > > bootflow, PXE/syslinux, EFI boot manager and EFI app.
> > > > 
> > > > By "EFI app", do you mean a way of booting "/efi/boot/bootXX.efi"
> > > > (default file name in case that no image path is specified)?
> > > > 
> > > > In fact, this behavior, or removable media support, is defined
> > > > as part of UEFI boot manager in UEFI specification. (See section 3.5)
> > > > What this means is that the boot order, including a removable media
> > > > case and user-provided BootXXXX cases, should be controlled solely
> > > > by "BootOrder" variable.
> > > > So the current combination of distro_bootcmd + UEFI boot manger doesn't
> > > > fully comply with the specification.
> > > > 
> > > > Even if those two cases are integrated, I don't know how "BootOrder"
> > > > semantics can be preserved in your approach.
> > > 
> > > I think the high level answer is that whereas today part of
> > > distro_bootcmd (and so iterating over boot_targets) "bootefi bootmgr"
> > > gets run, with what Simon is proposing we would have an easier / quicker
> > > way to get over to just running that.  Perhaps a clean-up to just use
> > > that, even?  Or are we not to the point yet where we could remove the
> > > direct fall-back to /efi/boot/bootXX.efi ?
> > 
> > I think "bootefi bootmgr" only works if the BootOrder variable is
> > defined, and currently that isn't the case.
> > 
> > The boot manager behaviour as specified in the UEFI specification is
> > somewhat problematic to implement in U-Boot because of the limitations
> > in how variables can be set at runtime.  This is one of the reasons
> > OpenBSD doesn't actually bother with setting the variables and simple
> > relies on the "removable media" support mentioned above.  All my

Well, if this is everyone's consensus in (U-Boot) community,
it would be fine. But different people may have different
opinions/requirements on different systems. So I believe that
the safest way is to make UEFI implementation as compliant to
the specification as possible.

> > OpenBSD systems that use U-Boot print the follwing lines:
> > 
> >   BootOrder not defined
> >   EFI boot manager: Cannot load any image
> >   Found EFI removable media binary efi/boot/bootaa64.efi
> > 
> > But maybe that last step can be intgrated into bootefi bootmgr at some
> > point?

Yes, it is doable, even I'm now trying to examine and address this issue
as I am fully aware of the situation.

> > Also note that manually manipulating the EFI variables to change the
> > boot order is quite cumbersome; it isn't a matter of just changed the
> > aforementioned BootOrder variable.  That's why I think boot_targets is
> > the preferable way to define the order in which devices should be
> > booted.  I don't think that violates the UEFI specification.  It
> > merely is the way U-Boot implements the boot device selection that
> > more traditional UEFI implementations implement using a menu.
> 
> As I don't want to side-track Simon's thread even further, I would like
> to see a bit more detailed explanation of why U-Boot cannot support EFI
> variables, or if it's just a matter of someone doing the work, and it's
> not been a priority yet.

It is more or less a matter of implementation.
Historically, removable media support was added to distro_bootcmd first
and then boot manager was implemented later, only both were integrated
at script level. Since then, nobody, AFAIK, has complained about
the lack of integrity. That's what we see now.

One of difficulties that I found in the implementation is
how to specify preferred boot media in a form of EFI device path.
Take an example; I want to do
=> efidebug boot add -b 1 USB usb 0 "" -s ""    ; USB port-1
                              ^^^^^^^^ no image file specified
=> efidebug boot add -b 2 USB usb 1 "" -s ""    ; USB port-2
=> efidebug boot add -b 3 MMC mmc 0 "" -s ""    ; MMC slot-1
=> efidebug boot add -b 4 MYKERNEL scsi 0:1 /.../vmlinux.efi -s "..."
=> efidebug boot order 1 2 3 4
=> bootefi bootmgr

First three commands will fail now.

Under the current implementation, neither "usb 0", "usb 1" nor "mmc 0",
which is set to be converted to device path representation in BootXXXX
variable, are recognized as a valid device since there is no physical
device attached to corresponding bus/port/slots at this time.
Internally, UEFI system/efidebug always tries to look for a U-Boot's
existing block device to identify the correct device path.
(Remember that *buses* and *controller* are even not udevices?).
So we would have to manage to create a valid internal representation of
bus or port for those media.

# FYI, my current prototype work already supports the following command line
# and enables booting the system from the USB media if some storage device
# is attached to the port-1:
# => efidebug boot add -b 1 USB usb 0:1 "" -s ""    ; USB port-1
                                ^^^^^^^

As I'm not well-versed in U-Boot's device model, the rework
would be simpler than I anticipate.

-Takahiro Akashi

> -- 
> Tom