bootstd: Support for distro specific EFI folders

Tue Dec 5 19:36:08 CET 2023

Hi Simon and Heinrich,

On Sun, Dec 3, 2023 at 8:38 PM Simon Glass <sjg at chromium.org> wrote:
>
> Hi Heinrich,
>
> On Sun, 3 Dec 2023 at 13:00, Heinrich Schuchardt
> <heinrich.schuchardt at canonical.com> wrote:
> >
> > On 12/3/23 20:50, Simon Glass wrote:
> > > Hi Heinrich,
> > >
> > > On Sun, 3 Dec 2023 at 11:33, Heinrich Schuchardt
> > > <heinrich.schuchardt at canonical.com> wrote:
> > >>
> > >> On 12/3/23 19:22, Simon Glass wrote:
> > >>> Hi Heinrich,
> > >>>
> > >>> On Sun, 3 Dec 2023 at 10:53, Heinrich Schuchardt
> > >>> <heinrich.schuchardt at canonical.com> wrote:
> > >>>>
> > >>>> On 12/3/23 18:44, Simon Glass wrote:
> > >>>>> Hi Heinrich,
> > >>>>>
> > >>>>> On Sun, 3 Dec 2023 at 03:55, Heinrich Schuchardt
> > >>>>> <heinrich.schuchardt at canonical.com> wrote:
> > >>>>>>
> > >>>>>> On 12/3/23 00:38, Shantur Rathore wrote:
> > >>>>>>> Hi Simon,
> > >>>>>>>
> > >>>>>>> On Sat, Dec 2, 2023 at 6:33 PM Simon Glass <sjg at chromium.org> wrote:
> > >>>>>>>>
> > >>>>>>>> Hi,
> > >>>>>>>>
> > >>>>>>>> On Mon, 20 Nov 2023 at 00:02, Ilias Apalodimas
> > >>>>>>>> <ilias.apalodimas at linaro.org> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Hi Mark,
> > >>>>>>>>>
> > >>>>>>>>> On Sun, 19 Nov 2023 at 19:38, Mark Kettenis <mark.kettenis at xs4all.nl> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Date: Sat, 18 Nov 2023 23:52:11 +0100
> > >>>>>>>>>>> From: Heinrich Schuchardt <heinrich.schuchardt at canonical.com>
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Heinrich,
> > >>>>>>>>>>
> > >>>>>>>>>>> On 11/18/23 22:28, Shantur Rathore wrote:
> > >>>>>>>>>>>> Hi Heinrich,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, Nov 17, 2023 at 3:12 PM Heinrich Schuchardt
> > >>>>>>>>>>>> <heinrich.schuchardt at canonical.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On 11/16/23 19:45, Shantur Rathore wrote:
> > >>>>>>>>>>>>>> On Thu, Nov 16, 2023 at 6:15 PM Heinrich Schuchardt
> > >>>>>>>>>>>>>> <heinrich.schuchardt at canonical.com> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On 11/16/23 17:52, Shantur Rathore wrote:
> > >>>>>>>>>>>>>>>> Hi Simon,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Currently bootstd - bootmethod_efi only looks for the default
> > >>>>>>>>>>>>>>>> bootxx64.efi in /EFI/boot folder only.
> > >>>>>>>>>>>>>>>> Generally many distros end up putting their bootloaders in
> > >>>>>>>>>>>>>>>> EFI/<distro> folders like EFI/ubuntu and EFI/debian etc.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> In x86 worlds, the NVRAM is modified and new boot entries are added to
> > >>>>>>>>>>>>>>>> support these but in the U-boot world the NVRAM variables are
> > >>>>>>>>>>>>>>>> read-only.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I guess you are referring to UEFI boot options. These typically are not
> > >>>>>>>>>>>>>>> stored in non-volatile RAM but on a SPI flash device.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks for correcting me.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> What would be the best way to implement this?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I was thinking of having a "efi_prefixes" environment variable which
> > >>>>>>>>>>>>>>>> can be set to "ubuntu debian centos" etc and bootmethod_efi can try
> > >>>>>>>>>>>>>>>> all of them. Will bootmethod_efi be able to support multiple entries (
> > >>>>>>>>>>>>>>>> thinking of multiboot ) ?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On my laptop I have:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> EFI/Microsoft/Boot/bootmgr.efi
> > >>>>>>>>>>>>>>> EFI/Microsoft/Boot/memtest.efi
> > >>>>>>>>>>>>>>> EFI/Boot/bootx64.efi
> > >>>>>>>>>>>>>>> EFI/Boot/fbx64.efi
> > >>>>>>>>>>>>>>> EFI/Boot/mmx64.efi
> > >>>>>>>>>>>>>>> EFI/debian/shimx64.efi
> > >>>>>>>>>>>>>>> EFI/debian/grubx64.efi
> > >>>>>>>>>>>>>>> EFI/debian/mmx64.efi
> > >>>>>>>>>>>>>>> EFI/debian/fbx64.efi
> > >>>>>>>>>>>>>>> EFI/ubuntu/grubx64.efi
> > >>>>>>>>>>>>>>> EFI/ubuntu/shimx64.efi
> > >>>>>>>>>>>>>>> EFI/ubuntu/mmx64.efi
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Obviously each installed operating system provides multiple EFI binaries
> > >>>>>>>>>>>>>>> and non uses the fallback file name BOOT<ARCH>.EFI. A value "ubuntu
> > >>>>>>>>>>>>>>> debian centos" would not be able to describe which file you are looking
> > >>>>>>>>>>>>>>> for.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> We already have the U-Boot command line eficonfig and efidebug commands
> > >>>>>>>>>>>>>>> to set up UEFI boot options which can describe which EFI binary to load
> > >>>>>>>>>>>>>>> and which command line to pass to it. These are considered by the
> > >>>>>>>>>>>>>>> existing boot flows.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> So, I am building a new RockPro64 based cluster and using Canonical
> > >>>>>>>>>>>>>> MAAS to set them up automatically, booting them up using DHCP and
> > >>>>>>>>>>>>>> installing them over the network.
> > >>>>>>>>>>>>>> I configured an Armbian image using Packer to be compatible with MAAS
> > >>>>>>>>>>>>>> and it happily installs it. As part of installation process, a
> > >>>>>>>>>>>>>> grub-install is run which installs the grub efi,
> > >>>>>>>>>>>>>> this EFI ends up in EFI/debian instead of expected EFI/boot.
> > >>>>>>>>>>>>>> To be able to make it boot, I have to add commands to move it to
> > >>>>>>>>>>>>>> EFI/boot. I am trying to find a way in U-Boot that would allow me to
> > >>>>>>>>>>>>>> skip this step.
> > >>>>>>>>>>>>>> With eficonfig if I understand correctly, it would need manual
> > >>>>>>>>>>>>>> intervention to create boot entries.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> If you are installing the shim-signed package on Ubuntu, the EFI boot
> > >>>>>>>>>>>>>>> option for Ubuntu is set up by EFI/BOOT/BOOT<ARCH>.EFI using the content
> > >>>>>>>>>>>>>>> of EFI/ubuntu/BOOT<ARCH>.CSV. This is done before ExitBootServices() and
> > >>>>>>>>>>>>>>> therefore should work with current U-Boot.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Patches are pending upstream to make EFI variables writable from Linux
> > >>>>>>>>>>>>>>> if they are stored in the RPMB partition in the eMMC. See this series:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> https://lore.kernel.org/linux-efi/20231107054057.1893-2-masahisa.kojima@linaro.org/
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Would it be possible to save it in SPI Flash as the U-Boot environment ?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Currently this is not supported by U-Boot.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> The U-Boot environment variables can be stored in lots of different
> > >>>>>>>>>>>>> places SPI flash is only one of these. But none of these locations is
> > >>>>>>>>>>>>> protected from OS access which would be preferable for UEFI variables
> > >>>>>>>>>>>>> for security reasons.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> To support boards without eMMC the right way forward would be writing a
> > >>>>>>>>>>>>> new implementation of the OP-TEE standalone MM which writes the
> > >>>>>>>>>>>>> variables to SPI flash instead of the RPMB partition and ensures that
> > >>>>>>>>>>>>> the SPI flash' MMIO registers are protected against access from the
> > >>>>>>>>>>>>> non-secure world.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks for explaining this to me.
> > >>>>>>>>>>>> This seems like a long way to go, for now what would be an acceptable
> > >>>>>>>>>>>> solution, some options are
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1. Allow to set a space separated efi_prefixes (e.g. "boot ubuntu
> > >>>>>>>>>>>> debian") variable which is ready by bootmeth_efi and used as
> > >>>>>>>>>>>> efi/<efi_prefix> instead of efi/boot.
> > >>>>>>>>>>>> 2. Improve bootmeth_efi to find all bootxxxx.efi in efi/ folder and
> > >>>>>>>>>>>> present all them as bootflows to bootstd.
> > >>>>>>>>>>>
> > >>>>>>>>>>> As mentioned in a prior mail ubuntu/bootxxxx.efi and debian/bootxxxx.efi
> > >>>>>>>>>>> don't exist. I would prefer not to add any distro specific stuff in
> > >>>>>>>>>>> upstream U-Boot. Instead we will continue to drive what Linaro has
> > >>>>>>>>>>> suggested to improve U-Boot EFI variables support in Linux.
> > >>>>>>>>>>
> > >>>>>>>>>> I agree that adding hacks like this is not a good idea.
> > >>>>>>>>>
> > >>>>>>>>> + 1
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> The Linaro approach that involves OP-TEE makes for a fairly complex
> > >>>>>>>>>> solution.  And there are plenty of boards that have neither eMMC nor
> > >>>>>>>>>> SPI flash.  If Secure Boot is not a requirement (and I'd argue that
> > >>>>>>>>>> for most "hobbyist" boards it isn't) storing the EFI variables in a
> > >>>>>>>>>> file on the ESP (as implemented by the CONFIG_EFI_VARIABLE_FILE_STORE
> > >>>>>>>>>> Kconfig option) is a workable alternative.  And this is actually what
> > >>>>>>>>>> the rockpro64-rk3399_defconfig enables.
> > >>>>>>>>>
> > >>>>>>>>> Even in that case, preseeding the variables can enable uefi secure
> > >>>>>>>>> boot. But you have to establish a chain of trust since the
> > >>>>>>>>> authenticated EFI variables are part of  the u-boot binary.
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> I noticed that the latest EBBR attempts to standardize this:
> > >>>>>>>>>>
> > >>>>>>>>>>       https://arm-software.github.io/ebbr/index.html#document-chapter5-variable-storage
> > >>>>>>>>>>
> > >>>>>>>>>> Not sure what the status here is.
> > >>>>>>>>>
> > >>>>>>>>> Heinrich and I were the ones who proposed the standardization. The
> > >>>>>>>>> idea is to eventually fix it for all boards and we are working on it,
> > >>>>>>>>>
> > >>>>>>>>>> But if the Linux kernel folks
> > >>>>>>>>>> accept that alternative implementations for runtime EFI variable
> > >>>>>>>>>> access are a thing, then a method that modifies the file would be
> > >>>>>>>>>> possible as well.  Or maybe it is good enough to implement support for
> > >>>>>>>>>> this in the efivar library.
> > >>>>>>>>>
> > >>>>>>>>> Yes to both. I discussed the idea during Linux Plumbers. All
> > >>>>>>>>> approaches have some pros and cons, you can find some details here
> > >>>>>>>>> [0].
> > >>>>>>>>> Implementing support to efitool is straightforward, but a bit too
> > >>>>>>>>> hacky for my taste. An obvious disadvantage is that it's hard to sync
> > >>>>>>>>> the kernel/file view after an update.
> > >>>>>>>>> The plan right now is to investigate overwriting RT functions and use
> > >>>>>>>>> kernel-provided ones that modify the file (which is probably going to
> > >>>>>>>>> be passed in a config table)
> > >>>>>>>>>
> > >>>>>>>>> [0] https://lpc.events/event/17/contributions/1653/
> > >>>>>>>>
> > >>>>>>>> I don't know much about this area, but it strikes me that things are
> > >>>>>>>> getting a little too complicated.
> > >>>>>>>>
> > >>>>>>>> U-Boot wants to show a menu of available Operating Systems. With UEFI
> > >>>>>>>> it isn't even possible to know what is being booted. We just have to
> > >>>>>>>> jump into the .efi file and hope.
> > >>>>>>>>
> > >>>>>>>> Can we work together on adding a new, separate file which lists the
> > >>>>>>>> .efi files and what they are for? Then the user can select one.
> > >>>>>>>>
> > >>>>>>> To add - EFI/BOOT/BOOTXXXX.efi is optional [0] and isn't installed by
> > >>>>>>> generic ARM
> > >>>>>>> distros (debian, armbian.. maybe more)
> > >>>>>>>
> > >>>>>>> +1 for this. This can be implemented as part of bootmeth_efi (in blk
> > >>>>>>> mode) where instead
> > >>>>>>> of checking hardcoded EFI/BOOT/BOOTXXXX.efi, it can follow one of the
> > >>>>>>> following options
> > >>>>>>>
> > >>>>>>> 1. Scan and list all available .efi files - EFI/**/*.efi
> > >>>>>>> Pros - Allows all the possible boot options to be shown to user
> > >>>>>>> Cons - A 'bootflow scan -b' might not boot the system by default as it
> > >>>>>>> might depend on
> > >>>>>>> which efi file is scanned first. Some installs have mmaa64.efi
> > >>>>>>>
> > >>>>>>> 2. Scan and list all available .efi files that match the
> > >>>>>>> "bootmeth_efi_regex" variable if available.
> > >>>>>>> This variable can be configured to allow valid names example
> > >>>>>>> "EFI\/.*\/(boot|grub|fb|shim).*efi"
> > >>>>>>> Pros - Allows to control only efi files that can boot the system by default.
> > >>>>>>> Cons - Might be an issue in environments without CONFIG_REGEX
> > >>>>>>>
> > >>>>>>> 3. Scan all the folders EFI/* for EFI file prefixes controlled by
> > >>>>>>> "bootmeth_efi_name_prefixes" variable
> > >>>>>>> This variable can be a space separated list of all valid prefixes the
> > >>>>>>> user is interested in. e.g. "boot grub fb shim"
> > >>>>>>> Pros - This allows control on what EFI files to look for without being
> > >>>>>>> depended on CONFIG_REGEX
> > >>>>>>> Cons - I can't think of, please do tell.
> > >>>>>>>
> > >>>>>>> After this, bootmeth_efi can set up more than 1 bootflows that are bootable.
> > >>>>>>>
> > >>>>>>> [0] - https://www.rodsbooks.com/efi-bootloaders/principles.html
> > >>>>>>>
> > >>>>>>> Kind regards,
> > >>>>>>> Shantur
> > >>>>>>
> > >>>>>> U-Boot should remain be a compliant UEFI implementation. We shall not
> > >>>>>> scan any other directories than those described in the UEFI specification.
> > >>>>>

As per UEFI spec v2.10, Link - [0], Section  3.4.3 Boot Option
Variables Default Boot Behavior
It explains what is expected behaviour when none of the variables are
present (or after Bios reset).

It says -
"It is expected that this default boot will load an operating system
or a maintenance utility. If this is an
operating system setup program it is then responsible for setting the
requisite environment variables for
subsequent boots. The platform firmware may also decide to recover or
set to a known set of boot options."

So as per the spec, the firmware is allowed to set up some known set
of boot options which in our case can
be done with bootflow scan. I don't think it breaks the spec, please
correct me if I am wrong.

Even in EDK2, while creating new Boot variables first time, it states that
Link [1]

"      //
      // Try best to add the boot options so continue upon failure.
      //
"

All in all, spec allows firmware developers to handle cases that would
allow OS to boot in exceptional
circumstances. With u-boot and lack of SetVariables from OS we always
land in Default Boot behaviour
and only use EFI/BOOT/BOOTXXXX.EFI as fallback which is supposed to be
used for removable
boot media - Section 3.5.1.1 Removable Media Boot Behavior

> > >>>>> For the efi bootmeth we can actually do what we like, can't we? Its
> > >>>>> behaviour isn't defined by the spec as I understand it.
> > >>>>>
> > >>>>> I wonder how this works, if the user installs both Debian and Ubuntu
> > >>>>> on the machine? I suppose each one separately updates its grub.cfg
> > >>>>> with all the OSes it can find on the machine? Then just one of them
> > >>>>> gets to be the initial .efi app?
> > >>>>>
> > >>>>> Can we work together on adding a new, separate file which lists the
> > >>>>> .efi files and what they are for?
> > >>>>
> > >>>> There is already a place defined by the UEFI specification where to put
> > >>>> that information: the UEFI boot options defined in variables BOOT####.
> > >>>> And this is exactly what all operating system use. The information what
> > >>>> the entries are for are in the label part of the boot options. The
> > >>>> sequence in which to try loading is in variable BootOrder.
> > >>>>
> > >>>> There is no point in reinventing the wheel and making it square.
> > >>>
> > >>> You have not addressed my question, or I don't understand your answer.
> > >>>
> > >>> I am not worried about what the computer does. I am worried about what
> > >>> the user sees.
> > >>>
> > >>> So far as I can tell there is currently no way to know what you are
> > >>> booting (Ubuntu, Debian) until you boot it
> > >>
> > >> Of course I know. These are some of the boot options on my laptop:
> > >>
> > >> Boot0000* Windows Boot Manager
> > >> HD(2,GPT,54e58a37-c1db-4c6b-afda-24340c39cda5,0x109000,0x32000)/File(\EFI\Microsoft\Boot\bootmgfw.efi)WINDOWS.........x...B.C.D.O.B.J.E.C.T.=.{.9.d.e.a.8.6.2.c.-.5.c.d.d.-.4.e.7.0.-.a.c.c.1.-.f.3.2.b.3.4.4.d.5.3.7.2.}...M................
> > >> Boot0001* debian
> > >> HD(2,GPT,54e58a14-c1db-4c6b-afda-24340c39cda5,0x109000,0x32000)/File(\EFI\debian\shimx64.efi)
> > >> Boot0002* ubuntu
> > >> HD(2,GPT,54e58a14-c1db-4c6b-afda-24340c39cda5,0x109000,0x32000)/File(\EFI\ubuntu\shimx64.efi)
> > >> Boot0063* EFI Shell
> > >> HD(2,GPT,54e58a14-c1db-4c6b-afda-24340c39cda5,0x109000,0x32000)/File(Shell.efi)
> > >>
> > >> If I open the boot menu (F12 key on a Lenovo laptop, F8 key on an ASUS
> > >> mainboards), I see a menu which allows me to choose which of Windows,
> > >> Debian, Ubuntu, or the EFI Shell I want to boot.
> > >>
> > >> In U-Boot we already have the same. Please, have a look at the section
> > >> 'UEFI boot variable enumeration' in doc/usage/cmd/bootmenu.rst.
> > >
> > > OK thank you. So what sets that up? When you install Ubuntu does it
> > > add a new Boot0002 variable? Then you install Debian and it adds some
> > > other one??
> >
> > Yes, the OS installation process sets up the boot variables. For Debian
> > or Ubuntu this happens when shim or GRUB is installed. If Debian or
> > Ubuntu shim is started as /efi/boot/boot<arch>.efi, it also adds the
> > missing entry based on a CSV file.
> >
> > E.g. on my laptop I find /EFI/ubuntu/BOOTX64.CSV with content
> >
> >    shimx64.efi,ubuntu,,This is the boot entry for ubuntu
> >
> > and file /boot/efi/EFI/debian/BOOTX64.CSV with content
> >
> >    shimx64.efi,debian,,This is the boot entry for debian
>
> OK, thank you. That explains the link to the boot order. But what
> creates these CSV files? It is the os prober, or is it Ubuntu/Debian?

AFAIK, these are created by the OS installer programme by running
grub-install.

>
> >
> > >
> > > Also the information is very limited, e.g. it does not show the Ubuntu
> > > version, kernel version, etc.
> >
> > The length of the labels is only limited by available storage. It is the
> > OS'es choice to opt for a such a short label.
>
> Yes, really I am looking for metadata that includes more detail at
> different leaves, not just a label.
>

The job of these EFI is to load grub bootloader and rest all details
(kernel version,
OS, etc) is supposed to be handled by Grub.

[0] - https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[1] - https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Library/UefiBootManagerLib/BmBoot.c#L2471

Kind Regards,
Shantur