[PATCH v4 00/33] Initial implementation of standard boot

[Tom Rini]

On Thu, Mar 24, 2022 at 02:04:57PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Thu, 24 Mar 2022 at 13:27, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Thu, Mar 24, 2022 at 12:33:38PM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Thu, 24 Mar 2022 at 08:46, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Wed, Mar 23, 2022 at 08:13:42PM -0600, Simon Glass wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Wed, 23 Mar 2022 at 14:09, Tom Rini <trini at konsulko.com> wrote:
> > > > > >
> > > > > > On Wed, Mar 23, 2022 at 12:45:50PM -0600, Simon Glass wrote:
> > > > > > > Hi Tom,
> > > > > > >
> > > > > > > On Wed, 23 Mar 2022 at 08:05, Tom Rini <trini at konsulko.com> wrote:
> > > > > > > >
> > > > > > > > On Sun, Mar 06, 2022 at 05:49:43AM -0700, Simon Glass wrote:
> > > > > > > > >
> > > > > > > > > The bootflow feature provide a built-in way for U-Boot to automatically
> > > > > > > > > boot an Operating System without custom scripting and other customisation.
> > > > > > > > > This is called 'standard boot' since it provides a standard way for
> > > > > > > > > U-Boot to boot a distro, without scripting.
> > > > > > > > >
> > > > > > > > > It introduces the following concepts:
> > > > > > > > >
> > > > > > > > >    - bootdev - a device which can hold a distro
> > > > > > > > >    - bootmeth - a method to scan a bootdev to find bootflows (owned by
> > > > > > > > >                 U-Boot)
> > > > > > > > >    - bootflow - a description of how to boot (owned by the distro)
> > > > > > > > >
> > > > > > > > > This series provides an implementation of these, enabled to scan for
> > > > > > > > > bootflows from MMC, USB and Ethernet. It supports the existing distro
> > > > > > > > > boot as well as the EFI loader flow (bootefi/bootmgr). It works
> > > > > > > > > similiarly to the existing script-based approach, but is native to
> > > > > > > > > U-Boot.
> > > > > > > > >
> > > > > > > > > With this we can boot on a Raspberry Pi 3 with just one command:
> > > > > > > > >
> > > > > > > > >    bootflow scan -lb
> > > > > > > > >
> > > > > > > > > which means to scan, listing (-l) each bootflow and trying to boot each
> > > > > > > > > one (-b). The final patch shows this.
> > > > > > > > >
> > > > > > > > > With a standard way to identify boot devices, booting become easier. It
> > > > > > > > > also should be possible to support U-Boot scripts, for backwards
> > > > > > > > > compatibility only.
> > > > > > > > >
> > > > > > > > > This series relies on the PXE clean-up series, posted here:
> > > > > > > > >
> > > > > > > > >    https://patchwork.ozlabs.org/project/uboot/list/?series=267078
> > > > > > > > >
> > > > > > > > > For documentation, see the 'doc' patch.
> > > > > > > > >
> > > > > > > > > For version 2, a new naming scheme is used as above:
> > > > > > > > >
> > > > > > > > >    - bootdev is used instead of bootdevice, because 'device' is overused,
> > > > > > > > >        is everywhere in U-Boot, can be confused with udevice
> > > > > > > > >    - bootmeth - because 'method' is too vanilla, appears 1300 times in
> > > > > > > > >        U-Boot
> > > > > > > > >
> > > > > > > > > Also in version 2, drivers are introduced for the boot methods, to make
> > > > > > > > > it more extensible. Booting a custom OS is simply a matter of creating a
> > > > > > > > > bootmeth for it and implementing the read_file() and boot() methods.
> > > > > > > > >
> > > > > > > > > Version 4 makes some minor improvements and leaves out the RFC patch for
> > > > > > > > > rpi conversion, in the hope of getting the base support applied sooner
> > > > > > > > > rather than later.
> > > > > > > > >
> > > > > > > > > The design is described in these two documents:
> > > > > > > > >
> > > > > > > > > https://drive.google.com/file/d/1ggW0KJpUOR__vBkj3l61L2dav4ZkNC12/view?usp=sharing
> > > > > > > > >
> > > > > > > > > https://drive.google.com/file/d/1kTrflO9vvGlKp-ZH_jlgb9TY3WYG6FF9/view?usp=sharing
> > > > > > > >
> > > > > > > > I keep putting off commenting more here, but, I still feel this is the
> > > > > > > > wrong direction.  What problems do we have today with distro boot?
> > > > > > > > Well, we haven't figured out how to move configuring it out of the board
> > > > > > > > config.h file.  But that's just one of a half dozen or so examples of
> > > > > > > > how we haven't figured out a good solution to configuring the default
> > > > > > > > environment.  And only some of those other examples are boot related
> > > > > > > > (the NXP chain of trust booting stuff is another boot example, ETHPRIME,
> > > > > > > > HOSTNAME, etc, are non-boot examples).
> > > > > > > >
> > > > > > > > We also aren't improving testing of "can we boot" here, because what
> > > > > > > > THAT needs is setting up LAVA and booting some installers on some
> > > > > > > > hardware (and some QEMU).  That's testing that Linux boot works.  Today
> > > > > > > > we have tests for hush parsing, and if distro boot makes use of
> > > > > > > > something we don't have a test for, we need a test for it.  This adds
> > > > > > > > tests for itself, which is good.
> > > > > > > >
> > > > > > > > And I still don't see an example of where this demonstrates that
> > > > > > > > existing non-UEFI boot cases are now easier to handle or cleaner to
> > > > > > > > handle or otherwise better.
> > > > > > > >
> > > > > > > > In that this is an attempt to tackle one of the long standing needed
> > > > > > > > migrations (be able to drop board config.h files), something here needs
> > > > > > > > doing.  But I don't see this as the right direction, sorry.
> > > > > > >
> > > > > > > Does anyone have a better idea for all of this? This is a solid base
> > > > > > > we can build on but we can't make any progress while this is just
> > > > > > > patches. What not apply it and we can move forward?
> > > > > > >
> > > > > > > - solves the env problem for distro boot in that we don't need the scripts
> > > > > >
> > > > > > Solves what problem?  That distro boot is done in hush?  Yes, we have
> > > > > > many examples of needing to define the default environment in a
> > > > > > non-static way.  This is not a solution to that problem.
> > > > >
> > > > > It removes the need for the boot scripts that are currently in the
> > > > > environment. I'm not sure what you mean by defining a default
> > > > > environment in a non-static way.
> > > >
> > > > I'm not sure what you're missing.  We have a number of examples today of
> > > > modifying the default environment based on CONFIG options.  How we do
> > > > the generic distro boot is one of those examples, and you're trying to
> > > > replace it.  It should be able to address the NXP (fsl_...) secure boot
> > > > scripts as well.  It doesn't address at all the non-boot related things.
> > >
> > > The text environment does support CONFIG options.
> >
> > Then I misremembered or misunderstood something you had said previously,
> > sorry.
> >
> > > > > > > - gets rid of the scripts which are a confusing mess
> > > > > >
> > > > > > Yes, a generic "figure out where a bootable might be, boot it" is
> > > > > > complex.  That's basically why the UEFI boot manager code exists, and
> > > > > > replaces that logic.
> > > > >
> > > > > I thought we already decided that U-Boot was going to continue to
> > > > > support non-UEFI booting?
> > > >
> > > > Yes?  CONFIG_BOOTCOMMAND can be whatever you like.  No one is talking at
> > > > all anywhere about removing anything that's not bootefi.  I don't
> > > > understand your point here.
> >
> > I want to come back here for a moment because maybe we're talking at
> > cross purposes.  I absolutely want there to be CONFIG_BOOTCOMMAND and/or
> > non-hush-based (for device security) ways to run whatever, to boot the
> > OS/next stage.
> >
> > But for something to deal with the combinatorial matrix hell of
> > heuristic and then some configurability (or not!) of find out which
> > arbitrary devices to boot from, or not boot from, or override booting
> > from and so on, no, I don't want to DIY something, I want the UEFI
> > standard for that.  If there's some other standard for dealing with all
> > of that as well, OK, lets talk about implementing it.  If UEFI standard
> > could be improved in some way, well, lets push back and get it improved.
> > But I do not want another implementation of find an arbitrary thing and
> > boot it.  I'm not sure who is asking for that, either.
> 
> That is quite a change in what I thought you meant, but of course
> email is a terrible medium for this sort of thing. What you say seems
> to rule out bootstd, because the goal there is to provide a means to
> systematically boot things that might be UEFI, but might not be. The
> UEFI boot manager will presumably grow in usage, but it only covers
> UEFI cases. Even that is not good IMO because the current
> implementation is a bit of a mess, e.g. the hacks to decide on boot
> devices and detect when things are loaded. I believe bootstd can
> improve those bottom-level pieces of UEFI. Anyway, for everything
> else, UEFI is not useful.
> 
> So please let's be clear about this. I suspect I would struggle to
> maintain any professional or personal interest in a UEFI-only
> bootloader, or where the only innovation going on is on the UEFI side.

Then perhaps we need to step back and figure out what to move forward
on.  The combinational nightmare that is an arbitrary set of devices
times permissions times override times per device / SoC limitations is
something I want to avoid duplication of.  But I also think in some
ways, that's the less interesting case.  Embedded devices tend to boot
from point A (or point B, or perhaps golden master point C), which
doesn't require massive loads of discovery and probing work.  Of course,
which SDIO is an eMMC vs an SD slot vs a WiFi card is per-board.

> > > > > > > - provides proper high-level concepts of boot device and boot method
> > > > > >
> > > > > > I don't see the inherent value.  Or maybe to put things another way, the
> > > > > > "now go boot" API has always been implemented via hush, even when the
> > > > > > interactive part is disabled.  You're providing a programatic way of
> > > > > > doing it instead.  And I'm not seeing it being more flexible / easier to
> > > > > > implement.  It's really sounded harder, not easier, than #define
> > > > > > BOOT_TARGET_DEVICES.
> > > > >
> > > > > It is easier in that you don't need to define the boot targets. It is
> > > > > done automatically using the devices that are available.
> > > >
> > > > So you're replacing defining the system to be correct with heuristics to
> > > > guess that it's correct?
> > > >
> > > > > > > - allows testing of the U-Boot part of 'can we boot' because we have
> > > > > > > tests for all the cases - we can expand this over time
> > > > > >
> > > > > > No, it doesn't.  It provides a different interface and tests for itself.
> > > > > > If it's not LAVA (or something like that) booting real Linux
> > > > > > distribution images it's not a real can we boot test.
> > > > >
> > > > > I didn't say it would test booting linux or real distros. You
> > > > > obviously need LAVA (or something like that) for doing that.
> > > > >
> > > > > But it does provide tests for the boot flow which are completely
> > > > > lacking at present. So at least we are testing the code booting code.
> > > >
> > > > We have hush tests, the distro boot logic is therefore tested as well as
> > > > what you're adding here as neither of them are today testing that we can
> > > > actually boot something.
> > > >
> > > > > > > - allows non-UEFI boot cases like Chrome OS, which is currently just a
> > > > > > > hack for one board[1]
> > > > > >
> > > > > > I don't see a Chrome OS example in your series.
> > > > >
> > > > > Would that help? I don't mind doing it if we can make some progress,
> > > > > but your previous objection was about code size. I addressed that and
> > > > > have not heard any comment about that.
> > > >
> > > > I object to the whole thing.  The lack of implementations where this is
> > > > showing that it's actually better than what we have today is just one of
> > > > the problems.  That we replace something that's basically size-free and
> > > > replace it with several kilobytes is another problem.
> > > >
> > > > > > > - provides a programmatic base for A/B boot, etc.
> > > > > >
> > > > > > And I've noted before that I don't see how this makes life easier for
> > > > > > any of the existing A/B mechanisms.  Some of which are just going with
> > > > > > "Hey, just UEFI boot and let grub deal with it".
> > > > >
> > > > > It makes it much easier since we now have the means to locate and read
> > > > > in boot flows. Then it is just a case of selecting the one we want,
> > > > > e.g. by examining the list of bootflows and selecting one. It is a
> > > > > helpful division of labour.
> > > >
> > > > That's not the hard part of integrating an A/B update mechanism or
> > > > implementing it for a given board.
> > > >
> > > > > > > I feel the same way with Takahiro's series, which has been out-of-tree
> > > > > > > for too long.
> > > > > > >
> > > > > > > Please reconsider this. What do we have to lose?
> > > > > >
> > > > > > What we'll gain seems to be another chunk of code that needs boards to
> > > > > > migrate to, maybe.  And I don't see examples where this, now, or in the
> > > > > > next iteration, provides a smaller, or clearer implementation of
> > > > > > something.
> > > > >
> > > > > Yes it requires boards to migrate, although in fact the migration can
> > > > > be driven by features rather than board-by-board. For example, we
> > > > > could migrate boards that use USB, MMC, etc. first, and then migrate
> > > > > more as we add more features.
> > > > >
> > > > > But at least it is a chunk of code, with full tests. The scripts are
> > > > > just a pain.
> > > >
> > > > So we just add unused code until someone makes use of it?  We already
> > > > have that with the environment rework that can't be really used because
> > > > it doesn't handle distro boot support, which this attempts to handle by
> > > > removing that functionality from the environment and to C code.
> > >
> > > My summary of the above is that you see a lot of other/bigger problems
> > > and want to solve those instead. But that is not the purpose of this
> > > series.
> > >
> > > >
> > > > And as I'm testing some Kconfig migrations in another window and seeing
> > > > what feels like an endless scroll of "migrate to ..." messages flow by,
> > > > I have little confidence in more than a few boards getting migrated to
> > > > this functionality, while leaving the to me bigger problems entirely
> > > > unresolved (how do we delete board config.h files when the environment
> > > > needs CONFIG based tweaking, which is more than just distro boot).
> > >
> > > This seems to be the nub of it. I am reluctant to invest more time in
> > > this when there is so little mindshare about the problems we face. You
> > > are reluctant to move it forward when you worry it will just be
> > > another migration effort that leads nowhere.
> > >
> > > I am more than happy to commit to migrating all boards over to bootstd
> > > and removing the distro scripts from the environment. This is a
> > > per-feature migration, rather than per-board.
> > >
> > > Does that help?
> >
> > I see a large number of problems that need to be solved, most urgently,
> > in U-Boot.  A good subset of these are DM migration related and you're
> > in the best position to make some initial passes at.  I'm also not
> > convinced that this series has been worked far enough to be an
> > improvement on the boot cases that are NOT "just use UEFI boot manager".
> > And the time / effort for getting there I'd personally rather see put in
> > to some other migrations / improvements on the DM front.
> 
> This is off-topic, but I'm not even sure what the DM issues are.

High level, memory pressure.  As talked about in that thread about
SYS_MALLOC_F_LEN, we have many SoCs that defaulted to 0x1000/0x2000 are
increasing it well past that on some boards, but not all.  A 'git log'
checking for commits that increase SYS_MALLOC_F_LEN shows it's not just
the specific clock tree changes doing it.

> Perhaps you could provide a list here, or perhaps in the issue tracker
> if you like. To me, a big challenge is that we are slow to finish
> migrations like Kconfig and relaxed about boards which miss the
> deadlines. For example, if we could just get the CONFIG_SPL stuff
> finished then we could migrate away from the SPL_TPL_ /
> CONFIG_IS_ENABLED() stuff. Surely no one expected the Kconfig
> migration to take 8 years...

So yes, no one expected Kconfig to take so long to migrate to, I had
hoped for more people to pick up and move values.  The problem being a
lot of them are hard.  I just today finished doing
CONFIG_SYS_MONITOR_BASE so that we could kill off
CONFIG_SYS_EXTRA_OPTIONS and that required dropping in CONFIG_VAL()
usage for MPC85xx.  I'm confident enough I got things right, but there's
a bunch of little changes this also introduces as platforms had been
doing stuff like:
#ifdef CONFIG_SPL_BUILD
#define CONFIG_SYS_MONITOR_BASE ...
#endif
and (a) not strictly needing the value I think and (b) also missing out
on the places in generic code where we check for that value.

And frankly, at this point I'd rather see some blind but should work
conversions of boards / drivers because given our long standing well
defined release schedule, if a broken migration comes in to when the
-next merge window opens, and it's not found to be a problem until after
the release, that speaks to a need to have vendors boards more,
especially so long as the migration patches cc the maintainers, since
every defconfig has a maintainer these days.

> Re bootstd it seems completely obvious to me that U-Boot needs this
> since it creates a much-needed structure for finding and configuring
> boot devices and things to boot. That is perhaps why I am so lost
> here. I think UEFI is muddying the waters hugely, so perhaps this
> would have been a much easier discussion 5 years ago.

Perhaps?  In my mind the hard part about "things to boot" is when you
want something to work for some arbitrary set of board and storages.
But every board knows what storages are possibly valid.  And that's part
of "treat this like a desktop PeeCee" rather than an embedded device.
My thermostat (which has an imx6 in it) doesn't need to figure out
what's where, it knows which SDIO has an eMMC (I think, haven't dared
crack it open) and which is probably a wilink.

Which is why I guess I keep coming back to things like the existing off
the shelf update mechanisms.  The pain points to any of them aren't
figuring out which device / partition to use (since much of that has to
be configured so the mechanism knows where to write things to) but
rather integration of the environment and ensuring required support is
enabled and so forth.

The use case of "I want to boot $BIGNAME_DISTRO on $RANDOM_SBC_EVAL" has
seemingly all of the people in the former asking for UEFI bootmanager
booting since that defines all of the hows and whys to find things on
the latter.  But that leaves what I tend to think of has a rather huge
swath of use cases unhandled.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20220324/05a95aa7/attachment.sig>