[U-Boot] Hart lottery and CONFIG_XIP

[Anup Patel]

On Tue, Jun 4, 2019 at 3:00 PM Auer, Lukas
<lukas.auer at aisec.fraunhofer.de> wrote:
>
> On Tue, 2019-06-04 at 09:42 +0800, Rick Chen wrote:
> > Hi Troy
> >
> > Bin Meng <bmeng.cn at gmail.com> 於 2019年6月3日 週一 下午10:56寫道：
> > > Hi Troy,
> > >
> > > On Mon, Jun 3, 2019 at 10:53 PM Troy Benjegerdes
> > > <troy.benjegerdes at sifive.com> wrote:
> > > >
> > > >
> > > > > On Jun 3, 2019, at 9:19 AM, Bin Meng <bmeng.cn at gmail.com> wrote:
> > > > >
> > > > > +Anup
> > > > >
> > > > > Hi Troy,
> > > > >
> > > > > On Mon, Jun 3, 2019 at 9:19 PM Troy Benjegerdes
> > > > > <troy.benjegerdes at sifive.com> wrote:
> > > > > >
> > > > > >
> > > > > > > On Jun 2, 2019, at 9:22 PM, Rick Chen <rickchen36 at gmail.com> wrote:
> > > > > > >
> > > > > > > Hi Troy
> > > > > > >
> > > > > > > > > From: Troy Benjegerdes [mailto:troy.benjegerdes at sifive.com]
> > > > > > > > > Sent: Saturday, June 01, 2019 12:24 AM
> > > > > > > > > To: Auer, Lukas
> > > > > > > > > Cc: Rick Jian-Zhi Chen(陳建志); bmeng.cn at gmail.com; palmer at sifive.com
> > > > > > > > > Subject: Re: Hart lottery and CONFIG_XIP
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > On May 31, 2019, at 9:13 AM, Auer, Lukas <lukas.auer at aisec.fraunhofer.de>
> > > > > > > > > wrote:
> > > > > > > > > > Hi Troy,
> > > > > > > > > >
> > > > > > > > > > On Fri, 2019-05-31 at 08:18 -0500, Troy Benjegerdes wrote:
> > > > > > > > > > > Wouldn't the following line in head.S fail when running from flash
> > > > > > > > > > > (although maybe not in a way that prevents booting)
> > > > > > > > > > >
> > > > > > > > > > >      /* save the boot hart id to global_data */
> > > > > > > > > > >      SREG    tp, GD_BOOT_HART(gp)
> > > > > > > > > > >
> > > > > > > > > > > Shouldn’t this be protected by CONFIG_XIP?
> > > > > > > > > >
> > > > > > > > > > The boot hart ID is stored in global data, which is allocated from the
> > > > > > > > > > stack (in board_init_f_alloc_reserve() ). It is therefore writable and
> > > > > > > > > > won't cause any issues when running from flash.
> > > > > > > > >
> > > > > > > > > Sorry about the confusion, I was reading it wrong earlier.
> > > > > > > > >
> > > > > > > > > I’m hoping to have a couple of patches ready later today to change the
> > > > > > > > > CONFIG_XIP patch to ‘CONFIG_HART_LOTTERY’ since it is also useful to
> > > > > > > > > remove the potential indeterminism of which hart wins the lottery when doing
> > > > > > > > > board bringup and debugging.
> > > > > > >
> > > > > > > I am OK with that.
> > > > > > > Actually my preliminary patch about
> > > > > > > [PATCH 0/4] AE350 support SMP boot from flash
> > > > > > > did as your wish.
> > > > > > >
> > > > > > > You can refer it :
> > > > > > > [PATCH 1/4] riscv: hart_lottery and available harts feature can be seletable
> > > > > > > https://www.mail-archive.com/u-boot@lists.denx.de/msg323419.html
> > > > > > >
> > > > > > > Rick
> > > > > >
> > > > > > To follow up on https://www.mail-archive.com/u-boot@lists.denx.de/msg323526.html
> > > > > >
> > > > > > It is confusing and misleading to mix CONFIG_XIP and CONFIG_HART_LOTTERY.
> > > > > >
> > > > >
> > > > > I am not sure what caused the confusion. CONFIG_XIP was added to
> > > > > support U-Boot executing from ROM directly.
> > > > >
> > > >
> > > > The confusion is use cases where you don’t necessarily need CONFIG_XIP,
> > > > but you do want deterministic SMP booting, and the code is a lot more understandable
> > > > with ‘#ifdef CONFIG_HART_LOTTERY’, and a Kconfig option and/or documentation
> > > > warning that says CONFIG_HART_LOTTERY depends on !CONFIG_XIP
> > > >
> > >
> > > OK, as I pointed out, the deterministic SMP booting was originally
> > > posted, but later was changed to the lottery mechanism per Anup's
> > > review comments. I am personally fine with both.
> >
> > Yes. I am agree with that.
> > But it shall also support booting form ROM and RAM as now.
>
> Just to give a bit of context information. Anup's reasoning was that
> with a fixed boot hart, we have to rely on it to work. If there is a
> hardware failure in just the boot hart, the system won't be able to
> boot anymore. With a hart lottery that is not a problem.
>
> With that in mind, it makes sense to have to option of a fixed boot
> hart, but I would not enable that option in any of the upstream
> defconfigs. That way it can be manually enabled if required for the
> application.

Sorry for pitching in late.

Apart from hardware failure on fixed boot hart, another reason in
support of lottery based system is that it will work nicely with upcoming
SBI HART power managment where more HARTs/CPUs are kept in
power-off state by OpenSBI (RUNTIME firmware) and Linux will
power-on HARTs/CPUs as required at boot-time/run-time.

Regards,
Anup