[U-Boot] Hart lottery and CONFIG_XIP

[Bin Meng]

Hi Troy,

On Mon, Jun 3, 2019 at 10:53 PM Troy Benjegerdes
<troy.benjegerdes at sifive.com> wrote:
>
>
>
> > On Jun 3, 2019, at 9:19 AM, Bin Meng <bmeng.cn at gmail.com> wrote:
> >
> > +Anup
> >
> > Hi Troy,
> >
> > On Mon, Jun 3, 2019 at 9:19 PM Troy Benjegerdes
> > <troy.benjegerdes at sifive.com> wrote:
> >>
> >>
> >>
> >>> On Jun 2, 2019, at 9:22 PM, Rick Chen <rickchen36 at gmail.com> wrote:
> >>>
> >>> Hi Troy
> >>>
> >>>>> From: Troy Benjegerdes [mailto:troy.benjegerdes at sifive.com]
> >>>>> Sent: Saturday, June 01, 2019 12:24 AM
> >>>>> To: Auer, Lukas
> >>>>> Cc: Rick Jian-Zhi Chen(陳建志); bmeng.cn at gmail.com; palmer at sifive.com
> >>>>> Subject: Re: Hart lottery and CONFIG_XIP
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On May 31, 2019, at 9:13 AM, Auer, Lukas <lukas.auer at aisec.fraunhofer.de>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi Troy,
> >>>>>>
> >>>>>> On Fri, 2019-05-31 at 08:18 -0500, Troy Benjegerdes wrote:
> >>>>>>> Wouldn't the following line in head.S fail when running from flash
> >>>>>>> (although maybe not in a way that prevents booting)
> >>>>>>>
> >>>>>>>      /* save the boot hart id to global_data */
> >>>>>>>      SREG    tp, GD_BOOT_HART(gp)
> >>>>>>>
> >>>>>>> Shouldn’t this be protected by CONFIG_XIP?
> >>>>>>
> >>>>>> The boot hart ID is stored in global data, which is allocated from the
> >>>>>> stack (in board_init_f_alloc_reserve() ). It is therefore writable and
> >>>>>> won't cause any issues when running from flash.
> >>>>>
> >>>>> Sorry about the confusion, I was reading it wrong earlier.
> >>>>>
> >>>>> I’m hoping to have a couple of patches ready later today to change the
> >>>>> CONFIG_XIP patch to ‘CONFIG_HART_LOTTERY’ since it is also useful to
> >>>>> remove the potential indeterminism of which hart wins the lottery when doing
> >>>>> board bringup and debugging.
> >>>
> >>> I am OK with that.
> >>> Actually my preliminary patch about
> >>> [PATCH 0/4] AE350 support SMP boot from flash
> >>> did as your wish.
> >>>
> >>> You can refer it :
> >>> [PATCH 1/4] riscv: hart_lottery and available harts feature can be seletable
> >>> https://www.mail-archive.com/u-boot@lists.denx.de/msg323419.html
> >>>
> >>> Rick
> >>
> >> To follow up on https://www.mail-archive.com/u-boot@lists.denx.de/msg323526.html
> >>
> >> It is confusing and misleading to mix CONFIG_XIP and CONFIG_HART_LOTTERY.
> >>
> >
> > I am not sure what caused the confusion. CONFIG_XIP was added to
> > support U-Boot executing from ROM directly.
> >
>
> The confusion is use cases where you don’t necessarily need CONFIG_XIP,
> but you do want deterministic SMP booting, and the code is a lot more understandable
> with ‘#ifdef CONFIG_HART_LOTTERY’, and a Kconfig option and/or documentation
> warning that says CONFIG_HART_LOTTERY depends on !CONFIG_XIP
>

OK, as I pointed out, the deterministic SMP booting was originally
posted, but later was changed to the lottery mechanism per Anup's
review comments. I am personally fine with both.

> >> From an system testing and validation point of view, I would find it much better
> >> (especially at very early boot stages, where U-boot might be the first non-ROM code
> >> running) to have a deterministic process to determine what core runs U-boot. This
> >
> > I remember when SMP patches were submitted by Lukas for the first time
> > it was deterministic using some macros like CONFIG_BOOT_HART, however
> > per Anup's request, the hart lottery feature, similar to what Linux
> > has, was added.
>
> I’d like to have CONFIG_BOOT_HART available as a config option as well, particularly
> for system validation testing. Also along those lines, if we are going to use a lottery
> to determine what CPU boots the system, how do we know afterwards which one
> it was?
>

Yes, there is a way to know. Type 'dm tree' from U-Boot shell, then
look at which cpu node has a child node of "riscv_timer".

Regards,
Bin