[U-Boot] Hart lottery and CONFIG_XIP

[Troy Benjegerdes]

> On Jun 3, 2019, at 9:19 AM, Bin Meng <bmeng.cn at gmail.com> wrote:
> 
> +Anup
> 
> Hi Troy,
> 
> On Mon, Jun 3, 2019 at 9:19 PM Troy Benjegerdes
> <troy.benjegerdes at sifive.com> wrote:
>> 
>> 
>> 
>>> On Jun 2, 2019, at 9:22 PM, Rick Chen <rickchen36 at gmail.com> wrote:
>>> 
>>> Hi Troy
>>> 
>>>>> From: Troy Benjegerdes [mailto:troy.benjegerdes at sifive.com]
>>>>> Sent: Saturday, June 01, 2019 12:24 AM
>>>>> To: Auer, Lukas
>>>>> Cc: Rick Jian-Zhi Chen(陳建志); bmeng.cn at gmail.com; palmer at sifive.com
>>>>> Subject: Re: Hart lottery and CONFIG_XIP
>>>>> 
>>>>> 
>>>>> 
>>>>>> On May 31, 2019, at 9:13 AM, Auer, Lukas <lukas.auer at aisec.fraunhofer.de>
>>>>> wrote:
>>>>>> 
>>>>>> Hi Troy,
>>>>>> 
>>>>>> On Fri, 2019-05-31 at 08:18 -0500, Troy Benjegerdes wrote:
>>>>>>> Wouldn't the following line in head.S fail when running from flash
>>>>>>> (although maybe not in a way that prevents booting)
>>>>>>> 
>>>>>>>      /* save the boot hart id to global_data */
>>>>>>>      SREG    tp, GD_BOOT_HART(gp)
>>>>>>> 
>>>>>>> Shouldn’t this be protected by CONFIG_XIP?
>>>>>> 
>>>>>> The boot hart ID is stored in global data, which is allocated from the
>>>>>> stack (in board_init_f_alloc_reserve() ). It is therefore writable and
>>>>>> won't cause any issues when running from flash.
>>>>> 
>>>>> Sorry about the confusion, I was reading it wrong earlier.
>>>>> 
>>>>> I’m hoping to have a couple of patches ready later today to change the
>>>>> CONFIG_XIP patch to ‘CONFIG_HART_LOTTERY’ since it is also useful to
>>>>> remove the potential indeterminism of which hart wins the lottery when doing
>>>>> board bringup and debugging.
>>> 
>>> I am OK with that.
>>> Actually my preliminary patch about
>>> [PATCH 0/4] AE350 support SMP boot from flash
>>> did as your wish.
>>> 
>>> You can refer it :
>>> [PATCH 1/4] riscv: hart_lottery and available harts feature can be seletable
>>> https://www.mail-archive.com/u-boot@lists.denx.de/msg323419.html
>>> 
>>> Rick
>> 
>> To follow up on https://www.mail-archive.com/u-boot@lists.denx.de/msg323526.html
>> 
>> It is confusing and misleading to mix CONFIG_XIP and CONFIG_HART_LOTTERY.
>> 
> 
> I am not sure what caused the confusion. CONFIG_XIP was added to
> support U-Boot executing from ROM directly.
> 

The confusion is use cases where you don’t necessarily need CONFIG_XIP,
but you do want deterministic SMP booting, and the code is a lot more understandable
with ‘#ifdef CONFIG_HART_LOTTERY’, and a Kconfig option and/or documentation
warning that says CONFIG_HART_LOTTERY depends on !CONFIG_XIP

>> From an system testing and validation point of view, I would find it much better
>> (especially at very early boot stages, where U-boot might be the first non-ROM code
>> running) to have a deterministic process to determine what core runs U-boot. This
> 
> I remember when SMP patches were submitted by Lukas for the first time
> it was deterministic using some macros like CONFIG_BOOT_HART, however
> per Anup's request, the hart lottery feature, similar to what Linux
> has, was added.

I’d like to have CONFIG_BOOT_HART available as a config option as well, particularly
for system validation testing. Also along those lines, if we are going to use a lottery
to determine what CPU boots the system, how do we know afterwards which one
it was?

> 
>> necessarily seems as it will require a CONFIG_BOOT_HART option, as we are not
>> in a position to parse a device tree in head.S
>> 
> 
> Yes, this is understood. So the CONFIG_XIP was added to support such
> case, for which the lottery feature relies on a memory location which
> isn't writable yet.
> 
> Regards,
> Bin