OpenBSI and U-Boot

[Heinrich Schuchardt]

On 8/11/20 3:55 AM, Rick Chen wrote:
> Hi Heinrich
>
>> Am 9. August 2020 22:08:23 MESZ schrieb Atish Patra <atishp at atishpatra.org>:
>>> On Sat, Aug 8, 2020 at 9:17 AM Heinrich Schuchardt <xypron.glpk at gmx.de>
>>> wrote:
>>>>
>>>> On 8/8/20 5:32 PM, Sean Anderson wrote:
>>>>> On 8/8/20 10:59 AM, Heinrich Schuchardt wrote:
>>>>>> Hello Anup,
>>>>>>
>>>>>> I have looking at you OpenSBI code firmware/payloads/test_head.S.
>>> Here
>>>>>
>>>>> I think the real start is in firmware/fw_base.S. From there,
>>> secondary
>>>>> harts loop first in _wait_relocate_copy_done, and then in
>>>>> _wait_for_boot_hart, and then they execute the next stage via
>>>>> _start_warm and sbi_init.
>>>>>
>>>>>> like in U-Boot's common/spl/spl_opensbi.c you put all but one hart
>>> in to
>>>>>> an enless loop (hang).
>>>>>
>>>>> As far as I can tell, U-Boot has all harts execute the next stage
>>> when
>>>>> SMP is enabled. smp_call_function has all harts execute that
>>> function.
>>>>
>>>> U-Boot can only run on one hart. Are the other harts trapped in
>>>> secondary_hart_loop()? How do we ensure that an UEFI payload does not
>>>> overwrite this memory location?
>>>>
>>>
>>> If you are booting Linux, U-Boot runs in S-mode and SMP is disabled.
>>
>> Would that also hold true on the Kendryte K210? For all what can see the secondary hart enters U-Boot and is only restrained by WFI and otherwise kept in an endless loop.
>>
>> I am wondering if that endless loop needs to be marked as reserved memory for Linux.
>
> Currently if U-Boot run in S-Mode, SMP is disable, there will exist
> OpenSBI version compatible issue.
> You shall use OpenSBI v0.7 with HSM, thus it will trap the other harts
> in OpenSBI and only main hart will jump to kernel from U-Boot proper.
>
> Thanks,
> Rick
>

HSM is described here:
https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc#hart-state-management-extension-extension-id-0x48534d-hsm

Currently the Kendryte K210 has no SPL. So OpenSBI would be running
before U-Boot.

Will OpenSBI v0.7 by itself stop the other harts or do we need to call
sbi_hart_stop() in U-Boot? Unfortunately opensbi/README.md and
riscv-sbi.adoc both leave this open.

Best regards

Heinrich

>>
>>> All memory regions containing OpenSBI code are reserved. Thus, U-Boot
>>> won't touch that.
>>> U-Boot also marks the run time services memory region as reserved as
>>> well.
>>> Thus, Linux kernel won't touch any of those memory regions either.
>>>
>>>> spl_secondary_hart_stack_gd_setup() can jump to hang() if the call to
>>>> secondary_hart_relocate() fails.
>>>>
>>>>>
>>>>>>
>>>>>> When Linux boots via UEFI it will wake up the extra harts after
>>>>>> ExitBootServices(). So I assume we should define function hang()
>>> in
>>>>>> lib/hang.c as __efi_runtime to avoid seeing it overwritten by the
>>> EFI
>>>>>> payload.
>>>>>>
>>>>>> @Ard:
>>>>>> Does Linux take care of the hanging harts and redirect them to its
>>> own
>>>>>> routine before SetVirtualAddressMap()? Otherwise anything could
>>> happen.
>>>>>>
>>>>>> On the Kendryte K210 we don't have SPL. So we will not boot in the
>>>>>> sequence SPL->OpenSBI->U-Boot but OpenSBI->U-Boot. Does this imply
>>> that
>>>>>> we have to implement the hart lottery at the entry point of main
>>> U-Boot
>>>>>> in this case?
>>>>>
>>>>> Isn't the hart lottery already implemented for U-Boot? E.g. around
>>> line
>>>>> 100 of arch/riscv/cpu/start.S.
>>>>
>>>> Thanks for the hint.
>>>>
>>>>>
>>>>> On another note, does Linux support S-Mode NOMMU? I was under the
>>>>> impression that NOMMU implied M-Mode (or the other way around).
>>>>
>>>> Have a look at
>>>>
>>>>
>>> https://linuxplumbersconf.org/event/4/contributions/386/attachments/298/502/RISC-V-NOMMU-Linux-Plumbers-2019.pdf
>>>>
>>>> Best regards
>>>>
>>>> Heinrich
>>