OpenBSI and U-Boot

[Atish Patra]

On Mon, Aug 10, 2020 at 10:30 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
>
> On 8/11/20 3:55 AM, Rick Chen wrote:
> > Hi Heinrich
> >
> >> Am 9. August 2020 22:08:23 MESZ schrieb Atish Patra <atishp at atishpatra.org>:
> >>> On Sat, Aug 8, 2020 at 9:17 AM Heinrich Schuchardt <xypron.glpk at gmx.de>
> >>> wrote:
> >>>>
> >>>> On 8/8/20 5:32 PM, Sean Anderson wrote:
> >>>>> On 8/8/20 10:59 AM, Heinrich Schuchardt wrote:
> >>>>>> Hello Anup,
> >>>>>>
> >>>>>> I have looking at you OpenSBI code firmware/payloads/test_head.S.
> >>> Here
> >>>>>
> >>>>> I think the real start is in firmware/fw_base.S. From there,
> >>> secondary
> >>>>> harts loop first in _wait_relocate_copy_done, and then in
> >>>>> _wait_for_boot_hart, and then they execute the next stage via
> >>>>> _start_warm and sbi_init.
> >>>>>
> >>>>>> like in U-Boot's common/spl/spl_opensbi.c you put all but one hart
> >>> in to
> >>>>>> an enless loop (hang).
> >>>>>
> >>>>> As far as I can tell, U-Boot has all harts execute the next stage
> >>> when
> >>>>> SMP is enabled. smp_call_function has all harts execute that
> >>> function.
> >>>>
> >>>> U-Boot can only run on one hart. Are the other harts trapped in
> >>>> secondary_hart_loop()? How do we ensure that an UEFI payload does not
> >>>> overwrite this memory location?
> >>>>
> >>>
> >>> If you are booting Linux, U-Boot runs in S-mode and SMP is disabled.
> >>
> >> Would that also hold true on the Kendryte K210? For all what can see the secondary hart enters U-Boot and is only restrained by WFI and otherwise kept in an endless loop.
> >>
> >> I am wondering if that endless loop needs to be marked as reserved memory for Linux.
> >
> > Currently if U-Boot run in S-Mode, SMP is disable, there will exist
> > OpenSBI version compatible issue.
> > You shall use OpenSBI v0.7 with HSM, thus it will trap the other harts
> > in OpenSBI and only main hart will jump to kernel from U-Boot proper.
> >
> > Thanks,
> > Rick
> >
>
> HSM is described here:
> https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc#hart-state-management-extension-extension-id-0x48534d-hsm
>
> Currently the Kendryte K210 has no SPL. So OpenSBI would be running
> before U-Boot.
>
> Will OpenSBI v0.7 by itself stop the other harts or do we need to call
> sbi_hart_stop() in U-Boot? Unfortunately opensbi/README.md and
> riscv-sbi.adoc both leave this open.
>

OpenSBI will put all non-booting harts into wfi in sbi_hsm_wait
https://github.com/riscv/opensbi/blob/master/lib/sbi/sbi_hsm.c#L119

The S-mode OS is required to call sbi_hsm_start for all non-booting harts
so that they enter S-mode.

Are you planning to add HSM extension just for Kendryte ?

> Best regards
>
> Heinrich
>
> >>
> >>> All memory regions containing OpenSBI code are reserved. Thus, U-Boot
> >>> won't touch that.
> >>> U-Boot also marks the run time services memory region as reserved as
> >>> well.
> >>> Thus, Linux kernel won't touch any of those memory regions either.
> >>>
> >>>> spl_secondary_hart_stack_gd_setup() can jump to hang() if the call to
> >>>> secondary_hart_relocate() fails.
> >>>>
> >>>>>
> >>>>>>
> >>>>>> When Linux boots via UEFI it will wake up the extra harts after
> >>>>>> ExitBootServices(). So I assume we should define function hang()
> >>> in
> >>>>>> lib/hang.c as __efi_runtime to avoid seeing it overwritten by the
> >>> EFI
> >>>>>> payload.
> >>>>>>
> >>>>>> @Ard:
> >>>>>> Does Linux take care of the hanging harts and redirect them to its
> >>> own
> >>>>>> routine before SetVirtualAddressMap()? Otherwise anything could
> >>> happen.
> >>>>>>
> >>>>>> On the Kendryte K210 we don't have SPL. So we will not boot in the
> >>>>>> sequence SPL->OpenSBI->U-Boot but OpenSBI->U-Boot. Does this imply
> >>> that
> >>>>>> we have to implement the hart lottery at the entry point of main
> >>> U-Boot
> >>>>>> in this case?
> >>>>>
> >>>>> Isn't the hart lottery already implemented for U-Boot? E.g. around
> >>> line
> >>>>> 100 of arch/riscv/cpu/start.S.
> >>>>
> >>>> Thanks for the hint.
> >>>>
> >>>>>
> >>>>> On another note, does Linux support S-Mode NOMMU? I was under the
> >>>>> impression that NOMMU implied M-Mode (or the other way around).
> >>>>
> >>>> Have a look at
> >>>>
> >>>>
> >>> https://linuxplumbersconf.org/event/4/contributions/386/attachments/298/502/RISC-V-NOMMU-Linux-Plumbers-2019.pdf
> >>>>
> >>>> Best regards
> >>>>
> >>>> Heinrich
> >>
>


-- 
Regards,
Atish