OpenBSI and U-Boot

[Rick Chen]

Hi Atish

> On Mon, Aug 10, 2020 at 10:30 PM Heinrich Schuchardt <xypron.glpk at gmx.de> wrote:
> >
> > On 8/11/20 3:55 AM, Rick Chen wrote:
> > > Hi Heinrich
> > >
> > >> Am 9. August 2020 22:08:23 MESZ schrieb Atish Patra <atishp at atishpatra.org>:
> > >>> On Sat, Aug 8, 2020 at 9:17 AM Heinrich Schuchardt <xypron.glpk at gmx.de>
> > >>> wrote:
> > >>>>
> > >>>> On 8/8/20 5:32 PM, Sean Anderson wrote:
> > >>>>> On 8/8/20 10:59 AM, Heinrich Schuchardt wrote:
> > >>>>>> Hello Anup,
> > >>>>>>
> > >>>>>> I have looking at you OpenSBI code firmware/payloads/test_head.S.
> > >>> Here
> > >>>>>
> > >>>>> I think the real start is in firmware/fw_base.S. From there,
> > >>> secondary
> > >>>>> harts loop first in _wait_relocate_copy_done, and then in
> > >>>>> _wait_for_boot_hart, and then they execute the next stage via
> > >>>>> _start_warm and sbi_init.
> > >>>>>
> > >>>>>> like in U-Boot's common/spl/spl_opensbi.c you put all but one hart
> > >>> in to
> > >>>>>> an enless loop (hang).
> > >>>>>
> > >>>>> As far as I can tell, U-Boot has all harts execute the next stage
> > >>> when
> > >>>>> SMP is enabled. smp_call_function has all harts execute that
> > >>> function.
> > >>>>
> > >>>> U-Boot can only run on one hart. Are the other harts trapped in
> > >>>> secondary_hart_loop()? How do we ensure that an UEFI payload does not
> > >>>> overwrite this memory location?
> > >>>>
> > >>>
> > >>> If you are booting Linux, U-Boot runs in S-mode and SMP is disabled.
> > >>
> > >> Would that also hold true on the Kendryte K210? For all what can see the secondary hart enters U-Boot and is only restrained by WFI and otherwise kept in an endless loop.
> > >>
> > >> I am wondering if that endless loop needs to be marked as reserved memory for Linux.
> > >
> > > Currently if U-Boot run in S-Mode, SMP is disable, there will exist
> > > OpenSBI version compatible issue.
> > > You shall use OpenSBI v0.7 with HSM, thus it will trap the other harts
> > > in OpenSBI and only main hart will jump to kernel from U-Boot proper.
> > >
> > > Thanks,
> > > Rick
> > >
> >
> > HSM is described here:
> > https://github.com/riscv/riscv-sbi-doc/blob/master/riscv-sbi.adoc#hart-state-management-extension-extension-id-0x48534d-hsm
> >
> > Currently the Kendryte K210 has no SPL. So OpenSBI would be running
> > before U-Boot.
> >
> > Will OpenSBI v0.7 by itself stop the other harts or do we need to call
> > sbi_hart_stop() in U-Boot? Unfortunately opensbi/README.md and
> > riscv-sbi.adoc both leave this open.
> >
>
> OpenSBI will put all non-booting harts into wfi in sbi_hsm_wait
> https://github.com/riscv/opensbi/blob/master/lib/sbi/sbi_hsm.c#L119
>
> The S-mode OS is required to call sbi_hsm_start for all non-booting harts
> so that they enter S-mode.
>
> Are you planning to add HSM extension just for Kendryte ?

I have finished this part of jobs internally but still need some time
to do code cleanup and prepare patchs.

Thanks,
Rick

>
> > Best regards
> >
> > Heinrich
> >
> > >>
> > >>> All memory regions containing OpenSBI code are reserved. Thus, U-Boot
> > >>> won't touch that.
> > >>> U-Boot also marks the run time services memory region as reserved as
> > >>> well.
> > >>> Thus, Linux kernel won't touch any of those memory regions either.
> > >>>
> > >>>> spl_secondary_hart_stack_gd_setup() can jump to hang() if the call to
> > >>>> secondary_hart_relocate() fails.
> > >>>>
> > >>>>>
> > >>>>>>
> > >>>>>> When Linux boots via UEFI it will wake up the extra harts after
> > >>>>>> ExitBootServices(). So I assume we should define function hang()
> > >>> in
> > >>>>>> lib/hang.c as __efi_runtime to avoid seeing it overwritten by the
> > >>> EFI
> > >>>>>> payload.
> > >>>>>>
> > >>>>>> @Ard:
> > >>>>>> Does Linux take care of the hanging harts and redirect them to its
> > >>> own
> > >>>>>> routine before SetVirtualAddressMap()? Otherwise anything could
> > >>> happen.
> > >>>>>>
> > >>>>>> On the Kendryte K210 we don't have SPL. So we will not boot in the
> > >>>>>> sequence SPL->OpenSBI->U-Boot but OpenSBI->U-Boot. Does this imply
> > >>> that
> > >>>>>> we have to implement the hart lottery at the entry point of main
> > >>> U-Boot
> > >>>>>> in this case?
> > >>>>>
> > >>>>> Isn't the hart lottery already implemented for U-Boot? E.g. around
> > >>> line
> > >>>>> 100 of arch/riscv/cpu/start.S.
> > >>>>
> > >>>> Thanks for the hint.
> > >>>>
> > >>>>>
> > >>>>> On another note, does Linux support S-Mode NOMMU? I was under the
> > >>>>> impression that NOMMU implied M-Mode (or the other way around).
> > >>>>
> > >>>> Have a look at
> > >>>>
> > >>>>
> > >>> https://linuxplumbersconf.org/event/4/contributions/386/attachments/298/502/RISC-V-NOMMU-Linux-Plumbers-2019.pdf
> > >>>>
> > >>>> Best regards
> > >>>>
> > >>>> Heinrich
> > >>
> >
>
>
> --
> Regards,
> Atish