[PATCH] sunxi: board: provide CPU idle states to loaded OS

[Andre Przywara]

On Wed, 6 Sep 2023 23:53:51 +0300
Andrey Skvortsov <andrej.skvortzov at gmail.com> wrote:

Hi Andrey,

we seem to have a very different approach to the whole booting process,
which makes this conversation quite involved.

I understand how things have been traditionally handled in the past, in
the embedded world (DTBs bundled with the kernel, distros installing
boot firmware), but we should really move on from this ill-fated
approach. For a while now we try hard to keep the DTs compatible across
several kernel version, making only compatible changes. In particular I
have been deliberately keeping compatibility with pre-v5.13 kernels in
the U-Boot copy of the Allwinner DTs, so you can always take the latest
version and boot that with a wide range of kernels.

> On 23-09-06 01:12, Andre Przywara wrote:
> > On Tue, 5 Sep 2023 11:37:31 +0300
> > Andrey Skvortsov <andrej.skvortzov at gmail.com> wrote:
> > 
> > Hi,
> >   
> > > On 23-09-05 09:27, Andre Przywara wrote:  
> > > > On Mon,  4 Sep 2023 23:54:30 +0300
> > > > Andrey Skvortsov <andrej.skvortzov at gmail.com> wrote:
> > > > 
> > > > Hi Andrey,
> > > >     
> > > > > When using SCPI as the PSCI backend, firmware can wake up the CPUs and
> > > > > cluster from sleep, so CPU idle states are available for loaded OS to
> > > > > use. TF-A modifies DTB to advertise available CPU idle states, when
> > > > > SCPI is detected. This change copies nodes added by TF-A to any new
> > > > > dtb that is used for loaded OS.    
> > > > 
> > > > Why do you need that, exactly? Why not just use $fdtcontroladdr for the
> > > > kernel? We now keep the U-Boot copy of the .dts files in sync with the
> > > > kernel. If you need to modify the DT in U-Boot, for instance by applying
> > > > overlays, you can copy that DTB into a better suitable location first:    
> > > > => fdt move $fdtcontroladdr $fdt_addr_r    
> > > > 
> > > > In any case, there shall be only one DT, that one in the U-Boot image. Why
> > > > do you need to load another one for the kernel?    
> > > 
> > > extlinux is used by distributions (sometimes with device-specific changes especially  
> > 
> > What distros are that? I guess some special ones, targeting embedded
> > devices, like the Pinephone?  
> 
> It's more likely universal operating system. In my particular case
> this is Mobian (several device-specific packages on top of
> Debian). It's very-very close to Debian with a goal to be
> pure Debian in the near future.
> 
> Rhino Linux is using extlinux as well and will benefit for this change
> as well.

We seem to have a very different understanding of "universal operating
systems" ;-)
You seem to talk about specific distros for embedded devices, whereas I
think of Ubuntu/Debian/Fedora/Arch/you-name-it.

And I still don't understand what a particular distribution has to do
with that: this is purely a question of boot firmware and who provides
the DTB.

> > And who is generating extlinux.conf then? Is that some distro specific
> > scripting, similar to how grub is configured?  
> 
> extlinux.conf is automatically generated by standard Debian
> u-boot-menu package. See [1] and [2]. 

Ah, thanks, I was looking for the extlinux package, which only seems to
exist for x86.

> > Honest questions, I am not a user of extlinux, I mostly use UEFI
> > booting, or type U-Boot commands directly for experiments, or use
> > boot.scr, as a quick-and-dirty hack.  
> 
> 
> > > for platforms not fully supported by mainline yet),  
> > 
> > Do you need any changes to the DT? Do you need to apply overlays?
> > If you run on a non-mainlined platform, you could still put your DT
> > into the U-Boot tree, then you wouldn't need to load another DTB, which
> > also simplifies the deployment on the kernel/distro side.  
> 
> Currently Mobian linux kernel for sunxi-devices contains 36 extra patches with DT

That does not sound good. Why are those patches not upstream?

> modifications (add/remove nodes, modify existing properties). One of
> them unconditionally adds cpuidle states to DT, that I'm trying to fix
> upstream here with the proposed change.

That's very honourable, but not every patch that works(TM) is the right
approach to take upstream.

> DT overlays are not used.
> 
> Honestly I don't think that using dtb from u-boot will simplify
> distribution deployment and maintenance. IMHO, it will make things
> more complex. Currently on my device there are records in extlinux.conf
> for 5.9, 5.10, 5.15, several 6.1 kernels, that I'm working on. All of
> them have slightly (or sometimes more) different device-trees.

And why is that, exactly? The DT describes the *device*, so why do you
have different DTs for different kernels? Sure, they evolve, devices
get added and we will have fixes, but conceptually there should be one
best DT for each device, for every kernel, and that could be the one
in the U-Boot tree - or the one in the latest kernel *tree*.

And why do you have different kernels to begin with? Are those various
vendor kernels, for different SoCs, each with their own pile of
downstream patches?

> Having
> kernel and dtb deployed together makes life so much easier.

Easier in the long term, because you can make quick hacks, but it's all
getting a mess over time.

> Mobian's dtb can't be put into u-boot.

Why not? It's a matter of copying whatever .dts files you have into
U-Boot's arch/arm/dts directory and making sure the Makefile covers
every file. Ignoring the question what "Mobian's DTB" really is.

> Mobian doesn't provide
> device-specific bootloader (u-boot) and there is on-going work on
> making device-independent universal images.

With "universal image" you mean a single rootfs, and a single kernel for
every device? Of course, this is how Linux works everywhere, and the
arm64 kernel was a single-image kernel from day one (same as x86). And
taking the DTs out there and into firmware is just even more
consequential.

> As many other mobile Linux distributions Mobian relies on
> Tow-Boot. That is used as a cross-distribution firmware (u-boot/tf-a/crust). [3]

I don't know much about tow-boot, but it seems just like a U-Boot
*distribution*, so I don't see why you can't use upstream U-Boot
techniques like sharing U-Boot's control DTB with the kernel
(by using $fdtcontroladdr).

> > > then U-Boot loads DT defined in
> > > extlinux.conf file. u-boot scripts are not used in case of extlinux at all.  
> > 
> > That's fine, you don't need any U-Boot scripts for this to work. If
> > there is no "fdt" or "fdtdir" label in extlinux.conf, then the U-Boot
> > PXE code will eventually fall back to $fdtcontroladdr - I just tested
> > that.
> > So could you make that work for you? I guess all you need to change is
> > to remove any fdtdir label from extlinux.conf?
> > 
> > Cheers,
> > Andre  
> 
> Regardless proposed change. Changes to dtb nodes already copied in
> u-boot on the fly in boot/image-fdt.c:image_setup_libfdt. For example,
> created optee nodes are copied there.

Yes, that is an example, just not a particularly good one. This seems
to originate from 2019, again building on the idea that you need to load
a DTB for the kernel from somewhere. I don't like to proliferate those
ideas anymore.

> I've just put platform specific
> changes into platform specific ft_board_setup, that was made
> apparently exactly for that.

Not really, that's for platform specific runtime *adjustments*, like
adding a unique MAC address, or adding simplefb DT nodes.

Cheers,
Andre

> 1. https://packages.debian.org/trixie/u-boot-menu
> 2. https://salsa.debian.org/debian/u-boot-menu
> 3. https://tow-boot.org/
>