[U-Boot] [PATCH v2 3/3] image: Allow images to indicate they're loadable at any address

[Wolfgang Denk]

Dear Nicolas,

may I suggest that you please try to relax for a moment, and try to
look at things from a completely unprejudiced point of view?  We will
come back to your arguments later, promised.


In message <alpine.LFD.2.02.1111071840080.3307 at xanadu.home> you wrote:
>
> > I understand you are referring here to zImages only. Correct?
> 
> Correct.  Anything else is not relocatable.
> 
> > Or will raw images (without the preloader) be fully relocatable, too?
> 
> No.

OK. So the situation on ARM is basicly not different from what we
have on other architectures.

The uImage format has been designed to handle this situation: we
normally wrap the raw image (eventually after compressing it) into an
uImage, and insert the load and entry point addresses that are well
known at kernel build time.  uImages are not directly executable.

zImages on ARM include a preloader, that wraps the the raw image
(eventually after compressing it) and packs it together with some
code that internally accounts for the fixed load and entry point
addresses of the kernel which implements the needed relocation and
uncompression routines to form a new executable, position independent
format.

In both cases the _kernel_ image is not position independent. It must
be loaded to a specific address and started at a specific entry point.
The exact information where these are is known at built time, and
somehow encoded in the images (here in the image header, there in the
preloader code).


Is this summary correct so far?


> > Why is it so important to load it at specific (different) addresses
> > when it can be started from any address?
> 
> The kernel code can be started from any address.  We want the _code_ to 
> be that way.

This is not quite correct.  The _kernel_ code has a fixed address.
It is the preloader code which can be started from any address.


> However a particular board may or may not load the kernel at any 
> address.  This is a machine specific restriction, not a kernel 
> restriction.

Correct.  Stephen Warren also writes in <4EB87122.3050602 at nvidia.com>:
"Presumably whoever constructed the environment has the most detailed
knowledge of the HW's and U-Boot environment's expected memory layout
for that particular board."   And in <4EB87375.1040100 at nvidia.com>:
"The only place that has full knowledge of the board's memory layout
is the U-Boot environment for that board, and hence I assert that the
U-Boot environment should define where to load the kernel (and initrd
and FDT), and if U-Boot must copy them, where to copy them to. In
particular, the creator of the uImage can't possibly pick these
values and expect them to work everywhere."

I mostly agree with this.  That's the reason why U-Boot offers to
handle this based on a better understanding of the low level hardware
details.  This seems to me to be an easier way that to handle this in
the Linux kernel environment (in the form of the preloader).


> > Maybe this is a key point.  I simply fail to understand this.
> 
> Let me repeat again.  We want one single kernel image binary that ARM 
> distributions can use for all their target machines.  It is therefore 
> necessary that uImage be free of any hardcoded load address (absolute or 
> relative).  If a particular board require a particular load address for 
> the kernel, this must be encoded in its own u-Boot environment and not 
> in the distributed uImage.  Failing that, uImage simply cannot be used 
> as a distribution format for the kernel because any address/offset 
> enforced by the uImage format is going to be incompatible with the needs 
> of a particular machine somewhere.

uImage is _intented_ to carry a raw OS image and provide the boot
loader the information where to load and start it.

You do not want to do that, so you should not attempt to assault
uImages to make it fit your completely different purposes.  If you
want to use relocatable zImages which have all the needed information
encoded internally then just do so.  But do not insist that uImage
does things it was not designed for.

> No.  The requirements on raw images are unchanged.  you can use them if 
> you wish, but generic ARM distributions can't use that if they want to 
> target more than one SOC.  Therefore raw images are not interesting by 
> the use case at hand.

They are not interesting for you.  But that does not mean that they
are not interesting in general.  Simon Glass brings up this point,
too, in <CAPnjgZ1WajMLznSDNDLBys-C3Um2rVvZjOLk3BfSO2M=FRW2Dg at mail.gmail.com>:

| Can I assume that we have (or can have) a 'make uImage' target or
| similar in the kernel which can pack together:
| 
| - a compressed kernel (not zImage, I mean something that U-Boot can
| decompress), with a rel_offset of 32KB
| - a DTB
| - a ramdisk
| 
| and that with Stephen's patch (committed to U-Boot) today, we can, in
| U-Boot, with a script, load this uImage to somewhere and have U-Boot
| decompress the kernel and set the bits out nicely in RAM, no matter
| where that RAM is? The kernel will start at 32KB, and the other bits
| will be somewhere above that. Then U-Boot can enter the kernel at 32KB
| and all will be well, yes?

Why are attempts blocked to provide such support for people who prefer
this approach?  A "one size fits all" approach like "we have always
been using zImages, so it must be a perfect solution for everybody" is
eventually _not_ the optimal solution for all.


> What do you not understand in the fact that such a specific address 
> makes the resulting image not universal?  I'm telling you that I need to 
> produce a kernel image that doesn't carry with it any machine specific 
> load address so that same image can be installed unmodified on any 
> machine.

And what I'm telling you is that you could probably have that for free
if you dropped the preloader.

> Instead, you insist on making that image less useful by attaching to it 
> some restrictions on its load address at build time rather than applying 
> those restrictions only at load time and only on machines where that 
> matters.
> 
> For the last time, we don't want any address encoded in the kernel 
> image for the simple fact that we don't know at build time what machine 
> the kernel will be used on, and therefore what address to use.  This is 

But instead of letting the boot loader who has the detailled knowledge
about which memory is actually present on the target system you build
a non-trivial preloader and insist that the ultimate way is to wrape
the kernel with it.


> why zImage was made totally relocatable and totally position 
> independent so it can figure out at run time what address to use.  
> Figuring out the address to use at "make uImage" time only works for a 
> kernel that will boot on a single specific machine.

There is no actual reason this has to be that way.  If you want then
risk a view over the rim of your plate and look how it's working for
other architectures.  It's not really new problems we're dealing with
here.  Kernel images that run on a number of different processors and
boards are not new, and device tree support has been available for
years, too.  It's not that there is no preexisting experience with
such issues.

On Power Architecture systems, we use uImage format and/or FIT image
format to wrap raw kernel images (without preloader).  If somebody
wants, he can attach / include the DT blob to that image, and
eventually also a ramdisk/initrd image.  This is not mandatory, it is
all optimal.  And it just works.



> > Please explain _why_ you consider it a problem.  Describe use cases
> > where it doesn't work.
> 
> That has been explained a few times already.
> 
> > > specific information must live with the boot loader (ideally as a 
> > > script) and not embodied into an image that could otherwise be totally 
> > > generic.
> > 
> > Must it?  I don't see the need.  I don't even see the benefit.
> 
> Why do I care having this conversation with you then?  Please tell me.

I can ask you the same.  I ask you for a real use case example, and
all you reply is "That has been explained a few times already."
Without firther reference.

At least I was trying to understand you, and listening to your
arguments.  I cannot say that I feel you did so, too.


> It is already the case, more or less.  The raw image _must_ be loaded at 
> TEXT_OFFSET from start of RAM, regardless of where that RAM is.  So if 
> all you care about is raw image kernels, then having a relative a load 
> address for uImage makes perfect sense.

Thanks for this clear statement.  So I will apply the patches to add
such support to U-Boot.

> The role of the zImage code is to figure out at run time about its 
> position in RAM, deduce where the RAM starts, relocate itself if it is 
> in the way of the decompressed kernel, and decompress the actual kernel 
> in its final location (RAM start + TEXT_OFFSET) without having to 
> relocate the bigger decompressed data.  Unless it is in the way of the 
> decompressed kernel, this code can be executed in place and doesn't need 
> to be relocated to any particular address.

In U-Boot, this is actually not needed, as U-Boot offers to do exactly
that. As an additional benefit, you don't have to download and store
the wrapper code, which results in smaller and thus even faster
booting images.

> But u-Boot will then ignore the wish of the user and relocate the image 
> content to 32KB after start of RAM, causing one copy.  Now zImage knows 
> it must not overwrite itself with decompressed data, so it will move 
> itself away i.e. a second copy.  Bot those copies are totally useless 
> and could be avoided if only u-Boot would stop ignoring the user's wish 
> to load and run zImage from the address indicated by the provided 
> argument to the load command.

...or they could be avoided if you did not use the zImage wrapper in
the first place.

> This is a rigid restriction that a well intended bootloader would not 
> impose all the time if the user wishes to do otherwise.  You might not 
> be concerned by use cases where we want images to be executed right 
> where they've been loaded irrespective of what that might be, but that 
> doesn't make that any less of a valid use case.  And Stephen's patches 
> are providing just that.

No, they aren't.

I already NAKed these patches, and this discussion has made it clear
to me that this was a correct decision.  What you want is not uImages.


I suggest the following:

- I will apply Stephen's previous patches to support relative images
  as they are useful for people who want to use "proper" uImages
  (containing a raw kernel image as payload) on different boards /
  architectures.

- If you want to boot zImages (with the kernel wrapper included and
  thus fully relocatable), then please feel free and submit patches to
  add support for booting zImage format in U-Boot.  Then you get what
  you want, and you can use zImages directly, without the 64 byte
  uImage header that is only hindering for your usage mode.

- It would be appreciated if we could get support for real uImages
  (wrapping a raw kernel image) into Linux, then.

This way those who want to use zImages can do so, and those who prefer
a different approach can have their ways, too.