[U-Boot] [PATCH v2 3/3] image: Allow images to indicate they're loadable at any address

[Nicolas Pitre]

On Tue, 8 Nov 2011, Wolfgang Denk wrote:

> In message <alpine.LFD.2.02.1111071840080.3307 at xanadu.home> you wrote:
> >
> > > I understand you are referring here to zImages only. Correct?
> > 
> > Correct.  Anything else is not relocatable.
> > 
> > > Or will raw images (without the preloader) be fully relocatable, too?
> > 
> > No.
> 
> OK. So the situation on ARM is basicly not different from what we
> have on other architectures.
> 
> The uImage format has been designed to handle this situation: we
> normally wrap the raw image (eventually after compressing it) into an
> uImage, and insert the load and entry point addresses that are well
> known at kernel build time.  uImages are not directly executable.
> 
> zImages on ARM include a preloader, that wraps the the raw image
> (eventually after compressing it) and packs it together with some
> code that internally accounts for the fixed load and entry point
> addresses of the kernel which implements the needed relocation and
> uncompression routines to form a new executable, position independent
> format.
> 
> In both cases the _kernel_ image is not position independent. It must
> be loaded to a specific address and started at a specific entry point.
> The exact information where these are is known at built time, and
> somehow encoded in the images (here in the image header, there in the
> preloader code).
> 
> 
> Is this summary correct so far?

No.  Your statement that says "The exact information where these are is 
known at built time, and somehow encoded in the images (here in the 
image header, there in the preloader code)" is false.  None of that 
information is known at build time nor encoded in the preloader as you 
call it.

> > > Why is it so important to load it at specific (different) addresses
> > > when it can be started from any address?
> > 
> > The kernel code can be started from any address.  We want the _code_ to 
> > be that way.
> 
> This is not quite correct.  The _kernel_ code has a fixed address.
> It is the preloader code which can be started from any address.

The preloader is part of the kernel, whether you like it or not, and it 
is not going away any time soon.  That would help the conversation if 
you accepted it as such.

> > However a particular board may or may not load the kernel at any 
> > address.  This is a machine specific restriction, not a kernel 
> > restriction.
> 
> Correct.  Stephen Warren also writes in <4EB87122.3050602 at nvidia.com>:
> "Presumably whoever constructed the environment has the most detailed
> knowledge of the HW's and U-Boot environment's expected memory layout
> for that particular board."   And in <4EB87375.1040100 at nvidia.com>:
> "The only place that has full knowledge of the board's memory layout
> is the U-Boot environment for that board, and hence I assert that the
> U-Boot environment should define where to load the kernel (and initrd
> and FDT), and if U-Boot must copy them, where to copy them to. In
> particular, the creator of the uImage can't possibly pick these
> values and expect them to work everywhere."
> 
> I mostly agree with this.  That's the reason why U-Boot offers to
> handle this based on a better understanding of the low level hardware
> details.  This seems to me to be an easier way that to handle this in
> the Linux kernel environment (in the form of the preloader).

You are mixing up issues.  The bootloader has knowledge of where is the 
best place to load kernel images on a per machine basis.  This is why 
the zImage code was made to accept and execute from any address in order 
to give the bootloader any possible flexibility in that regard.
However it is the kernel's responsibility to deal with its own 
environment and change it at will when we see fit, and therefore this is 
best if the bootloader doesn't take any responsibility here.

> > > Maybe this is a key point.  I simply fail to understand this.
> > 
> > Let me repeat again.  We want one single kernel image binary that ARM 
> > distributions can use for all their target machines.  It is therefore 
> > necessary that uImage be free of any hardcoded load address (absolute or 
> > relative).  If a particular board require a particular load address for 
> > the kernel, this must be encoded in its own u-Boot environment and not 
> > in the distributed uImage.  Failing that, uImage simply cannot be used 
> > as a distribution format for the kernel because any address/offset 
> > enforced by the uImage format is going to be incompatible with the needs 
> > of a particular machine somewhere.
> 
> uImage is _intented_ to carry a raw OS image and provide the boot
> loader the information where to load and start it.
> 
> You do not want to do that, so you should not attempt to assault
> uImages to make it fit your completely different purposes.  If you
> want to use relocatable zImages which have all the needed information
> encoded internally then just do so.  But do not insist that uImage
> does things it was not designed for.

OK, fine.  I don't really care about uImage if it cannot accommodate our 
needs.  I thought that the -1 extension was a pretty agile and 
non-obstrusive way to make u-Boot more flexible and versatile, and not 
only for Linux loading.  Failing that, can we have support for loading 
zImage directly then?  There are precedents in the upstream u-Boot for 
doing that on other architectures already.

> > No.  The requirements on raw images are unchanged.  you can use them if 
> > you wish, but generic ARM distributions can't use that if they want to 
> > target more than one SOC.  Therefore raw images are not interesting by 
> > the use case at hand.
> 
> They are not interesting for you.  But that does not mean that they
> are not interesting in general.  Simon Glass brings up this point,
> too, in <CAPnjgZ1WajMLznSDNDLBys-C3Um2rVvZjOLk3BfSO2M=FRW2Dg at mail.gmail.com>:
> 
> | Can I assume that we have (or can have) a 'make uImage' target or
> | similar in the kernel which can pack together:
> | 
> | - a compressed kernel (not zImage, I mean something that U-Boot can
> | decompress), with a rel_offset of 32KB
> | - a DTB
> | - a ramdisk
> | 
> | and that with Stephen's patch (committed to U-Boot) today, we can, in
> | U-Boot, with a script, load this uImage to somewhere and have U-Boot
> | decompress the kernel and set the bits out nicely in RAM, no matter
> | where that RAM is? The kernel will start at 32KB, and the other bits
> | will be somewhere above that. Then U-Boot can enter the kernel at 32KB
> | and all will be well, yes?
> 
> Why are attempts blocked to provide such support for people who prefer
> this approach?  A "one size fits all" approach like "we have always
> been using zImages, so it must be a perfect solution for everybody" is
> eventually _not_ the optimal solution for all.

I never intended to prevent raw images from being used with uImage.  
This is however not the use case I'm looking after.  That doesn't mean 
that because I'm interested in a different usage model than yours that 
I'm going to 
actively prevent raw Image from being used.  I see obvious limitations 
with it, but if that is what you want then by all means just use that.  
We cannot say as much from the u-Boot side for the use case I'm 
interested in though.

> > What do you not understand in the fact that such a specific address 
> > makes the resulting image not universal?  I'm telling you that I need to 
> > produce a kernel image that doesn't carry with it any machine specific 
> > load address so that same image can be installed unmodified on any 
> > machine.
> 
> And what I'm telling you is that you could probably have that for free
> if you dropped the preloader.

But that is not what I want.

> > Instead, you insist on making that image less useful by attaching to it 
> > some restrictions on its load address at build time rather than applying 
> > those restrictions only at load time and only on machines where that 
> > matters.
> > 
> > For the last time, we don't want any address encoded in the kernel 
> > image for the simple fact that we don't know at build time what machine 
> > the kernel will be used on, and therefore what address to use.  This is 
> 
> But instead of letting the boot loader who has the detailled knowledge
> about which memory is actually present on the target system you build
> a non-trivial preloader and insist that the ultimate way is to wrape
> the kernel with it.

Yes, because the kernel still requires to take total control on its 
execution environment, and we are now doing that at run time instead of 
compile time, which does include techniques like self modifying code 
i.e. runtime patching of the kernel binary, etc.

> > why zImage was made totally relocatable and totally position 
> > independent so it can figure out at run time what address to use.  
> > Figuring out the address to use at "make uImage" time only works for a 
> > kernel that will boot on a single specific machine.
> 
> There is no actual reason this has to be that way.  If you want then
> risk a view over the rim of your plate and look how it's working for
> other architectures.  It's not really new problems we're dealing with
> here.  Kernel images that run on a number of different processors and
> boards are not new, and device tree support has been available for
> years, too.  It's not that there is no preexisting experience with
> such issues.
> 
> On Power Architecture systems, we use uImage format and/or FIT image
> format to wrap raw kernel images (without preloader).  If somebody
> wants, he can attach / include the DT blob to that image, and
> eventually also a ramdisk/initrd image.  This is not mandatory, it is
> all optimal.  And it just works.

I'm coming to the conclusion that your world view is different from mine 
then.  Maybe ARM used to be similar to PPC in the past, but ARM is 
moving on and things are changing.  The requirements are changing.  
u-Boot is trying to hold us in the past and making itself more and more 
an obstacle.  The only thing you could do is to merge the provided 
patches which doesn't affect in any way your own view, but would allow 
for u-Boot not to be an obstacle anymore and everybody would be happy.  
Given those patches are 
rather simple and non intrusive, I'm failing to understand why you are 
putting so much resistance in the name of some philosophical agenda or I 
don't know what other reasons.

> > It is already the case, more or less.  The raw image _must_ be loaded at 
> > TEXT_OFFSET from start of RAM, regardless of where that RAM is.  So if 
> > all you care about is raw image kernels, then having a relative a load 
> > address for uImage makes perfect sense.
> 
> Thanks for this clear statement.  So I will apply the patches to add
> such support to U-Boot.

Good.  I've given you confirmation that you can use the kernel in the 
way you want to use it.  I consider it a valid although suboptimal use 
case, but I'm not denying you that possibility.

> > This is a rigid restriction that a well intended bootloader would not 
> > impose all the time if the user wishes to do otherwise.  You might not 
> > be concerned by use cases where we want images to be executed right 
> > where they've been loaded irrespective of what that might be, but that 
> > doesn't make that any less of a valid use case.  And Stephen's patches 
> > are providing just that.
> 
> No, they aren't.
> 
> I already NAKed these patches, and this discussion has made it clear
> to me that this was a correct decision.  What you want is not uImages.

You are therefore denying me the ability to use the kernel according to 
the use case I care about.  Maybe I should reconsider my willingness to 
let you use raw kernel image then?  Because if you are not willing to 
collaborate for the case I care about, why should I care about yours?


Nicolas