[U-Boot] early_malloc outline

[Graeme Russ]

Hi Tomas & Wolfgang,

On Thu, Aug 9, 2012 at 5:32 AM, Wolfgang Denk <wd at denx.de> wrote:
> Dear Tomas Hlavacek,
>
> In message <CAEB7QLD3kSzX8r9q-gox8aww6wOiHKqd-AVCU_Ux7vY7V7TbSA at mail.gmail.com> you wrote:

[snip]

> If so, my argument goes, you must not use the standard malloc() /
> calloc() / free() API for the early_malloc implementation.  If you do,
> there my be any code that is not related to DM, but which happens to
> be used early, which suddenly is allocating memory from the DM arena,
> without you being able to track any of the pointers potentialy
> pointing into this area, which in turn means as soon as you relocate
> it the pointers will break.

I pointed out that those pointers can only by in either GD or the early
malloc heap. But either way, there is no generic way to adjust them. Any
code that uses the early malloc heap and wants to retain that data for
use after relocation will need to implement it's own relocation code.

Having a specific early_malloc() has the advantage of being explicit. The
problem is, the initialisation for some drivers may not necessarily be
globally 'early' or 'late' depending on the board. So for some driver:

int foo_init(struct some_dm_struct *blah)
{
  struct foo_struct *my_data = malloc(sizeof(foo_struct));
  .
  .
  .
}

runs into trouble if the foo driver is neaded early be one board, but not
by another.

> I agree with Graeme that it would be nice to have an early malloc that
> automagically preserves the alocations until the full U-Boot is
> running, but I cannot see how such a thing could be implemented.

That was never how I thought about it. Anything that uses the early heap
needs to 'help' the relocation process. My thought was to make the glue
logic for the helper as seamless as possible.

>> > I am convinced that you _cannot_ reliably relocate the malloc arena if
>> > you use the standard malloc//calloc/free interface for early
>> > allocation.
>>
>> Forgive me my ignorance but why?
>
> Because you cannot track which pointers point into it. They can be

Yes

> distributed all over the code.  Any function anybody calls might use
> malloc() internally, and keep static pointers to allocated data.

No - There can be no static pointers before relocation - All pointers must
be in either GD or in structures allocated on the early malloc heap

>> Assuming that there is a DM driver core with certain dm_core_init()
>> function that calls malloc() and then registers the DM driver core
>> into the DM tree, it is still the same function for early and late
>
> Face it: there will be, and actually is already (on some systems)
> other code that uses malloc(), and that doesn't (and should not have
> to) know anything about the specific requirements or implementationof
> the DM early allocator.
>
>> (Sure I can create a "private" wrapper, for example dm_malloc(),
>> dm_calloc(), dm_free(),... strictly for DM needs.)
>
> You wil need such a separate interface, but it will definitely not be
> any kind of wrapper.  dm_malloc() and malloc() will have to be kept
> strictly separated in the general case.

OK, this got me to thinking about the 'relocation' function in the driver
structure and how we can make the early heap more generic. My thoughts tie
into the DM tree structure being discussed in another thread...

Instead of putting a 'relocation function' pointer in the driver core
structure, we don't we (as I have suggested before) explicitly register the
relocation function. e.g.

int foo_init(struct some_dm_struct *blah)
{
  register_ealry_malloc_relocation(foo_relocation_function);

  struct foo_struct *my_data = malloc(sizeof(foo_struct));
  .
  .
  .
}

(Of course, we can still have the pointer in the driver core struct and do
this in the core code, but it may be a bit wasteful - I don't know)

int foo_relocation_function()
{
  ... do relocation of foo data structures ...
}

Now register_ealry_malloc_relocation() is in the early malloc code

int register_ealry_malloc_relocation(function *blah)
{
  if (gd->flags | relocated)
    return 0;

  ... Add blah to the relocation function list ...
}

Then in relocate_code() we call relocate_early_malloc() which simply walks
the relocation function list and calls each one. Each function is
responsible for allocating new memory form the final heap and copying the
data from the early heap to the newly allocated memory

This way, anyone (not just drivers) can take advantage of the early heap.
And if a user of early heap does not care about the data being available
post relocation, they just don't bother implementing and registering a
relocation function.

So to summarise:
  - Early malloc() needs to be a malloc()
  - Any code using early malloc needs to be aware of this and provide a
    relocation function
  - Why has this taken 6+ months to sort out?

Regards,

Graeme