[U-Boot] early_malloc outline

[Wolfgang Denk]

Dear Tomas Hlavacek,

In message <CAEB7QLD3kSzX8r9q-gox8aww6wOiHKqd-AVCU_Ux7vY7V7TbSA at mail.gmail.com> you wrote:
> 
> > But "relocation" means that you have to add the address difference
> > (aka relocation offset) to _all_ pointers pointing into this area.
> > And there is no way to keep track of _all_ such pointers.
>
> Sure there is no way to do relocation of all data in the early_malloc
> arena generically (= for all data present regardless of their origin
> and use). And neither I want to do so.

OK, we agree on this.

> It comes down to the reason for implementing early_malloc: The reason
> is to facilitate minimalistic heap allocation for driver model tree in
> the early init phase. It is essential to start building the tree in
> the early phase (= in board_init_f) in order to have drivers bound to
> their cores before first use and to be able to call particular drivers
> through DM driver cores from the very beginning. (Driver cores are
> structs that are DM abstractions of particular drivers; this structs
> are held in DM tree.) It is also needed to build DM tree in runtime,
> otherwise the DM tree generated in compile-time would be board
> specific and it would disallow prospective unification of resulting
> ROMs for more than one board.

Agreed, too.

If so, my argument goes, you must not use the standard malloc() /
calloc() / free() API for the early_malloc implementation.  If you do,
there my be any code that is not related to DM, but which happens to
be used early, which suddenly is allocating memory from the DM arena,
without you being able to track any of the pointers potentialy
pointing into this area, which in turn means as soon as you relocate
it the pointers will break.

> presented here in near future. I would rather have an input for DM
> design from this discussion, than discussion about
> output/decisions/suggestions from DM project.

I intend to provide input for the DM design:  you provide a
specialized implementaion, so make sure to use also a specialized
interface, to avoid that other code unintentionally uses your
functions.

> Regarding relocation I want to implement it only for DM tree assuming
> that I can retain the root pointer in GD and I can traverse the tree
> using computed offsets. But there are obvious problems when the
> early_malloc is used for another purpose than DM. Another users (which

Correct. So such thing must not happen.

> is what Graeme suggested to take into account) might want to use the
> early_heap during early init but he might want not to retain data and
> therefore copying that data to RAM would be waste of CPU time and RAM.

I agree with Graeme that it would be nice to have an early malloc that
automagically preserves the alocations until the full U-Boot is
running, but I cannot see how such a thing could be implemented.

Especially with the new SPL technology we allow users to use all kinds
of code very early - for example there are systems which use malloc()
and file system code before relocation.  Such usage would blow your
implementation.

I want to have the DM code to be as simple and robust as possible, so
I recommend against any fancy design which attempts to solve all
problems of this world at once (and fails). Instead, use a simple and
robust design that is tailored to just the DM purposes, and not used
by anything else.

> > I am convinced that you _cannot_ reliably relocate the malloc arena if
> > you use the standard malloc//calloc/free interface for early
> > allocation.
>
> Forgive me my ignorance but why?

Because you cannot track which pointers point into it. They can be
distributed all over the code.  Any function anybody calls might use
malloc() internally, and keep static pointers to allocated data.

> Assuming that there is a DM driver core with certain dm_core_init()
> function that calls malloc() and then registers the DM driver core
> into the DM tree, it is still the same function for early and late

Face it: there will be, and actually is already (on some systems)
other code that uses malloc(), and that doesn't (and should not have
to) know anything about the specific requirements or implementationof
the DM early allocator.

> (Sure I can create a "private" wrapper, for example dm_malloc(),
> dm_calloc(), dm_free(),... strictly for DM needs.)

You wil need such a separate interface, but it will definitely not be
any kind of wrapper.  dm_malloc() and malloc() will have to be kept
strictly separated in the general case.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
When Granny says a task is impossible, she means it is impossible for
anyone but herself.