Fwd: dlmalloc hanging

[Heinrich Schuchardt]

Am 27. Mai 2022 09:52:04 MESZ schrieb Oleksii Kurochko <oleksii.kurochko at gmail.com>:
>Hello Heinrich,
>
>get_maintainer.pl script told me that you are a maintainer of dlmalloc.c

Hello Oleksii,

I am not the maintainer of dlmalloc.c but only a contributor.

>so if you don't mind I would like to ask you for advice about
>how could debugging of dlmalloc allocation be done?
>
>I faced the issue that hanging happens during the freeing of
>resources [
>https://elixir.bootlin.com/u-boot/latest/source/arch/arm/lib/bootm.c#L77]
>before U-boot finally will transfer control to Linux Kernel.
>I did a little investigation and found out that hanging
>happens here [
>https://elixir.bootlin.com/u-boot/latest/source/common/dlmalloc.c#L940]
>and it happens because FD->fd points to itself so a cycle occurs.

Does the problem exist in mainline U-Boot?
How can it be reproduced in mainline U-Boot?

If it is a Bootlin specific problem, please, consider contacting that company.

Possible reasons for running into problems in dlmalloc.c are double frees and out of bound writing to memory.

Best regards

Heinrich 

>
>Should be a check added here which will check that FD->fd does not point to
>itself or
>does a guarantee exist that the cycle can't be at all?
>
>Also, I tried to do "#define DEBUG" at the top of dlmalloc.c
>and I started to be asserted here:
>https://elixir.bootlin.com/u-boot/latest/source/common/dlmalloc.c#L842
>during the mentioned transfer from U-boot to kernel.
>
>I also tried to understand why it is in the used state and I couldn't find
>where it is actually switched to the unused state. At least clear_inuse()
>macros aren't used elsewhere in file dlmalloc.c.
>
>Interesting behavior has been obtained. It looks like that for
>the number of lines in CONFIG_EXTRA_ENV_SETTINGS there is no
>hanging at all during mentioned above transfer. So I can assume that
>it might be allocated a buffer or pointer to its buffer is overwritten
>because of the memory alignment of an array that uses
>CONFIG_EXTRA_ENV_SETTINGS.
>
>Could you please give me some advice on how such behavior
>could be debugged?
>
>Thanks in advance.
>
>Best regards,
>
> Oleksii