[U-Boot] bug in malloc/calloc and setting of MORECORE_CLEARS

[Kumar Gala]

On Nov 13, 2010, at 10:34 AM, Kumar Gala wrote:

> I've been debugging a "memory corruption" issue and it seems to come down to how we set MORECORE_CLEARS
> 
> from malloc.h:
> 
>  MORECORE                  (default: sbrk)
>     The name of the routine to call to obtain more memory from the system.
>  MORECORE_FAILURE          (default: -1)
>     The value returned upon failure of MORECORE.
>  MORECORE_CLEARS           (default 1)
>     True (1) if the routine mapped to MORECORE zeroes out memory (which
>     holds for sbrk).
> 
> 
> So the issue I'm seeing is that we zero out memory in mem_malloc_init().  Thus our sbrk implementation just does simple record keeping.  However we can get into the following situation:
> 
> a1 = malloc(LARGE_SZ);	/* causes internals to call sbrk */
> ...
> /* use memory at 'a1' */
> ...
> free(a1);	/* causes internals to call malloc_trim, malloc_trim decides to return memory to sbrk, memory is now nonzero */
> 
> a2 = calloc(SIZE);	/* internals assume sbrk memory is zero because of MORECORE_CLEARS setting, however its not */
> 
> ... BAD THINGS HAPPEN ...
> 
> 
> I'm guessing we haven't seen much of this because there aren't that many users of calloc today.  I happen to see it related to NAND code which has kzalloc defined as calloc.


Here's an example test case that fails if execute right after:

        mem_malloc_init (malloc_start, TOTAL_MALLOC_LEN);

        {
                void *p1;
                u8 *p2;
                int i;

                printf("MALLOC TEST\n");
                p1 = malloc(135176);
                printf("P1 = %p\n", p1);
                memset(p1, 0xab, 135176);

                free(p1);
                p2 = calloc(4097, 1);
                printf("P2 = %p %p\n", p2, p2 + 4097);

                for (i = 0; i < 4097; i++) {
                        if (p2[i] != 0)
                                printf("miscompare at byte %d got %x\n", i, p2[i]);
                }

                free(p2);
                printf("END MALLOC TEST\n\n");
        }