[PATCH 3/4] dlmalloc: Fix integer overflow in sbrk()

[Richard Weinberger]

Make sure that the new break is within mem_malloc_start
and mem_malloc_end before making progress.
ulong new = old + increment; can overflow for extremely large
increment values and memset() can get wrongly called.

Signed-off-by: Richard Weinberger <richard at nod.at>
---
 common/dlmalloc.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/common/dlmalloc.c b/common/dlmalloc.c
index 44b06e38b2..c8d1da1cb1 100644
--- a/common/dlmalloc.c
+++ b/common/dlmalloc.c
@@ -581,6 +581,9 @@ void *sbrk(ptrdiff_t increment)
 	ulong old = mem_malloc_brk;
 	ulong new = old + increment;
 
+	if ((new < mem_malloc_start) || (new > mem_malloc_end))
+		return (void *)MORECORE_FAILURE;
+
 	/*
 	 * if we are giving memory back make sure we clear it out since
 	 * we set MORECORE_CLEARS to 1
@@ -588,9 +591,6 @@ void *sbrk(ptrdiff_t increment)
 	if (increment < 0)
 		memset((void *)new, 0, -increment);
 
-	if ((new < mem_malloc_start) || (new > mem_malloc_end))
-		return (void *)MORECORE_FAILURE;
-
 	mem_malloc_brk = new;
 
 	return (void *)old;
-- 
2.35.3