[PATCH V6 3/4] lib: hashtable: Prevent recursive calling of callback functions

[Christoph Niedermaier]

In case there are two variables which each implement env callback
that performs env_set() on the other variable, the callbacks will
call each other recursively until the stack runs out. Prevent such
a recursion from happening.

Example which triggers this behavior:
static int on_foo(...) { env_set("bar", 0); ... }
static int on_bar(...) { env_set("foo", 0); ... }
U_BOOT_ENV_CALLBACK(foo, on_foo);
U_BOOT_ENV_CALLBACK(bar, on_bar);

Signed-off-by: Christoph Niedermaier <cniedermaier at dh-electronics.com>
Suggested-by: Marek Vasut <marex at denx.de>
---
Cc: Tom Rini <trini at konsulko.com>
Cc: Simon Glass <sjg at chromium.org>
Cc: Marek Vasut <marex at denx.de>
Cc: u-boot at dh-electronics.com
---
V4: - Add this patch to the series
v5: - No functional changes, but changed the order of the patches
V6: - No changes
---
 lib/hashtable.c | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/lib/hashtable.c b/lib/hashtable.c
index e8a59e2dca..75c263b505 100644
--- a/lib/hashtable.c
+++ b/lib/hashtable.c
@@ -221,11 +221,32 @@ static int
 do_callback(const struct env_entry *e, const char *name, const char *value,
 	    enum env_op op, int flags)
 {
+	int ret = 0;
+
 #ifndef CONFIG_XPL_BUILD
-	if (e->callback)
-		return e->callback(name, value, op, flags);
+	static bool in_callback;
+
+	if (!e->callback || in_callback)
+		return 0;
+
+	/*
+	 * In case there are two variables which each implement env callback
+	 * that performs env_set() on the other variable, the callbacks will
+	 * call each other recursively until the stack runs out. Prevent such
+	 * a recursion from happening.
+	 *
+	 * Example which triggers this behavior:
+	 * static int on_foo(...) { env_set("bar", 0); ... }
+	 * static int on_bar(...) { env_set("foo", 0); ... }
+	 * U_BOOT_ENV_CALLBACK(foo, on_foo);
+	 * U_BOOT_ENV_CALLBACK(bar, on_bar);
+	 */
+	in_callback = true;
+	ret = e->callback(name, value, op, flags);
+	in_callback = false;
 #endif
-	return 0;
+
+	return ret;
 }
 
 /*
-- 
2.30.2