Frustrations of running testing

[Alex G.]

Hi,

I keep being hit by two frustrating issues when trying to run tests. My 
expectation is that I can run a test at any time when working on 
something -- usually to check the correctness of that something. That's 
not the case today.


The first one is an error message about mrproper:

	$ ./test/py/test.py --bd sandbox_spl
	.. is not clean, please run 'make mrproper'

I don't think this message is very useful. Running 'mrproper' is out of 
the question, as it would wipe out the branch I'm currently working on. 
So what I end up doing is deleting the mrimproper check from the 
Makefile. And while it's a fast workaround, it dirties the tree and gets 
in the way of rebasing patches or working with git.

I can continue just fine without 'mrproper death' so I really don't 
understand the over-abundance of caution in breaking the build.



The second issue is the how the tests are trying to build graphics:

	$ ./test/py/test.py --bd sandbox_spl -k "whatever"
	make[1]: sdl2-config: Command not found
	../arch/sandbox/cpu/sdl.c:10:10: fatal error: SDL2/SDL.h:
	No such file or directory
	   10 | #include <SDL2/SDL.h>

The solution is to set NO_SDL=1, but this is far from obvious. The first 
instinct is always to look up the command help:

	$ ./test/py/test.py --help

But this really isn't at all useful. A simple grep for sdl2-config also 
doesn't immediately reveal the solution. Most u-boot tools get their 
arguments via the commandline. Thus, a reasonably competent u-boot 
developer will not think that environment variables are the solution. 
Ergo, environment variables are not the ideal way to solve this.

Maybe we could have a commandline option, and at the very least, catch 
this error and print something useful on the console.


I'm not sure what the preferred way would be to solve the above. For me, 
these issues cause a significant enough disruption to my workflow, that 
I am very likely to not run tests regularly. I suspect I'm not alone.

Alex