[tbot] Testcases duplicated

[Harald Seiler]

Hi Stefano,

On Thu, 2018-12-20 at 13:40 +0100, Stefano Babic wrote:
> Hi,
> 
> I have maybe misunderstood how to compose testcases. I have a general
> testcase (=update the system) in swupdate.py. This takes as parameter
> the new update package.
> 
> Then, each board as a composed testcase where I run more as one test case.
> 
> General tc (swupdate.py):
> -------------------------
> 
> @tbot.testcase
> def checkswupdate(
> ....
> 
> @tbot.testcase
> def swupdateweb(
>     mach: typing.Optional[linux.LinuxMachine] = None,
>     ip = None,
>     path = None,
> ) -> None:
> .....
> 
> board tc:
> ----------
> 
> import contextlib
> import typing
> import time
> import tbot
> from tbot.machine import linux
> from tbot import tc
> from swupdate import *
> 
> @tbot.testcase
> def gtuswupdate(
>     mach: typing.Optional[linux.LinuxMachine] = None,
> ) -> None:
> 
>         .......
>         tc.testsuite(
>             checkswupdate,
>             swupdateweb,
>             mach=lh,
>             path=path,
>             ip=ip
>         )

One thing first:  Are you sure you want tc.testsuite in this case?
tc.testsuite will call ALL testcases, regardless of failures.  I think
you'd want to abort if "checkswupdate" fails?  Or have I misunderstood
your intentions here?

If I am correct, you are better off using:

	@tbot.testcase
	def gtuswupdate(
	    mach: typing.Optional[linux.LinuxMachine] = None,
	) -> None:

	    .......
	    checkswupdate(mach=lh, path=path, ip=ip)
	    swupdateweb(mach=lh, path=path, ip=ip)
	
This will abort if `checkswupdate` fails and not call `swupdateweb`
regardless of that.

> This works when I have just one board. If I add a second board, and I
> want to add a "newboardupdate", I have to import the testcases and tbot
> reports "duplicated testcases" (all files seems to be loaded). What is
> wrong ?

Ah, I see what is going on here ;)  This is actually intended, but for other
reasons and I might look into fixing this specific failure you are seeing here,
because it is just annoying ...

What is going on:

	from swupdate import *

imports all globals from the swupdate module into your current namespace.  This
means, when tbot looks for globals that have a testcase signature, it will not only
find the ones defined in board-testcases.py, but also the ones imported from swupdate.

If your second file now does the same, tbot will find the swupdate testcases in there
a second time!  This is why the error is reported ... The quick solution is to not
import all testcases from swupdate into your own namespace, but keep them in theirs:

	import swupdate as swu

	...

	@tbot.testcase
	def testcase():
		swu.checkswupdate()
		swu.swupdateweb()

Generally I prefer this style anyway, because you can more easily see where a name is
imported from (* imports are discouraged [1]).

But I can also understand that this is irritating and as said before, I will look into
making duplicate detection smarter ...

> Best regards,
> Stefano
> 

[1]: https://stackoverflow.com/questions/2386714/why-is-import-bad
-- 
Harald

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws at denx.de