[tbot] Problem with Linux boot to prompt

[Wolfgang Denk]

Dear Harald,

In message <1548326964.2436.22.camel at denx.de> you wrote:
> 
> Skip does not really make sense in tbot's design, where other testcases
> might depend on one testcase's return value.

Can we please add this as a "nice to have"?

Maybe even as a general option/flag similar to what "make -k" 
(--keep-going) is doing?

Assume you have a system that has been thoroughly tested and then
released to the customer.  All tesats are running fine.  Later,we
upddate to a new U-Boot/Linux/Yocto version.  In such a situation a
quick test run that shows "8 tests are broken" is very useful - for
example, you can give the customer quick feedback about schedule and
expected efforts.

If each failure terminates the test sequence, this means you have to
actually fix each problem as it occurs, one after the other, and you
will never now how many breakages are still ahead.

> If you want to skip, do it
> like `selftest_board_linux`:
>
> 	https://github.com/Rahix/tbot/blob/master/tbot/tc/selftest/board_machine.py#L136

This seems unpractical to me if I think at the scenario above.  It
would be nice to be able to switch on such behaviour without
changing the test case code, just by flipping a switch :-)

Thanks!

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
I often quote myself; it adds spice to my conversation.  - G. B. Shaw