[tbot] Problem with Linux boot to prompt

[Lukasz Majewski]

Dear Wolfgang, Harald,

> Dear Harald,
> 
> In message <1548326964.2436.22.camel at denx.de> you wrote:
> > 
> > Skip does not really make sense in tbot's design, where other
> > testcases might depend on one testcase's return value.  
> 
> Can we please add this as a "nice to have"?
> 
> Maybe even as a general option/flag similar to what "make -k" 
> (--keep-going) is doing?

I've finally found the exact document about tests results (which I
couldn't found with the original question):

"POSIX standard 1003.3 defines what a testing framework needs to
provide, in order to permit the creation of POSIX conformance test
suites. "

https://ftp.gnu.org/pub/old-gnu/Manuals/dejagnu-1.3/html_node/dejagnu_6.html

> 
> Assume you have a system that has been thoroughly tested and then
> released to the customer.  All tesats are running fine.  Later,we
> upddate to a new U-Boot/Linux/Yocto version.  In such a situation a
> quick test run that shows "8 tests are broken" is very useful - for
> example, you can give the customer quick feedback about schedule and
> expected efforts.
> 
> If each failure terminates the test sequence, this means you have to
> actually fix each problem as it occurs, one after the other, and you
> will never now how many breakages are still ahead.
> 
> > If you want to skip, do it
> > like `selftest_board_linux`:
> >
> > 	https://github.com/Rahix/tbot/blob/master/tbot/tc/selftest/board_machine.py#L136  
> 
> This seems unpractical to me if I think at the scenario above.  It
> would be nice to be able to switch on such behaviour without
> changing the test case code, just by flipping a switch :-)
> 
> Thanks!
> 
> Best regards,
> 
> Wolfgang Denk
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma at denx.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.denx.de/pipermail/tbot/attachments/20190208/789bf4c9/attachment.sig>