[tbot] [DENX] tbot: board hangs if no autoload

[Stefano Babic]

Hi Wolfgang,

On 15/11/18 11:57, Wolfgang Denk wrote:
> Dear Stefano,
> 
> In message <4e6fafee-0a11-dbdc-5ebd-15af7529f8d8 at denx.de> you wrote:
>>
>>> U-Boot
>>>
>>> - get sources
>>> - may apply patches to it
>>>   list of patches in a directory ?
>>>   get a list of patches from patchwork todo list?
>>>
>>> - install u-boot on the board
>>> - check if really the new version of u-boot boots
>>
>> I admit that I am not very interested how to build and to push software
>> on the board. There are a lot of different paths, and this is not what
>> the customers ask. They have their own buildserver (Yocto / buildroot /
> 
> However, this is exactly what you do when you prepare a pull-request
> for Tom in your function as U-Boot custodian...
> 
> Saving time there is useful, too, isn't it?

ok, so we should split : tbot for customers (what they ask and what they
want to have), and tbot for internal use. In the second case, it helps,
of course I agree.

Anyway, I admit that most time as maintainer is taken by other
activities. First, review. This cannot be automatically done. Then,
applying patches is something that I have tried to automate, but there
are often a lot of (small) conflicts that must be solved by hand.

Currently, a PR to Tom requires that build on travis reports success,
else a PR is rejected. So yes, I build also myself (but buildman does
the most job), but the rest is done by Travis. This is a different way
to do things, but it was decided by Tom and he's the project leader for
U-Boot.

> 
>> 	- does network work ?
>> 	- does storage work ?
>> 	- do other u-boot peripherals work ?
>>
>> Such cases - they are unaware of which board is running, and we can at
> 
> Thisis not really true, unless you define a number of parameters
> (address ranges for storing images etc. in RAM, for example)
> for all boards.  In the old DUTS setup we did this, but I'm not sure
> if it has been a useful approach.
True, or we can have some generic testcases and the board has this setup
in the default environment (and this is part of the bootloader to be
tested).

> 
>>> - convert all DUTS testcases
>>>   http://git.denx.de/?p=duts.git
>>
>> I do not thing this is a great idea. This "duts" is obsolete and I think
>> we have now a more generic and better concept with tbot. I think we
> 
> DUTS is indeed obsolete, but the test cases have always been useful,
> and they still are.  One issue with U-Boot is the lack of up-to-date
> documentation, and the major reason for that is the lack of a test
> suite to generate the needed log files.
> 
> @Harald: I forgot to ask this during your presentation: can tbot
> generate log files that contain just the commands sent to the target
> and the replies received?
> 
> I understand you play some tricks with fancy prompts etc., but can
> we get something like this?
> 
>>> Linux:
>>>
>>> - get sources
>>> - may apply patches to it
>>> - install linux on the board
>>> - check if booted version is the expected one
>>> - create a register dump file
>>>   write register content into a register dump file
>>> - do register checks
>>
>> See above. I think this is useful during a porting, but it is less
>> useful for a customer who wants to test functionalities. I would like to
>> have first a catalog of testcases with functionalities, like:
> 
> Stefano, I think you miss one key point:  tbot  is _not_ only for
> regression testing for a customer.

ok - I am just focussing on customer, right.

>  It is (and actually most of all)
> a tool that is supposed to free _you_ in your daily work from
> routine tasks.  Your brain is way to precious to be wasted on such
> non-creative routine jobs.  Let tbot do these, and put your brain to
> better use!

ok

Stefano

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic at denx.de
=====================================================================