[tbot] [DENX] tbot: board hangs if no autoload

[Heiko Schocher]

Hello Stefano,

Am 15.11.2018 um 12:37 schrieb Stefano Babic:
> Hi Wolfgang,
> 
> On 15/11/18 11:57, Wolfgang Denk wrote:
>> Dear Stefano,
>>
>> In message <4e6fafee-0a11-dbdc-5ebd-15af7529f8d8 at denx.de> you wrote:
>>>
>>>> U-Boot
>>>>
>>>> - get sources
>>>> - may apply patches to it
>>>>    list of patches in a directory ?
>>>>    get a list of patches from patchwork todo list?
>>>>
>>>> - install u-boot on the board
>>>> - check if really the new version of u-boot boots
>>>
>>> I admit that I am not very interested how to build and to push software
>>> on the board. There are a lot of different paths, and this is not what
>>> the customers ask. They have their own buildserver (Yocto / buildroot /
>>
>> However, this is exactly what you do when you prepare a pull-request
>> for Tom in your function as U-Boot custodian...
>>
>> Saving time there is useful, too, isn't it?
> 
> ok, so we should split : tbot for customers (what they ask and what they
> want to have), and tbot for internal use. In the second case, it helps,
> of course I agree.
> 
> Anyway, I admit that most time as maintainer is taken by other
> activities. First, review. This cannot be automatically done. Then,

No, first let tbot do the tasks, which are automatable... !

Why reviewing a patch, which does not apply or has checkpatch errors?

> applying patches is something that I have tried to automate, but there
> are often a lot of (small) conflicts that must be solved by hand.

Ok, if you have dependencies on other patches ... yes. Else, incoming
patches must apply to current HEAD of mainline.

> Currently, a PR to Tom requires that build on travis reports success,
> else a PR is rejected. So yes, I build also myself (but buildman does
> the most job), but the rest is done by Travis. This is a different way
> to do things, but it was decided by Tom and he's the project leader for
> U-Boot.

Yes, this is a step I do also "from hand" ... but pushing the current
U-Boot tree to github, so travis starts compiling, is also just a testcase

But parsing the result is not autoamted yet ...

>>> 	- does network work ?
>>> 	- does storage work ?
>>> 	- do other u-boot peripherals work ?
>>>
>>> Such cases - they are unaware of which board is running, and we can at
>>
>> Thisis not really true, unless you define a number of parameters
>> (address ranges for storing images etc. in RAM, for example)
>> for all boards.  In the old DUTS setup we did this, but I'm not sure
>> if it has been a useful approach.
> True, or we can have some generic testcases and the board has this setup
> in the default environment (and this is part of the bootloader to be
> tested).
> 
>>
>>>> - convert all DUTS testcases
>>>>    http://git.denx.de/?p=duts.git
>>>
>>> I do not thing this is a great idea. This "duts" is obsolete and I think
>>> we have now a more generic and better concept with tbot. I think we
>>
>> DUTS is indeed obsolete, but the test cases have always been useful,
>> and they still are.  One issue with U-Boot is the lack of up-to-date
>> documentation, and the major reason for that is the lack of a test
>> suite to generate the needed log files.
>>
>> @Harald: I forgot to ask this during your presentation: can tbot
>> generate log files that contain just the commands sent to the target
>> and the replies received?
>>
>> I understand you play some tricks with fancy prompts etc., but can
>> we get something like this?
>>
>>>> Linux:
>>>>
>>>> - get sources
>>>> - may apply patches to it
>>>> - install linux on the board
>>>> - check if booted version is the expected one
>>>> - create a register dump file
>>>>    write register content into a register dump file
>>>> - do register checks
>>>
>>> See above. I think this is useful during a porting, but it is less
>>> useful for a customer who wants to test functionalities. I would like to
>>> have first a catalog of testcases with functionalities, like:
>>
>> Stefano, I think you miss one key point:  tbot  is _not_ only for
>> regression testing for a customer.
> 
> ok - I am just focussing on customer, right.

:-P

>>   It is (and actually most of all)
>> a tool that is supposed to free _you_ in your daily work from
>> routine tasks.  Your brain is way to precious to be wasted on such
>> non-creative routine jobs.  Let tbot do these, and put your brain to
>> better use!
> 
> ok
> 
> Stefano
> 

bye,
Heiko
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-52   Fax: +49-8142-66989-80   Email: hs at denx.de