[U-Boot] livetime of boards

[Heiko Schocher]

Hello Andreas,

Am 07.11.2013 12:24, schrieb Andreas Bießmann:
> Hello Heiko,
>
> On 11/07/2013 11:39 AM, Heiko Schocher wrote:
>> Am 07.11.2013 10:37, schrieb Andreas Bießmann:
>>> On 11/07/2013 09:17 AM, Heiko Schocher wrote:
>>>> Am 06.11.2013 08:50, schrieb Wolfgang Denk:
>>>>> In message<20131105203736.GM5925 at bill-the-cat>    you wrote:
>
> <snip>
>
>>>> But you are right, that approach leads in a lot of conflicting
>>>> patches ... but I think, we just pooled board information in boards.cfg,
>>>> so this would be the right place in my eyes ...
>>>>
>>>> Maybe we get such Information "a Boards is tested with current mainline"
>>>> inform of an EMail with an Text "Board xy tested with commit mm. Please
>>>> update livetime" ... and we can add a script, which updates the
>>>> livetime for this board, so we can prevent conflicting patches ... ?
>>>
>>> I agree here with Tom. Beside the possibility of conflicting pahces I
>>> see another problem here.
>>> We will get a lot of patches just for increasing the tested counter for
>>> a single board. These patches needs to be handled in some way. If we
>>> shift to some integrated system (gerrit comes to mind) this could be
>>> easier than today, but it will bind resources anyways.
>>> Therefore I think it is a bad idea to save a such often changing
>>> information in the source code repository.
>>
>> I see this info just changing once when releasing a new U-Boot version.
>
> The saved information how often a board was runtime tested with the
> correct SHA1 of the u-boot/master could be quite useful.
> In the end just the last tested commit will be interesting but it could
> give some information how often that specific board is used. The
> information must not be generated by a board maintainer ... the
> maintainer could then see if he needs to pull out a board or if one else
> run the test before.
>
> If we would save this in the repository we do not have this information
> in time.
> If we send the information to a list we need to parse it or use some
> other tool to provide the information.
> Beside that we will pollute the list with status updates about boards
> being tested. It could be hard to find real patches in that information
> flood then.

Hmm... I hope we get a lot of such EMails ... and think, this is not
a big problem ... Or, maybe, if we get a lot of such EMails, maybe we
open a u-boot-testing list?

> <snip mail proposal>
>
>> So (in current case Tom) should, before releasing a new U-Boot
>> version, first call this script "collect_livetime_info" and he get:
>>
>> ->  one livetime counter patch for current release
>> ->  one list for boards which reach end of life
>> ->  one list for boards, which should be deleted
>
> Good idea, but the information could also be saved on a website or in
> another database.
> It should be easily filled by the tester and also easily queried by
> wherever is interested in.

Ok, if we have this info, we can show it wherever we want ...

>> All Infos are "release info" I think, and fully fit in the commit
>> for the new release ...
>
> I also think that should be done on release only.

Yep! But collecting this infos can be done all the time.

>> ... maybe "deleting boards" can be done automatically, but this is
>> not a trivial job ...
>
> I think deleting should be done in next release then to give the board
> maintainer some time to check the boards. On a new release the board
> maintainer should be mailed that in the next release the board will be
> removed. We should also store this somewhere in the code (status in
> boards.cfg?).

See my proposal for the livetime counter:

livetime init value n (n=4)
livetimer decrement on every new release
livetimer set to n, if in release cycle comes a test report
livetimer == 0 -> EMail to board maintainer, board reached end of live in
                   mainline, please send test report.
livetimer == -1 -> board get deleted

So all info is in boards.cfg availiable ...

> Next question is what to do if the mail bounces ;)

Board gets deleted, as board maintainer didn;t send an update patch
for boards.cfg ...

>> So, with such a solution, I see no big additional cost for adding
>> such a feature (except the task "deleting old boards", which is maybe
>> not trivial)
>>
>> Do not understand me wrong, if we find another solution, I am
>> happy also ... just spinning around ...
>
> Me too.
>
> <snip>
>
>>>> If we decide to delete older boards after n release cycles without
>>>> testreports, we must not decide nor look in a database. We are
>>>> sure, we have only "good and working" boards ... and we just
>>>> do the necessary work for new features ... and we are sure, that
>>>> we get back testreports within n release cycles ...
>>>>
>>>> So let us decide first, if we want to go this way ...
>>>
>>> Yes, we should introduce some mechanism to check when a specific board
>>> was last runtime tested. But I fear the overhead with patches that
>>> update a tested counter.
>>
>> I thought with "decide": Do we want to delete "old boards"?
>> With this, we do not need a "MAKEALL --check-boards -s at91" when
>> we introduce new features, as all boards in mainline are in a well
>> tested shape ...
>>
>> Ok, two decisions:
>>
>> - Do we want to collect board testinginformation?
>
> I think we should do that i none way or another.

Yep.

>> - Do we want to delete old boards automatically after we do not get
>>    some test reports after a time intervall?
>
> And we should delete 'unmaintained' boards, when is to be discussed. I'm
> currently fiddling with at91 gpio and ask myself if I should adopt all
> the boards or just let them fail ...

You do not have this problem when we descide to delete old boards!

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany