[U-Boot] livetime of boards

[Heiko Schocher]

Hello Wolfgang, Tom,

Am 06.11.2013 08:50, schrieb Wolfgang Denk:
> Dear Tom,
>
> In message<20131105203736.GM5925 at bill-the-cat>  you wrote:
>>
>>> We have the real problem, that we have a lot of old boards, which
>>> are unmaintained in U-Boot, and we have no chance to find out, if this
>>> boards are longer used/tested ...
>>
>> We also have a feature, lots of hardware support because lots of things
>> don't change drastically, frequently.  That's not to say that I wouldn't
>> mind dropping old platforms (even that ones I have sentimental feelings
>> towards), and would certainly like to see more and frequent sanity
>> testing.

Yep, thats the reason I had in mind. We must have more real tests
on real hardware (or at least, get back the info, that this is done
somewhere ...) ... thats the idea behind to force the board maintainers
to give back us such info.

> I think Heiko's idea of documenting test reports is pretty cool - but
> of course we need to discuss in detail how to implement this, and
> also decide wether we use this for (semi-automatic) code cleanup (by
> removing boards that have not been tested for a long time).

I vote for removing boards from which we get no such test info back.
The board support code is just availiable, just not for current releases.

If code does not change drastically, such a "compile and try it on the
hardware, give feedback to mainline" should not cost much time the board
maintainer, and maybe we add a script, which the board maintainer just
have to call, to send an "board tested EMail" to the mailinglist ...

Maybe a lot of boardmaintainers do this tests, and this info is
important for mainline, but we have no mechanism to collect this
info!

And if code changes drastically (which it currently does and will do
in future I think) a board maintainer must decide, if it makes sense to
sync the board with current mainline or the board get dropped from
mainline ...

And I have this problem currently with i2c subsystem ... a lot of
boards to convert to new framework, and I cannot decide, which are
really maintained... or if currently, the converted boards, really
work. (And I think, not only the i2c subsystem has this problem)

>> This problem comes up when we talk about doing big changes, and I think
>> that's the right time to talk about things.  And I think the answer
>> should be, we try and convert things forward and when it's not obvious
>> if things will still work correctly, or how to do it, that's when we
>> need to make a hard push on the board maintainers to find some time to
>> work on things.
>
> Agreed. And here information how recently (or maybe even how
> frequently) a board has been tested (build tested, run on actual
> hardware) would come in really handy.  we can probbaly automate build
> testing one way or another, but for actual runtime tests we will lways
> depend on the board maintainers, or board users.

Hmm.. Is it always obvious, if changes are big? Do we really get
feedback when doing such big changes? I just reworking the i2c framework,
and I have not really much feedback from board maintainers for their
boards I changed ... Does this change really work on all boards I
converted? I have no chance to get this info. But if we have such a
livetime feature, I can be sure, that after n releases all boards in
mainline are tested ... automatically ... I want such a feature ...

>>> So, the question raises, should we introduce a column in boards.cfg,
>>> which shows the "livetime" of a board support in U-Boot?
>>
>> I sense a lot of conflicting patches.
>
> Again I agree.  Also, I fear that  boards.cfg  is becoming more and
> more unreadable by adding even more stuff.  If I see this correctly,
> the maximum line length in  boards.cfg  already exceeds 360 characters
> :-(

Right, boards.cfg gets unhandy ... Hmm .. what with the column
"Staus" ... instead of "Active" it would be more informative to have
there the livetime counter, and a single digit saves some characters ;-)

But you are right, that approach leads in a lot of conflicting
patches ... but I think, we just pooled board information in boards.cfg,
so this would be the right place in my eyes ...

Maybe we get such Information "a Boards is tested with current mainline"
inform of an EMail with an Text "Board xy tested with commit mm. Please
update livetime" ... and we can add a script, which updates the
livetime for this board, so we can prevent conflicting patches ... ?

> So nstead of adding this information to  boards.cfg  we could probably
> use separate files for such information.  We could provide tools to
> make test reports really easy, say something like
>
> 	scripts/build_test
> 	scripts/run_test
>
> which the user would just call with a "passed" or "failed" argument;
> the scripts could then auto-detect which configuration and which exact
> U-Boot version were in use, and send an email.  Whether that would be
> a patch against the source code or something that get's auto-added to
> a wiki page is just an implementation detail.  But if we had something
> like this, we could get a muchbetter understanding how actively boards
> are being tested.

Yes, that sound also good. I want to see the test information in the
sourcecode, not somewhere on a wiki...

> So when you're once again doing some change that requires touching
> files for some othe rboards, you could simply check that database.  If
> you see that 3 out of the last 5 releases have reported succesful
> run-time tests you will probably decide to accept the needed efforts,

Hmm.. that works, if you have to touch some (some < 5) boards. But
If you have to touch > 5 boards, this gets unhandy...

> but when you see the last test report is more than 5 years old, you
> will probably rather decide to initiate a code removal process.

If we decide to delete older boards after n release cycles without
testreports, we must not decide nor look in a database. We are
sure, we have only "good and working" boards ... and we just
do the necessary work for new features ... and we are sure, that
we get back testreports within n release cycles ...

So let us decide first, if we want to go this way ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany