[U-Boot-Custodians] [U-Boot-Board-Maintainers] [U-Boot] [ANN] U-Boot v2019.07-rc4 released

Tue Jun 25 03:51:48 UTC 2019

Hello Heinrich,

Am 22.06.2019 um 21:12 schrieb Heinrich Schuchardt:
> On 6/22/19 8:15 PM, Simon Glass wrote:
>> Hi,
>>
>> On Sat, 22 Jun 2019 at 16:10, Andreas Färber <afaerber at suse.de> wrote:
>>>
>>> Hi Simon,
>>>
>>> Am 22.06.19 um 16:55 schrieb Simon Glass:
>>>> I'd like to better understand the benefits of the 3-month timeline.
>>>
>>> It takes time to learn about a release, package and build it, test it on
>>> various hardware, investigate and report errors, wait for feedback and
>>> fixes, rinse and repeat with the next -rc. Many people don't do this as
>>> their main job.
>>>
>>> If we shorten the release cycle, newer boards will get out faster (which
>>> is good) but the overall quality of boards not actively worked on
>>> (because they were working good enough before) will decay, which is bad.
>>> The only way to counteract that would be to automatically test on real
>>> hardware rather than just building, and doing that for all these masses
>>> of boards seems unrealistic.
>>
>> Here I think you are talking about distributions. But why not just
>> take every second release?
>>
>> I have certain had the experience of getting a board our of the
>> cupboard and finding that the latest U-Boot doesn't work, nor the one
>> before, nor the three before that.
>>
>> Are we actually seeing an improvement in regressions? I feel that
>> testing is the only way to get that.
>>
>> Perhaps we should select a small subset of boards which do get tested,
>> and actually have custodians build/test on those for every rc?
> 
> What I have been doing before all my recent pull requests is to boot
> both an arm32 (Orange Pi) and and an aarch64 (Pine A64 LTS) board via
> bootefi and GRUB. To make this easier I am using a Raspberry with a
> relay board and a Tizen SD-Wire card (https://wiki.tizen.org/SDWire)
> controlling the system under test,
> cf https://pbs.twimg.com/media/D5ugi3iX4AAh1bn.jpg:large
> What would be needed is scripts to automate the testing including all
> the Python tests.
> 
> It would make sense to have such test automation for all of our
> architectures similar to what Kernel CI (https://kernelci.org/) does.

Yes ... my dream is also we have a lot of boards, on which we can test
automagically. My approach was tbot, with which I made weekly automated
commandline tests (u-boot and linux), see example results from my old
tbot version:

http://xeidos.ddns.net/tests/test_db_auslesen.php#987

(Intentionally link to latest good test, as I found no time yet to
  refresh my testsetup to new tbot, see later).

Above webpage is created through a tbot generator, which fills a
mysql database and the webpage is a simple php script. I also speculated
to write a generator to connect to kernel-ci ... but time is the missing
resource.

For the above example tbot and the webserver run on a raspberry pi,
also the raspberry pi is used as "Lab Host" which controlls the board
under test, so cheap hardware costs. It is easy to adapt tbot to your
hardware setup, see [2]

But tbot was a hack as my python skills are not the greatest, so
luckily Harald (added to cc) found time to rewrite it completly from
scratch (many thanks!), see [1] and [2]

Ok, missing here and there functionality from the old version, but
it is much more cleaner ... may you take a look at it, also it is
Open Source, feel free to help and improve tbot!

I use tbot for my daily work, as tbot has also an interactive mode [3],
with which you can use tbot for powering on your board and connect
to for example the u-boot command line). No need anymore to know where
your board is and how to power it on or how to connect to the console.
(Background I mostly have no real physical access to hardware I work
  on)

So you can work while developing with tbot, and at the end you have
immediately an automated setup you can integrate into your CI. As
tbot is a commandline tool, this is mostly easy to do.

Also it is possible to call pytest framework [4] from u-boot,
and of course you can call this out of tbot [5].

bye,
Heiko

[1] https://github.com/Rahix/tbot
[2] https://rahix.de/tbot/
[3] https://rahix.de/tbot/getting-started.html#interactive
[4] 
http://git.denx.de/?p=u-boot.git;a=blob;f=test/README;h=4bc9ca3a6ae9de0e3ed6ff420c5edfe6027ad2c6;hb=77f6e2dd0551d8a825bab391a1bd6b838874bcd4
[5] result from pytest framework called from tbot
     http://xeidos.ddns.net/tbot/id_985/test-log.html

> 
> Regards
> 
> Heinrich
> _______________________________________________
> U-Boot-Custodians mailing list
> U-Boot-Custodians at lists.denx.de
> https://lists.denx.de/listinfo/u-boot-custodians

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-52   Fax: +49-8142-66989-80   Email: hs at denx.de