[U-Boot] [announce] tbot: add a testcase for checking kconfig move

[Heiko Schocher]

Hello,

just for interests

I just added a testcase to tbot, which checks if U-Boot patch(es)
do not change the resulting U-Boot binaries(currently I check u-boot.bin
and spl/u-boot-spl.bin ... it should be easy to add more if needed)

see tbot testcase:
[1] https://github.com/hsdenx/tbot/commit/bd20fca143e49a1575df5a5532057abbb7234121

For installing tbot please read:
https://github.com/hsdenx/tbot/blob/master/doc/README.install
(if you only want to use this new testcase, you do not need the steps for
  adding a virtual lab, nor a connection to a boards console), you can use your
  PC where you installed tbot (ofcourse this PC must have the ability to compile
  U-Boot, toolchains for supported archs must be installed, also git, parmiko
  must be installed))

For using [1] with tbot, you need the config file:
https://github.com/hsdenx/tbot/blob/master/config/tbot_uboot_kconfig_check.cfg
and adapt it to your local settings. Hope the comments are helpful.

Do not hesitate to contact me, if you have problems to get tbot running.

bye,
Heiko

Here a log running the testcase [1]
(enable tbot to print status info to stdout:
https://github.com/hsdenx/tbot/blob/master/config/tbot_uboot_kconfig_check.cfg#L12
changing this line into "self.debugstatus=True"

also enabled line
https://github.com/hsdenx/tbot/blob/master/src/tc/uboot/tc_uboot_check_kconfig.py#L60
to test for each arch only one board):

$ python2.7 src/common/tbot.py -c config/tbot_uboot_kconfig_check.cfg -t tc_uboot_check_kconfig.py 
-l log/tbot.log
**** option cfg: config/tbot_uboot_kconfig_check.cfg log: log/tbot.log tc: tc_uboot_check_kconfig.py v 0
('CUR WORK PATH: ', '/home/hs/data/Entwicklung/tbot')
('CFGFILE ', 'config/tbot_uboot_kconfig_check.cfg')
('LOGFILE ', '/home/hs/data/Entwicklung/tbot/log/tbot.log')
get u-boot source
testing board adp-ag101p 1 / 13
testing board amcore 2 / 13
testing board ap_sh4a_4a 3 / 13
testing board vct_premium 4 / 13
testing board bf527-ezkit 5 / 13
testing board axs103 6 / 13
testing board axs103 first compile failed
testing board atngw100 7 / 13
testing board microblaze-generic 8 / 13
testing board 10m50 9 / 13
testing board openrisc-generic 10 / 13
testing board openrisc-generic first compile failed
testing board sandbox 11 / 13
sandbox u-boot bin diff da4496233033935d13e599de0b6c72e7 != 290b8d52b847b0d01b0c48d7b77b9aef
testing board gr_ep2s60 12 / 13
testing board chromebook_link 13 / 13
Boards      : 13
compile err : 2
not checked : 0
U-Boot good : 10 bad 1
SPL good    : 1 bad 0
('compile err :', ['axs103', 'openrisc-generic'])
('bad         :', ['sandbox'])
End of TBOT: failure

It seems I have a problem to get sandbox compiling identical binaries,
I must check this, also boards, which does not compile cannot checked ...
but I think, its ready for usage ...
-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany