[U-Boot] [RFC][DO NOT APPLY] binman: Switch to 'python-coverage'

[Tom Rini]

On Fri, Jun 01, 2018 at 08:27:03AM -0600, Simon Glass wrote:

> Hi Tom,
> 
> On 26 May 2018 at 11:06, Tom Rini <trini at konsulko.com> wrote:
> > The most portable way to get access to coverage is to invoke it as
> > 'python-coverage'.
> >
> > Cc: Simon Glass <sjg at chromium.org>
> > Signed-off-by: Tom Rini <trini at konsulko.com>
> > ---
> > So, here's the set of problems I have.  First, running 'pip install
> > coverage' doesn't get me a 'coverage' binary, so I've just never made
> > good use of 'make tests' and so never have these run.  It also means
> > they aren't strictly in travis-ci, so they're never run automatically.
> > With this patch, make tests executes for me, and now fails noting that a
> > lot of things don't have coverage.  I'm honestly not sure what the best
> > path forward here is.
> > ---
> >  tools/binman/README    | 3 +--
> >  tools/binman/binman.py | 4 ++--
> >  2 files changed, 3 insertions(+), 4 deletions(-)
> >
> 
> Reviewed-by: Simon Glass <sjg at chromium.org>
> 
> For me this patch works fine. The -T option works correctly.
> 
> Typing 'python-coverage html' produces this report:
> 
> https://pste.eu/p/NI7q.html
> 
> Note that your patch needs to update this line too:
> 
>         print "Type 'coverage html' to get a report in htmlcov/index.html"
> 
> Can you send me the output you see?

So, I need to amend this a bit.  On my Docker-based ubuntu 16.04 setup,
I see as non-fatal (and not coverage related, but binman test):
<unittest.result.TestResult run=79 errors=0 failures=0>
<unittest.result.TestResult run=11 errors=0 failures=4>
Traceback (most recent call last):
  File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 218, in testGood
    self.assertEqual(result.ok, True)
AssertionError: False != True

Traceback (most recent call last):
  File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 252, in testIndent
    self.assertEqual(len(result.problems), 1)
AssertionError: 2 != 1

Traceback (most recent call last):
  File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 230, in testNoSignoff
    self.assertEqual(len(result.problems), 1)
AssertionError: 2 != 1

Traceback (most recent call last):
  File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 241, in testSpaces
    self.assertEqual(len(result.problems), 3)
AssertionError: 4 != 3

Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/7.3.0/
<unittest.result.TestResult run=35 errors=0 failures=1>
Traceback (most recent call last):
  File "/home/trini/work/u-boot/u-boot/tools/buildman/test.py", line 426, in testToolchainDownload
    self.toolchains.LocateArchUrl('arm'))
AssertionError: 'https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/x86_64-gcc-4.9.0-nolibc_arm-unknown-linux-gnueabi.tar.xz' != 'https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/7.3.0/x86_64-gcc-7.3.0-nolibc_arm-linux-gnueabi.tar.xz'

<unittest.result.TestResult run=12 errors=0 failures=0>
<unittest.result.TestResult run=79 errors=0 failures=0>

And then the coverage part works fine.  With the changes in this RFC
patch so that my desktop itself could run coverage stuff I also see:
Missing tests for u_boot_with_ucode_ptr.py, u_boot_spl_bss_pad.py, entry.py, u_boot_spl.py, u_boot_spl_with_ucode_ptr.py, u_boot_spl_dtb.py, u_boot_img.py, intel_vbt.py, u_boot_dtb.py, intel_vga.py, intel_descriptor.py, u_boot_dtb_with_ucode.py, intel_cmc.py, x86_start16.py, u_boot_nodtb.py, u_boot.py, intel_fsp.py, blob.py, intel_mrc.py, u_boot_spl_nodtb.py, u_boot_ucode.py, x86_start16_spl.py, intel_me.py
ValueError: Test coverage failure
Tests FAILED
Makefile:1710: recipe for target 'tests' failed
make: *** [tests] Error 1

So I guess there's something wrong with trying to use coverage this way,
rather than via pip.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180601/062c09c2/attachment.sig>