[v4,0/6] Improve FIT signature handling

[Ludwig Nussel]

On 5/15/26 15:41, Simon Glass wrote:
> Hi Ludwig,
> 
> On 2026-05-13T14:08:10, Ludwig Nussel <ludwig.nussel at siemens.com> wrote:
> 
>> The tests were created with the help of copilot and manual fixups. I
>> have to admit that the log output of those tests is not something I can
>> grok. I've left the tags copilot added in the commits even though
>> checkpatch complains. Not sure what the policy is.
> 
> Thanks for being upfront. For my work I have adjusted patman to drop these tags.
> 
> The "log output I can't grok" part worries me more than the tag. If
> you can't follow what the test is exercising, neither can the next
> person debugging a failure :-)

Meanwhile I got it. The AI helped me to get started.
Anyway, here is what confused me. Suppose we introduce a small typo:
-                      'No signature node found', False)
+                      'No signature node fond', False)

Then run
$ test/py/test.py --bd sandbox "$@" -k 'test_vboot -vvx

You'll get a dump of all kinds of irrelevant things like openssl key 
generation or u-boot startup. Then finally:

FAILED 
test/py/tests/test_vboot.py::test_vboot[sha1-basic-sha1--None-False-True-False-False] 
- assert 'No signature node fond' in "1769 bytes read in 0 msWorking FDT 
set to 100image_pre_load_sig_setup() INFO: no info for image pre-load 
sig check\r\r\n## Loading kernel (any) from FIT Image at 00000100 
...\r\r\n   Using 'conf-1' configuration\r\r\n   Verifying Hash 
Integrity ... fit_config_verify_required_keys() No signature node found: 
FDT_ERR_NOTFOUND\r\r\nBad Data Hash\r\r\nERROR -2: No such file or 
directory: can't get kernel image!"
  +  where "1769 bytes read in 0 msWorking FDT set to 
100image_pre_load_sig_setup() INFO: no info for image pre-load sig 
check\r\r\n## Loading kernel (any) from FIT Image at 00000100 ...\r\r\n 
  Using 'conf-1' configuration\r\r\n   Verifying Hash Integrity ... 
fit_config_verify_required_keys() No signature node found: 
FDT_ERR_NOTFOUND\r\r\nBad Data Hash\r\r\nERROR -2: No such file or 
directory: can't get kernel image!" = <built-in method join of str 
object at 0xa46de8>(['1769 bytes read in 0 ms', 'Working FDT set to 
100', "image_pre_load_sig_setup() INFO: no info for image pre-load sig 
check\r\r\n## Loading kernel (any) from FIT Image at 00000100 ...\r\r\n 
  Using 'conf-1' configuration\r\r\n   Verifying Hash Integrity ... 
fit_config_verify_required_keys() No signature node found: 
FDT_ERR_NOTFOUND\r\r\nBad Data Hash\r\r\nERROR -2: No such file or 
directory: can't get kernel image!"])
  +    where <built-in method join of str object at 0xa46de8> = ''.join

To my untrained eyes that just looks like a lot of garbage :-)
To find the failing line of code you have to scroll back to the actual 
backtrace. Now in this example it's kind of easy as we know what we 
broke. If u-boot produces different output for some reason (like my 
patches) I found it really hard to spot what's wrong in all the noise.

The parametrized test_vboot function doesn't make it easier either, esp 
since some of the parameters select an entirely different set of tests 
(test_global_sign/test_required_key/test_with_algo).

The ubman.log.* calls would explain some context and progress but are 
not in the console, only in the html. The HTML could be more useful due 
to the structuring but isn't as the distracting stuff is on the same 
level as the actual tests. All output leading up to the failure gets 
displayed by the js but the relevant output that is actually looked at 
by assert() is in just one line.

Last but not least even if the presentation of all this information was 
improved, there's still some information about the good case missing. Ie 
what did the output look like when the test was written?

cu
Ludwig

-- 
Ludwig Nussel
Siemens AG
www.siemens.com