[U-Boot] [PATCH 1/1] test/py: provide example scripts for integrating qemu

[Heinrich Schuchardt]

On 09/18/2017 11:28 PM, Stephen Warren wrote:
> On 09/18/2017 01:55 PM, Heinrich Schuchardt wrote:
>> On 09/18/2017 08:27 PM, Stephen Warren wrote:
>>> On 09/17/2017 01:32 PM, Heinrich Schuchardt wrote:
>>>> The necessary parameters for running Python tests on qemu are
>>>> tediouus to find.
>>>
>>> Nit: tedious
>>>
>>> Let's wrap the commit description to 72-74 characters; it's rather
>>> narrow right now.
>>>
>>>>
>>>> The patch adds examples for u-boot-test-console and
>>>> u-boot-test-reset.
>>>
>>>> diff --git a/test/py/README.md b/test/py/README.md
>>>> index 829c7efbb2..f3ad10df5a 100644
>>>> --- a/test/py/README.md
>>>> +++ b/test/py/README.md
>>>> @@ -197,6 +197,23 @@ simulator includes a virtual reset button! If
>>>> not, you can launch the
>>>>    simulator from `u-boot-test-reset` instead, while arranging for
>>>> this console
>>>>    process to always communicate with the current simulator instance.
> 
>>>> +    #!/bin/sh
>>>> +    touch /tmp/u-boot-monitor-socket
>>>> +    qemu-system-x86_64 -bios build-qemu-x86/u-boot.rom -nographic
>>>> -netdev \
>>>> +
>>>> user,id=eth0,tftp=../tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
> 
>>> I think this (and the other) script should "exec" the commands to avoid
>>> leaving the shell instance around.
>>
>> Is this really needed? It just adds complexity.
>> You cannot execute anything in the lines after exec,
>> e.g. deleting the socket file.
> 
> Not using exec won't break functionality. However, there's no point
> leaving the shell process hanging about, and since you're editing the
> change anyway it's trivial to fix this up. At present, there's no need
> to run anything after qemu. If we ever have need to, we can remove the
> exec at that time.
> 
>>> This example seems to enable networking support in qemu, and a TFTP
>>> server. I believe you'll need to provide an example Python board
>>> configuration so that test/py knows to enable the network tests.
>>
>> tftp is used for testing bootefi hello
>>
>> Empty file
>> __init__.py
> 
> __init__.py isn't the correct filename. You would need to implement
> u_boot_boardenv_sandbox_na.py I believe. Also, the file can't be empty;
> it needs specific content to enable the TFTP test. See the comments in
> e.g. test/py/tests/test_net.py or test/py/tests/test_efi_loader.py. If
> you don't want to complicate the simple example with this, then I'd
> suggest simplifying the qemu command-line to remove all the
> network/TFTP-related options.
> 
>> u_boot_boardenv_qemu_x86.py
>> env__net_dhcp_server = True
>> env__efi_loader_helloworld_file = {
>>          "fn": "helloworld.efi",
>>          "size": 4298,
>>          "crc32": "55d96ef8",
>> }
>>
>> This is another file needed:
>>
>> u-boot-test-quit
>> #!/bin/sh
>> echo quit | socat - UNIX-CONNECT:/tmp/u-boot-monitor-socket
> 
> The test/py framework doesn't execute "u-boot-test-quit. Adding such a
> file won't affect anything.
> 
>> The following script comes in handy to create the .py file:
>>
>> #!/bin/bash
>> echo env__efi_loader_$(basename $1 | sed 's/\./_/g') = \{
>> echo '    "fn":' $(basename $1)
>> echo '    "len":' $(stat --printf="%s" $1)
>> echo '    "crc32":' $(crc32 $1)
>> echo \}
> 
> Let's just specify the content of the Python file (which the user can
> simply cut/past) rather than making life complicated by writing a shell
> script to create the Python file.
> 
>>>> +    -device e1000,netdev=eth0 -machine pc-i440fx-2.8 \
>>>> +    -monitor unix:/tmp/u-boot-monitor-socket,server,nowait
>>>> +
>>>> +In `u-boot-test-reset` call the socat command to send a system reset:
>>>> +
>>>> +    #!/bin/sh
>>>> +    echo system_reset | socat -
>>>> UNIX-CONNECT:/tmp/u-boot-monitor-socket
>>>> +    sleep 1
>>>> +    true
>>>
>>> Why is the sleep needed?
>>
>> This avoids race conditions.
>> Qemu will need some milliseconds to actually shut down qemu.
>> I want to be sure that Python does not execute any command before this
>> is completed.
> 
> I don't believe there's any issue here. test/py will wait for qemu to
> boot U-Boot before attempting to send any commands after the reset
> occurs, and that wait operation can start as soon as the reset trigger
> is sent. Did you observe any issue in practice?
> 
>>> The true command shouldn't have any effect
>>> given set -e isn't in use.
>>
>> man dash:
>> The shell will return the exit status of the last command executed.
>>
>> If the last command is false running the test suite fails.
> 
> OK. Why would either the echo or sleep fail? If they do, then that
> failure should be passed back to test/py so that it can record the
> problem. Errors shouldn't just be ignored.
> 

true is really needed here. The return code of the script otherwise is
always false even though the system reset succeeds.

If you really want to analyze if qemu successfully executes the control
commands you have to use qmp.

But I wanted to create a trivial example and not start parsing JSON.

Just for reference, this is how you would do a system reset in qmp:

$ socat - /tmp/u-boot-monitor-socket
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 8, "major": 2},
"package": "(Debian 1:2.8+dfsg-6+deb9u2)"}, "capabilities": []}}
==> { "execute": "qmp_capabilities" }
{"return": {}}
==> { "execute" : "system_reset" }
{"return": {}}
{"timestamp": {"seconds": 1505849828, "microseconds": 349723}, "event":
"RESET"}

This is what a failure looks like:

==> { "execute" : "system_reset0"}
{"error": {"class": "CommandNotFound", "desc": "The command
system_reset0 has not been found"}}

Best regards

Heinrich