[U-Boot] [PATCH 1/1] test/py: provide example scripts for integrating qemu

[Stephen Warren]

On 09/18/2017 01:55 PM, Heinrich Schuchardt wrote:
> On 09/18/2017 08:27 PM, Stephen Warren wrote:
>> On 09/17/2017 01:32 PM, Heinrich Schuchardt wrote:
>>> The necessary parameters for running Python tests on qemu are
>>> tediouus to find.
>>
>> Nit: tedious
>>
>> Let's wrap the commit description to 72-74 characters; it's rather
>> narrow right now.
>>
>>>
>>> The patch adds examples for u-boot-test-console and
>>> u-boot-test-reset.
>>
>>> diff --git a/test/py/README.md b/test/py/README.md
>>> index 829c7efbb2..f3ad10df5a 100644
>>> --- a/test/py/README.md
>>> +++ b/test/py/README.md
>>> @@ -197,6 +197,23 @@ simulator includes a virtual reset button! If
>>> not, you can launch the
>>>    simulator from `u-boot-test-reset` instead, while arranging for this console
>>>    process to always communicate with the current simulator instance.

>>> +    #!/bin/sh
>>> +    touch /tmp/u-boot-monitor-socket
>>> +    qemu-system-x86_64 -bios build-qemu-x86/u-boot.rom -nographic
>>> -netdev \
>>> +
>>> user,id=eth0,tftp=../tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \

>> I think this (and the other) script should "exec" the commands to avoid
>> leaving the shell instance around.
> 
> Is this really needed? It just adds complexity.
> You cannot execute anything in the lines after exec,
> e.g. deleting the socket file.

Not using exec won't break functionality. However, there's no point 
leaving the shell process hanging about, and since you're editing the 
change anyway it's trivial to fix this up. At present, there's no need 
to run anything after qemu. If we ever have need to, we can remove the 
exec at that time.

>> This example seems to enable networking support in qemu, and a TFTP
>> server. I believe you'll need to provide an example Python board
>> configuration so that test/py knows to enable the network tests.
> 
> tftp is used for testing bootefi hello
> 
> Empty file
> __init__.py

__init__.py isn't the correct filename. You would need to implement 
u_boot_boardenv_sandbox_na.py I believe. Also, the file can't be empty; 
it needs specific content to enable the TFTP test. See the comments in 
e.g. test/py/tests/test_net.py or test/py/tests/test_efi_loader.py. If 
you don't want to complicate the simple example with this, then I'd 
suggest simplifying the qemu command-line to remove all the 
network/TFTP-related options.

> u_boot_boardenv_qemu_x86.py
> env__net_dhcp_server = True
> env__efi_loader_helloworld_file = {
>          "fn": "helloworld.efi",
>          "size": 4298,
>          "crc32": "55d96ef8",
> }
> 
> This is another file needed:
> 
> u-boot-test-quit
> #!/bin/sh
> echo quit | socat - UNIX-CONNECT:/tmp/u-boot-monitor-socket

The test/py framework doesn't execute "u-boot-test-quit. Adding such a 
file won't affect anything.

> The following script comes in handy to create the .py file:
> 
> #!/bin/bash
> echo env__efi_loader_$(basename $1 | sed 's/\./_/g') = \{
> echo '    "fn":' $(basename $1)
> echo '    "len":' $(stat --printf="%s" $1)
> echo '    "crc32":' $(crc32 $1)
> echo \}

Let's just specify the content of the Python file (which the user can 
simply cut/past) rather than making life complicated by writing a shell 
script to create the Python file.

>>> +    -device e1000,netdev=eth0 -machine pc-i440fx-2.8 \
>>> +    -monitor unix:/tmp/u-boot-monitor-socket,server,nowait
>>> +
>>> +In `u-boot-test-reset` call the socat command to send a system reset:
>>> +
>>> +    #!/bin/sh
>>> +    echo system_reset | socat - UNIX-CONNECT:/tmp/u-boot-monitor-socket
>>> +    sleep 1
>>> +    true
>>
>> Why is the sleep needed?
> 
> This avoids race conditions.
> Qemu will need some milliseconds to actually shut down qemu.
> I want to be sure that Python does not execute any command before this
> is completed.

I don't believe there's any issue here. test/py will wait for qemu to 
boot U-Boot before attempting to send any commands after the reset 
occurs, and that wait operation can start as soon as the reset trigger 
is sent. Did you observe any issue in practice?

>> The true command shouldn't have any effect
>> given set -e isn't in use.
> 
> man dash:
> The shell will return the exit status of the last command executed.
> 
> If the last command is false running the test suite fails.

OK. Why would either the echo or sleep fail? If they do, then that 
failure should be passed back to test/py so that it can record the 
problem. Errors shouldn't just be ignored.