[U-Boot] [PATCH 1/1] test/py: provide example scripts for integrating qemu

[Heinrich Schuchardt]

On 09/18/2017 11:28 PM, Stephen Warren wrote:
> On 09/18/2017 01:55 PM, Heinrich Schuchardt wrote:
>> On 09/18/2017 08:27 PM, Stephen Warren wrote:
>>> On 09/17/2017 01:32 PM, Heinrich Schuchardt wrote:
>>>> The necessary parameters for running Python tests on qemu are
>>>> tediouus to find.
>>>
>>> Nit: tedious
>>>
>>> Let's wrap the commit description to 72-74 characters; it's rather
>>> narrow right now.
>>>
>>>>
>>>> The patch adds examples for u-boot-test-console and
>>>> u-boot-test-reset.
>>>
>>>> diff --git a/test/py/README.md b/test/py/README.md
>>>> index 829c7efbb2..f3ad10df5a 100644
>>>> --- a/test/py/README.md
>>>> +++ b/test/py/README.md
>>>> @@ -197,6 +197,23 @@ simulator includes a virtual reset button! If
>>>> not, you can launch the
>>>>    simulator from `u-boot-test-reset` instead, while arranging for
>>>> this console
>>>>    process to always communicate with the current simulator instance.
> 
>>>> +    #!/bin/sh
>>>> +    touch /tmp/u-boot-monitor-socket
>>>> +    qemu-system-x86_64 -bios build-qemu-x86/u-boot.rom -nographic
>>>> -netdev \
>>>> +
>>>> user,id=eth0,tftp=../tftp,net=192.168.76.0/24,dhcpstart=192.168.76.9 \
> 
>>> I think this (and the other) script should "exec" the commands to avoid
>>> leaving the shell instance around.
>>
>> Is this really needed? It just adds complexity.
>> You cannot execute anything in the lines after exec,
>> e.g. deleting the socket file.
> 
> Not using exec won't break functionality. However, there's no point
> leaving the shell process hanging about, and since you're editing the
> change anyway it's trivial to fix this up. At present, there's no need
> to run anything after qemu. If we ever have need to, we can remove the
> exec at that time.
> 
>>> This example seems to enable networking support in qemu, and a TFTP
>>> server. I believe you'll need to provide an example Python board
>>> configuration so that test/py knows to enable the network tests.
>>
>> tftp is used for testing bootefi hello
>>
>> Empty file
>> __init__.py
> 
> __init__.py isn't the correct filename.

We need 2 *.py files.

__init__.py to make the directory a package directory and the board
file. My example is not for the sandbox but for qemu-x86_defconfig.

> You would need to implement
> u_boot_boardenv_sandbox_na.py I believe. Also, the file can't be empty;
> it needs specific content to enable the TFTP test. See the comments in
> e.g. test/py/tests/test_net.py or test/py/tests/test_efi_loader.py. If
> you don't want to complicate the simple example with this, then I'd
> suggest simplifying the qemu command-line to remove all the
> network/TFTP-related options.
> 
>> u_boot_boardenv_qemu_x86.py
>> env__net_dhcp_server = True
>> env__efi_loader_helloworld_file = {
>>          "fn": "helloworld.efi",
>>          "size": 4298,
>>          "crc32": "55d96ef8",
>> }
>>
>> This is another file needed:
>>
>> u-boot-test-quit
>> #!/bin/sh
>> echo quit | socat - UNIX-CONNECT:/tmp/u-boot-monitor-socket
> 
> The test/py framework doesn't execute "u-boot-test-quit. Adding such a
> file won't affect anything.
> 
>> The following script comes in handy to create the .py file:
>>
>> #!/bin/bash
>> echo env__efi_loader_$(basename $1 | sed 's/\./_/g') = \{
>> echo '    "fn":' $(basename $1)
>> echo '    "len":' $(stat --printf="%s" $1)
>> echo '    "crc32":' $(crc32 $1)
>> echo \}
> 
> Let's just specify the content of the Python file (which the user can
> simply cut/past) rather than making life complicated by writing a shell
> script to create the Python file.
> 
>>>> +    -device e1000,netdev=eth0 -machine pc-i440fx-2.8 \
>>>> +    -monitor unix:/tmp/u-boot-monitor-socket,server,nowait
>>>> +
>>>> +In `u-boot-test-reset` call the socat command to send a system reset:
>>>> +
>>>> +    #!/bin/sh
>>>> +    echo system_reset | socat -
>>>> UNIX-CONNECT:/tmp/u-boot-monitor-socket
>>>> +    sleep 1
>>>> +    true
>>>
>>> Why is the sleep needed?
>>
>> This avoids race conditions.
>> Qemu will need some milliseconds to actually shut down qemu.
>> I want to be sure that Python does not execute any command before this
>> is completed.
> 
> I don't believe there's any issue here. test/py will wait for qemu to
> boot U-Boot before attempting to send any commands after the reset
> occurs, and that wait operation can start as soon as the reset trigger
> is sent. Did you observe any issue in practice?
> 
>>> The true command shouldn't have any effect
>>> given set -e isn't in use.
>>
>> man dash:
>> The shell will return the exit status of the last command executed.
>>
>> If the last command is false running the test suite fails.
> 
> OK. Why would either the echo or sleep fail? If they do, then that
> failure should be passed back to test/py so that it can record the
> problem. Errors shouldn't just be ignored.
> 

I saw your comments late. So I could not integrated them into v2.

Will do so tomorrow.

Regards

Heinrich