[U-Boot] [PATCH] test/py: support running sandbox under gdbserver

[Simon Glass]

Hi Stephen,

On 6 February 2016 at 13:34, Stephen Warren <swarren at wwwdotorg.org> wrote:
> On 02/06/2016 01:30 PM, Simon Glass wrote:
>> On 4 February 2016 at 16:11, Stephen Warren <swarren at wwwdotorg.org> wrote:
>>> Implement command--line option --gdbserver COMM, which does two things:
>>>
>>> a) Run the sandbox process under gdbserver, using COMM as gdbserver's
>>>    communication channel.
>>>
>>> b) Disables all timeouts, so that if U-Boot is halted under the debugger,
>>>    tests don't fail. If the user gives up in the middle of a debugging
>>>    session, they can simply CTRL-C the test script to abort it.
>>>
>>> This allows easy debugging of test failures without having to manually
>>> re-create the failure conditions. Usage is:
>>>
>>> Window 1:
>>> ./test/py/test.py --bd sandbox --gdbserver localhost:1234
>>>
>>> Window 2:
>>> gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'
>>>
>>> When using this option, it likely makes sense to use pytest's -k option
>>> to limit the set of tests that are executed.
>>>
>>> Simply running U-Boot directly under gdb (rather than gdbserver) was
>>> also considered. However, this was rejected because:
>>>
>>> a) gdb's output would then be processed by the test script, and likely
>>>    confuse it causing false failures.
>>>
>>> b) pytest by default hides stdout from tests, which would prevent the
>>>    user from interacting with gdb.
>>>
>>>    While gdb can be told to redirect the debugee's stdio to a separate
>>>    PTY, this would appear to leave gdb's stdio directed at the test
>>>    scripts and the debugee's stdio directed elsewhere, which is the
>>>    opposite of the desired effect. Perhaps some complicated PTY muxing
>>>    and process hierarchy could invert this. However, the current scheme
>>>    is simple to implement and use, so it doesn't seem worth complicating
>>>    matters.
>>>
>>> c) Using gdbserver allows arbitrary debuggers to be used, even those with
>>>    a GUI. If the test scripts invoked the debugger themselves, they'd have
>>>    to know how to execute arbitary applications. While the user could hide
>>>    this all in a wrapper script, this feels like extra complication.
>>>
>>> An interesting future idea might be a --gdb-screen option, which could
>>> spawn both U-Boot and gdb separately, and spawn the screen into a newly
>>> created window under screen. Similar options could be envisaged for
>>> creating a new xterm/... too.
>>>
>>> --gdbserver  currently only supports sandbox, and not real hardware.
>>> That's primarily because the test hooks are responsible for all aspects of
>>> hardware control, so there's nothing for the test scripts themselves can
>>> do to enable gdbserver on real hardware. We might consider introducing a
>>> separate --disable-timeouts option to support use of debuggers on real
>>> hardware, and having --gdbserver imply that option.
>>>
>>> Signed-off-by: Stephen Warren <swarren at nvidia.com>
>>> ---
>>>  test/py/conftest.py               |  8 ++++++++
>>>  test/py/tests/test_sleep.py       |  7 ++++---
>>>  test/py/u_boot_console_base.py    |  3 ++-
>>>  test/py/u_boot_console_sandbox.py |  5 ++++-
>>>  test/py/u_boot_spawn.py           | 12 ++++++++----
>>>  5 files changed, 26 insertions(+), 9 deletions(-)
>>
>> Can you please add info about this to the docs?
>>
>> Also for me this worked up to the point where it ran the
>> test_sandbox_exit.py test. Then the gdb process said that U-Boot
>> exited normally. Is that test not compatible with this feature?
>
> The sandbox_exit test deliberately causes the sandbox process to exit,
> to make sure that the "reset" command and "typing" Ctrl-C work. To
> continue the test, simply re-run gdb to re-attach to the new gdbserver
> and U-Boot process.

That's a bit annoying. Perhaps we should have a flag that disabled
such tests? Or perhaps a way to specify what tests are run?

Anyway, can you add this to the docs too?

Regards,
Simon