Question on running uboot_testpy with tbot

[Harald Seiler]

Hi Simon,

first of all, sorry for the super late response to this :/  I had it on
my list for a long time but had trouble finding enough time to revisit
the test/py integration properly.  You see, the existing test/py
integration has bitrotten quite a lot and I couldn't even get the
existing code working in my environment anymore...

I've now tried reworking all this code for a more robust solution [1].
It has reached a state where it works well for me and if you are still
interested, I'd be grateful for some feedback on how it fares for your
needs.  That said, if your lab and board config haven't changed, the old
integration might still work for you.

The new one is not yet in a tbot release but will be shortly.  In the
meantime you could check it out from the master branch and documentation
can be found here:

        https://tbot.tools/contrib/uboot.html#tbot_contrib.uboot.testpy

More regarding your actual problem below.

[1]: https://github.com/Rahix/tbot/pull/64

On Sat, 2021-10-30 at 13:34 -0600, Simon Glass wrote:
> Hi,
> 
> I am trying to run the pytests on my unit. The documentation doesn't really explain how it works.
> 
> This is what I am trying:
> 
> $ tbot -vv -l kea.py -b rpi3.py -T tbot/contrib  -p 'testpy_args=["-k", "help", "-vv"]' uboot_testpy
> tbot starting ...
> ├─Parameters:
> │     testpy_args = ['-k', 'help', '-vv']
> ├─Calling uboot_testpy ...
> │   ├─Logging in on sglass at kea:22 ...
> │   ├─[sglass] bash --norc --noprofile
> │   ├─Calling uboot_setup_testhooks ...
> │   │   ├─[sglass] echo " ${HOME}"
> │   │   │    ##  /home/sglass
> │   │   ├─[sglass] mkdir -p /home/sglass/tbot-workdir
> │   │   ├─[sglass] test -d /home/sglass/tbot-workdir/uboot-testpy-tbot
> │   │   ├─Creating FIFOs ...
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_send
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_send
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_recv
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_recv
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_commands
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_commands
> │   │   ├─[sglass] cat /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-scripts.sha256
> │   │   │    ## 2d30892b61eb713ce9413e06c4f2a0cd00d2a74b6b8c2ac6624e1e49909b1897
> │   │   ├─Hooks are up to date, skipping deployment ...
> │   │   ├─Adding hooks to $PATH ...
> │   │   ├─[sglass] echo " ${PATH}"
> │   │   │    ##
>  /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/cosarm/depot_tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/l
> ocal/games:/snap/bin:/cosarm/depot_tools:/home/sglass/.local/bin:/vid/software/devel/ubtest/u-boot-test-hooks/bin:/vid/software/devel/ubtest/standalone-hdctools
> │   │   ├─[sglass] export PATH=/home/sglass/tbot-workdir/uboot-testpy-
> tbot:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/cosarm/depot_tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/u
> sr/local/games:/snap/bin:/cosarm/depot_tools:/home/sglass/.local/bin:/vid/software/devel/ubtest/u-boot-test-hooks/bin:/vid/software/devel/ubtest/standalone-hdctools
> │   │   ├─Open console & command channels ...
> │   │   ├─[sglass] /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-console
> │   │   ├─[sglass] /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-commands
> │   │   └─Done. (0.028s)
> │   ├─Calling uboot_checkout ...
> │   │   ├─Builder: rpi3
> │   │   ├─[sglass] test -d /home/sglass/tbot-workdir/uboot-rpi3/.git
> │   │   ├─[sglass] git -C /home/sglass/tbot-workdir/uboot-rpi3 fetch
> │   │   └─Done. (0.121s)
> │   ├─[sglass] test -e /home/sglass/tbot-workdir/uboot-rpi3/.config
> │   ├─[sglass] test -e /home/sglass/tbot-workdir/uboot-rpi3/include/autoconf.mk
> │   ├─[sglass] picocom -q -b 115200 /dev/ttyusb_port1
> │   ├─POWERON (rpi3)
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --dut
> │   ├─[sglass] ykushcmd -s YK17698 -g 1
> │   │    ## 
> │   │    ## 
> │   │    ## Downstream port 1 is OFF
> │   │    ## 
> │   ├─[sglass] ykushcmd -s YK17698 -u 1
> │   ├─UBOOT (rpi3-u-boot)
> │   │    <> 
> │   │    <> 
> │   │    <> U-Boot 2020.10-rc2-00140-g392aa09f310 (Oct 30 2021 - 12:38:07 -0600)
> │   │    <> 
> │   │    <> DRAM:  992 MiB
> │   │    <> RPI 3 Model B (0xa22082)
> │   │    <> MMC:   mmc at 7e202000: 0, sdhci at 7e300000: 1
> │   │    <> Loading Environment from FAT... *** Warning - bad CRC, using default environment
> │   │    <> 
> │   │    <> In:    serial
> │   │    <> Out:   vidconsole
> │   │    <> Err:   vidconsole
> │   │    <> Net:   No ethernet found.
> │   │    <> Hit any key to stop autoboot:  0 
> │   │    <> U-Boot> 
> │   ├─[sglass] cd /home/sglass/tbot-workdir/uboot-rpi3
> │   ├─[sglass] ./test/py/test.py --build-dir . --board-type unknown -k help -vv
> │   │    ## +u-boot-test-flash unknown na
> │   │    ## ============================= test session starts ==============================
> │   │    ## platform linux -- Python 3.6.9, pytest-5.2.1, py-1.8.0, pluggy-0.13.0 -- /usr/bin/python3
> │   │    ## cachedir: .pytest_cache
> │   │    ## rootdir: /home/sglass/tbot-workdir/uboot-rpi3/test/py, inifile: pytest.ini
> │   │    ## collected 324 items / 322 deselected / 2 selected                              
> │   │    ## 
> │   │    ## test/py/tests/test_fpga.py::test_fpga_help SKIPPED                       [ 50%]│   ├─[sglass] ykushcmd -s YK17698 -d 1
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --ts
> │   ├─Exiting poweroff
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --dut
> │   ├─[sglass] ykushcmd -s YK17698 -g 1
> │   │    ## 
> │   │    ## 
> │   │    ## Downstream port 1 is OFF
> │   │    ## 
> │   ├─[sglass] ykushcmd -s YK17698 -u 1
> 
> │   ├─POWEROFF (rpi3)
> │   ├─[sglass] ykushcmd -s YK17698 -d 1
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --ts
> │   ├─Exiting poweroff
> │   └─Fail. (38.376s)
> ├─Exception:
> │   Traceback (most recent call last):
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/main.py", line 345, in main
> │       func(**params)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/decorators.py", line 92, in wrapped
> │       return tc(*args, **kwargs)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/decorators.py", line 157, in wrapped
> │       return tc(lh, *args, **kwargs)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/tc/uboot/testpy.py", line 321, in testpy
> │       os.write(chan_console.fileno(), data)
> │   OSError: [Errno 9] Bad file descriptor
> ├─────────────────────────────────────────
> └─FAILURE (38.488s)
> 
> 
> I don't see the output appearing and I'm not sure how pytest is
> sending the commands to the board?

The challenge here is that tbot is in control of the board (because we
need seamless integration of tbot's board config) while test/py directs
what should happen.  To facilitate this, IO if funneled through tbot by
means of some fifos/named pipes on the host where test/py is running.
Roughly speaking, the setup looks like this (details may differ,
depending on config):


                          +---------------------------------------------------------+
                          |                  "build host"                           |
   +---------------+      |                  has u-boot sources                     |
   | tbot host     |      |                                            +----------+ |
   |  //====================<bh channel>==[captures stdout/stderr]======test/py   | |
   |  |  //=================<m_command channel>==[powercycle trigger]===          | |
   |  |  |    //============<m_console channel>==[board rx/tx console]==invocation| |
   |  |  |    |    |      |                                            +----------+ |
   | +----+   |    |      +---------------------------------------------------------+
   | |tbot|   |    |
   | |runs|   |    |      +------------------------------------+
   | |here|   |    |      | lab host                           |
   | +----+   |    |      | from here, we                      |
   |     |    |    |      | can connect to                     |
   |     |    |    |      | the board                          |
   |     |    |    |      |                                    |     +----------------+
   |     |    \\============<board channel>==[board console]==========    DUT         |
   |     \\=================<lh channel>==[trigger powercycle] |     |  the device    |
   |               |      +------------------------------------+     | that is tested |
   +---------------+                                                 +----------------+

The board console is essentially forwared as is between test/py and the
board.  This is technically handled by tbot, but data is passed through
verbatim.

The m_command channel is used to notify when test/py requests a
powercycle of the board.  tbot then calls the relevant methods from the
board config.

The "bh" channel is where test/py is actually executed and from where
pytest stdout is captured.

Lastly, we usually have an lh channel where the command to actually do a
powercycle runs.

------

That all said, I am not really sure what is going wrong in your case.
The traceback frame

> │     File "tbot/tc/uboot/testpy.py", line 321, in testpy
> │       os.write(chan_console.fileno(), data)
> │   OSError: [Errno 9] Bad file descriptor

indicates that writing to the board's console failed (chan_console is
the <board channel> from above). To me, this means either the terminal
emulator you're using shut down (unlikely).  Or the channel you are
using does not support `.fileno()` correctly/returns some wrong file
descriptor?

Maybe try a different connector if at all possible?  E.g. using
`SSHConnector` instead of `ParamikoConnector` or using
`ConsoleConnector` instead of `PyserialConnector`...

If you can share you configs, I can take a closer look.

Regards,
-- 
Harald