[u-boot-test-hooks PATCH v4 3/3] Provide some basic scripts for Labgrid integration
neil.armstrong at linaro.org
neil.armstrong at linaro.org
Fri Aug 30 10:38:19 CEST 2024
On 30/08/2024 03:09, Simon Glass wrote:
> Hi Tom,
>
> On Thu, 29 Aug 2024 at 09:20, Tom Rini <trini at konsulko.com> wrote:
>>
>> On Thu, Aug 29, 2024 at 09:02:38AM -0600, Simon Glass wrote:
>>> Hi Neil,
>>>
>>> On Thu, 29 Aug 2024 at 08:44, <neil.armstrong at linaro.org> wrote:
>>>>
>>>> On 29/08/2024 14:17, Simon Glass wrote:
>>>>> Hi Peter,
>>>>>
>>>>> On Thu, 29 Aug 2024 at 04:43, Peter Robinson <pbrobinson at gmail.com> wrote:
>>>>>>
>>>>>> On Wed, 28 Aug 2024 at 22:25, Simon Glass <sjg at chromium.org> wrote:
>>>>>>>
>>>>>>> Hi Peter,
>>>>>>>
>>>>>>> On Wed, 28 Aug 2024 at 12:14, Peter Robinson <pbrobinson at gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Simon,
>>>>>>>>
>>>>>>>>> With Labgrid we don't need to specify the various methods, except for
>>>>>>>>> the console, which simply calls labgrid-client.
>>>>>>>>>
>>>>>>>>> This allows supporting any boards in your lab, without adding per-board
>>>>>>>>> configuration to these hooks.
>>>>>>>>>
>>>>>>>>> Provide ellesmere files as an example.
>>>>>>>>
>>>>>>>> What's ellesmere?
>>>>>>>
>>>>>>> It is a lake but also the name of a computer.
>>>>>>>
>>>>>>>>
>>>>>>>>> Signed-off-by: Simon Glass <sjg at chromium.org>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Changes in v4:
>>>>>>>>> - Support pytest fully with dual-build boards like Beagleplay
>>>>>>>>>
>>>>>>>>> Changes in v3:
>>>>>>>>> - Update scripts for latest version of Labgrid integration
>>>>>>>>> - Add poweroff.none and poweron.none
>>>>>>>>> - Provide -n flag when querying board info
>>>>>>>>> - Target the grpc version of Labgrid which is now in -master
>>>>>>>>> - Update README to cover the changes
>>>>>>>>>
>>>>>>>>> Changes in v2:
>>>>>>>>> - Make use of the common script (only) to set bin_dir
>>>>>>>>>
>>>>>>>>> README.md | 50 ++++++++++++++++++++++++++++++++++++
>>>>>>>>
>>>>>>>> Maybe that should be in a separate labsgrid readme?
>>>>>>>
>>>>>>> My hope is that Labgrid becomes the normal way of running these tests,
>>>>>>
>>>>>> Generally I agree with automated testing platforms, I think <insert
>>>>>> you're preferred platform here> makes sense, there's a bunch like
>>>>>> Linaro and a bunch in the arm ecosystem use LAVA [1] and there's a
>>>>>> bunch more that I'm aware of.
>>>>>
>>>>> I am somewhat familiar with LAVA and I believe it can be used to test
>>>>> U-Boot, although I need to learn how. Looking at a test run [2] for
>>>>> beaglebone black I see that it is using a recent kernel but the U-Boot
>>>>> seems to be older.
>>>>>
>>>>>>
>>>>>> Does it make sense, of course at some point in the future post this
>>>>>> being merged, make sense to look at a general way of making it easier
>>>>>> to plugin these sort of HW test platforms using this as a basis? I ask
>>>>>> mostly because putting a bunch of my devices into some sort of
>>>>>> platform can auto test things and of course everyone has an opinion to
>>>>>> which is the best one :-P
>>>>>
>>>>> Yes. I had heard from Tom that Labgrid is the new hotness for now.
>>>>> Having dug into it I believe it is a good solution, although it can
>>>>> certainly be improved to handle scale better.
>>>>>
>>>>> Anyway, IMO the current test hooks are not a great solution, just
>>>>> because the configuration is spread all over the place and it relies
>>>>> on lots of little shell scripts. So I believe that the Labgrid
>>>>> integration is a closer to where we want to be with others that come
>>>>> along.
>>>>
>>>> I'd say all those scripts are actually here to ease integration with any
>>>> system, booting U-Boot and Linux are two different beasts.
>>>
>>> That's fine, go ahead and use the scripts. My point is that Labgrid
>>> doesn't need them and in fact it makes everything pretty painful if we
>>> try to use all of them.
>>
>> I guess I really need to clean-up and post my former co-workers scripts
>> as I strongly disagree with that statement.
>
> See some examples below. Bear in mind also that my goal has been to
> get my lab fully running, that includes interactive access to boards,
> as well as running tests. For that I have been using tbot (with
> integrated build and software-loading features). WIth Labgrid I have
> been able to replace most of the pytest scripts, tbot and Labman with
> Labgrid + 60 patches and some configuration files.
>
> Let's look through the scripts:
>
> u-boot-test-common - new script
>
> u-boot-test-console - needed
>
> u-boot-test-flash - here's an example:
>
> . poweroff.${power_impl}
> sleep 0.1
> . flash.sdwire_common_mount
> . poweron.${power_impl}
>
> Here's another:
>
> # Handles the common SDwire mounting (caller does power control)
>
> mount_dir=/media/${mount_point}
>
> # Switch over to get USB card access
> sd-mux-ctrl --device-serial ${sdwire_serial} --ts
>
> complete=false
> for i in {0..9}; do
> if out="$(mount UUID=${mount_uuid} 2>&1)"; then
> complete=true
> break
> fi
> echo $out
>
> # If it is already mounted, try to unmount it first. It may have been
> # mounted by another user so we won't have the access we need. If this gives
> # an error then we know we cannot continue
> if [[ $out == *"already mounted"* ]]; then
> umount UUID=${mount_uuid}
> fi
> sleep 1
> done
> if [[ $complete = false ]]; then
> echo "Failed to mount UUID ${mount_uuid} after 10 tries"
> exit 1
> fi
>
> # Sanity check
> if ! mountpoint -q ${mount_dir}; then
> echo "Mount ${mount_dir} not available after 'mount'"
> exit 1
> fi
>
> # Perform the write, pass along as much environment as possible
> . writer.${flash_writer}
>
> complete=false
> for i in {0..9}; do
> if out="$(umount ${mount_dir} 2>&1)"; then
> complete=true
> break
> fi
> echo $out
> sleep 1
> done
>
> if [[ $complete = false ]]; then
> echo "Failed to umount UUID ${mount_uuid} after 10 tries"
> exit 1
> fi
>
> # Sanity check
> if mountpoint -q ${mount_dir}; then
> echo "Mount ${mount_dir} still available after 'umount'"
> exit 1
> fi
>
> # Back to card access for the DUT
> sd-mux-ctrl --device-serial ${sdwire_serial} --dut
>
>
> Of course that assumes an SDwire which may not always be the case.
> Abstracting all of this into shell scripts is just too painful. Also I
> have found it to be error-prone.
>
> There is also bin/flash.sdwire_relay_mount:
>
> . poweroff.${power_impl}
>
> sleep 0.1
>
> . flash.sdwire_common_mount
>
> . poweron.${power_impl}
>
>
> u-boot-test-getrole
> - New script
>
> u-boot-test-power-off
> - This one fits with Labgrid quite well, but of course there may not
> actually be power control
>
> u-boot-test-power-on
> - This one also fits with Labgrid, but since serial output can come
> out as soon as it is called, jumping back into Labgrid later to bring
> up a console results in lots of missing output. Also there may not
> actually be power control
>
> u-boot-test-release
> - New script. Actually this isn't really needed, since I added
> automatic release to Labgrid. I was just worried that Labgrid might
> crash
>
> u-boot-test-reset
> - Unfortunately labgrid doesn't provide a reset option, although I'm
> sure it could be added. You still need to know whether to use power or
> reset or both, though, so again you have shell scripts with this stuff
>
> reset.recovery_download - another thing I had to add, BTW
>
> . "${bin_dir}/recovery.${recovery_impl}"
>
> # USB enumeration delay
> for ((i = 0; i <= 100; i++)); do
> if [ -e "${board_usb_dev}" ]; then
> break
> fi
> sleep 0.1
> done
> if [ ! -e "${board_usb_dev}" ]; then
> echo "Cannot find device ${board_usb_dev}"
> exit 1
> fi
> sleep 1
>
> . "${bin_dir}/download.${download_impl}"
>
> or another case, this is Zynq which tries to wait a bit before calling
> picocom. It mostly works, but it loses output
>
> # Some boards have a UART built into them and it doesn't appear until the
> # board is powered on. An example is zynq_zybo. Try to handle this by waiting
> # for the console and then continuing as soon as possible (so we don't miss
> # the U-Boot banner).
> if [ -n "${console_wait}" ]; then
> for i in {0..99}; do
> if [ -e "${console_dev}" ]; then
> break
> fi
> sleep .1
> done
> if [ ! -e "${console_dev}" ]; then
> echo "Console ${console_dev} not found"
> exit 1
> fi
> fi
>
> exec picocom -b "${console_baud}" "${console_dev}"
>
> So that's about a third of the reason why I don't think the scripts scale.
My integration requires only this file:
bin/console.abcd
====><============
FIP_DIR=`mktemp -d`

pushd ${boot_fip_dir}
./build-fip.sh ${boot_fip_board} ${U_BOOT_BUILD_DIR}/u-boot.bin ${FIP_DIR}
cp ${FIP_DIR}/u-boot.bin ${U_BOOT_BUILD_DIR}/u-boot.bin.abcd
popd
rm -rf ${FIP_DIR}

exec "${abcd_binary}" -h ${abcd_host} -b ${abcd_board} ${abcd_extra_args} ${U_BOOT_BUILD_DIR}/u-boot.bin.abcd
====><============
the abcd_binary does all the dirty work, connecting the the server and connecting the board
TTY to stdout, and the server does all the dirty power-up and loading u-boot to memory.
I still wonder why you need so much changes if labgrid is so simple.
For LAVA, it would likely very similar, exact we should generate a job to load
u-boot, submit it and somehow interact with the board.
Neil
>
>> The only "painful" part is
>> the shared pain, regardless of framework, for "put blob $X at location
>> $Y", which I believe you abstract out in to python?
>
> Sort-of. The Python knows how to handle all the different writing
> methods and is at [2], but it is entirely controlled by yaml[3] (the
> example_env.yaml). See the ti,am625 method for example and the 'play'
> board right at the bottom of the end file.
>
> I very much like the configuration files (although I hope one day
> Labgrid will allow the two files to be combined). It makes it easier
> to add a new board, since there is much less to think about. It really
> did take a long time to fiddle with all those hook scripts and I got
> tired of it.
>
> Regards,
> Simon
>
> [1] https://github.com/sjg20/u-boot/tree/lab6a/tools/labman
> [2] https://github.com/labgrid-project/labgrid/pull/1411/commits/c7ea03987362af70bbbb94d14ff74c54a1ee5ed8#diff-e994502495dc8083c0b70ab20ace18fb5f85217805d01a54270a69c84ec31023R107
> [3] https://github.com/labgrid-project/labgrid/pull/1411/commits/6d3b4a0cfc7e35e0547dcd14065328df4b4b5e1f#diff-66f82743402c95b2eac8f9937c61c582bd0aca2b35e841477d62122c811b7750R1187
More information about the U-Boot
mailing list