[RFC] make sandbox UT more generic

[Simon Glass]

Hi,

On Wed, 30 Aug 2023 at 23:28, AKASHI Takahiro
<takahiro.akashi at linaro.org> wrote:
>
> Hi Simon,
>
> On Wed, Aug 30, 2023 at 08:49:05PM -0600, Simon Glass wrote:
> > Hi,
> >
> > On Wed, 30 Aug 2023 at 18:38, AKASHI Takahiro
> > <takahiro.akashi at linaro.org> wrote:
> > >
> > > Hi,
> > >
> > > I'm working on implementing SCMI-based pinctrl/gpio driver,
> > > and want to re-use sandbox UT to test the code. However,
> > > It is somehow sandbox-specific (with additional DT nodes).
> > > How can/should we make it more generic for other targets/drivers
> > > rather than just by copying the test code?
> > > (I have already created a test for pinmux since there is only
> > > one existing scenario, but gpio test has many.)
> > >
> > > Even if I say 'generic', my case may be special since real
> > > hardware (device drivers) cannot always run all the test cases,
> > > while SCMI-based drivers potentially can with a dummy SCMI server
> > > for sandbox.
> > > See:
> > >     drivers/firmware/scmi/sandbox-scmi_agent.c
> >
> > We don't have a good way to test drivers that talk to hardware, in general.
> >
> > For I2C, SPI and some PCI devices you can sometimes write an emulator
> > for the chip and then your driver can talk to the emulator as if it
> > were talking to the hardware. Sandbox does actually support that with
> > memory-mapped I/O too, although it is fairly rarely used.
>
> Well, I don't want or need to emulate some *real* hardware.
> Instead, I would like to emulate what the current sandbox drivers
> (pinctrl-sandbox.c and gpio/sandbox.c) emulate so that we can re-use
> (some portion of) test cases for sandbox (test/dm/pinmux.c and gpio.c).
>
> As you might know, SCMI protocol with associated drivers on U-Boot is
> so generic that it would be able to talk to any of real pinctrl/gpio
> drivers/firmware (say, run on OPTEE or SCP).
> By implementing/mimicking protocol messages in sandbox-scmi_agent.c,
> SCMI drivers are expected to provide *virtual* pinctrl/gpio devices
> similar to what sandbox does.

I actually know almost nothing about SCMI.

>
> I have already implemented pinmux test with some tweaks by copying
> test/dm/pinmux.c and duplicating almost the same DT nodes as "pinctrl-gpio"
> in test.dts.
> But I'm looking for any other means without test code duplication.
>
> Did I clarify my question a bit?

Well you should be able to factor out the test code into a function,
then call it from two places with the two different devices (or other
params) that are needed.

For the DT, copying a few nodes is not the end of the world, IMO.

BTW have you seen this talk? [2] It seems that you are moving pieces
into firmware which should be OS drivers?

Anyway, if you place a sandbox pinmux device under the SCMI node in
the DT, then you should end up with a pinmux device you can use likely
normal. Then if that device uses the sandbox emulator, you can run the
existing tests on it with little modification, I suspect.

But if I am still missing the point, a diagram or patch might help me
understand!

Regards,
Simon


>
> -Takahiro Akashi
>
>
> > We have done this a lot with Zephyr, as well[1] and achieved 90% code
> > coverage on some boards.
> >
> > But I'm not quite sure I am answering the right question, so I will stop here.
> >
> > Regards,
> > Simon
> >
> > [1] https://www.youtube.com/watch?v=usXCAXR2G_c

[2] https://www.usenix.org/conference/osdi21/presentation/fri-keynote