bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

On Fri, Sep 15, 2023 at 6:32 PM Tony Dinh <mibodhi at gmail.com> wrote:
>
> Hi Tom, Hi Simon,
>
> On Wed, Sep 13, 2023 at 9:53 PM Tony Dinh <mibodhi at gmail.com> wrote:
> >
> > Hi Simon,
> >
> > On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg at chromium.org> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini at konsulko.com> wrote:
> > > > > >
> > > > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > > > >
> > > > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > > > operation at range" error. However, this error did not affect the
> > > > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > > > correctly, and u-boot is running normally.
> > > > > > >
> > > > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > > > turned off?
> > > > > > >
> > > > > > > Please see the log after the break below.
> > > > > >
> > > > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > > > alignment fix there that's not in master, thanks.
> > > > >
> > > > > Unfortunately I got the same errors. This time the ranges are
> > > > > different, of course.
> > > > >
> > > > > master:
> > > > >
> > > > > N2350 > bootflow scan scsi
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > > > >
> > > > > next:
> > > > >
> > > > > N2350 > bootflow scan scsi
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> > > >
> > > > Can you debug to where these calls are so we can align these buffers?
> > > > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
> > >
> > > I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.
> >
> > Looks like we probably need to align the buffer tempbuff.
> >
> > /drivers/scsi/scsi.c
> > static int scsi_detect_dev(struct udevice *dev, int target, int lun,
> >   struct blk_desc *dev_desc)
> > {
> > unsigned char perq, modi;
> > lbaint_t capacity;
> > unsigned long blksz;
> > struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
> > int count, err;
> >
> > pccb->target = target;
> > pccb->lun = lun;
> > pccb->pdata = (unsigned char *)&tempbuff;
> > pccb->datalen = 512;
> >
> > If you look at the log I posted previously, this error shows up during
> > "bootflow scan scsi".
> >
>
> Taking the hint from Simon. I turned on log_debug and can see where
> the alignment is not correct. It is fs.c fs_read_alloc(). The
> memalign() call here probably needs to be revised to take into
> consideration ARCH_DMA_MINALIGN somehow? It is 64 for armv7.
>
> diff --git a/fs/fs.c b/fs/fs.c
> index 2b815b1db0..b70281532e 100644
> --- a/fs/fs.c
> +++ b/fs/fs.c
> @@ -1019,9 +1019,12 @@ int fs_read_alloc(const char *fname, ulong
> size, uint align, void **bufp)
>         int ret;
>
>         buf = memalign(align, size + 1);
> +       log_debug("aligned buf addr 0x%x\n", (unsigned int)buf);
> +
>         if (!buf)
>                 return log_msg_ret("buf", -ENOMEM);
>         addr = map_to_sysmem(buf);
> +       log_debug("aligned buf sysmem addr 0x%x\n", (unsigned int)addr);
>
>         ret = fs_read(fname, addr, 0, size, &bytes_read);
>         if (ret) {
>
> Please see the log below after the break.

Actually, it looks like the fix should be in bootmeth_script.c.

diff --git a/boot/bootmeth_script.c b/boot/bootmeth_script.c
index 0269e0f9b0..68e77aa50a 100644
--- a/boot/bootmeth_script.c
+++ b/boot/bootmeth_script.c
@@ -99,7 +99,7 @@ static int script_read_bootflow_file(struct udevice *bootstd,
        if (!bflow->subdir)
                return log_msg_ret("prefix", -ENOMEM);

-       ret = bootmeth_alloc_file(bflow, 0x10000, 1);
+       ret = bootmeth_alloc_file(bflow, 0x10000, ARCH_DMA_MINALIGN);

All the best,
Tony

> ========
>
> Log:
>
>    bootmeth_try_file()    /boot/boot.scr - err=0
>           fs_devread()  <2, 0, 1024>
>           fs_devread()  <8, 0, 32>
>           fs_devread()  <8216, 256, 128>
>
> bootmeth_alloc_file()    - script file size 4b7
>
>        fs_read_alloc() aligned buf addr 0x3eb4b608
>        fs_read_alloc() aligned buf sysmem addr 0x3eb4b608
>           fs_devread()  <12312, 0, 8>
>
> <snip>
>
>           fs_devread()  <551551112, 256, 128>
>           fs_devread()  <551592072, 0, 1207>
>
> scsi_read buffer addr=0x3eb4b608
> scsi_read_ext: startblk 20e0a88a, blccnt 0 buffer 3EB4B608
> scsi_exec pdata=0x3eb4b608
> ahci_scsi_exec
> ahci_scsi_exec cmd = 0x48
> ahci_scsi_exec pdata=0x3eb4b608
> ata_scsiop_read_write
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
> ata_scsiop_read_write exiting...
> ahci_scsi_exec exiting...
> scsi_read exiting...
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
>
> All the best,
> Tony