bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Tom, Hi Simon,

On Wed, Sep 13, 2023 at 9:53 PM Tony Dinh <mibodhi at gmail.com> wrote:
>
> Hi Simon,
>
> On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg at chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > > Hi Tom,
> > > >
> > > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini at konsulko.com> wrote:
> > > > >
> > > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > > >
> > > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > > operation at range" error. However, this error did not affect the
> > > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > > correctly, and u-boot is running normally.
> > > > > >
> > > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > > turned off?
> > > > > >
> > > > > > Please see the log after the break below.
> > > > >
> > > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > > alignment fix there that's not in master, thanks.
> > > >
> > > > Unfortunately I got the same errors. This time the ranges are
> > > > different, of course.
> > > >
> > > > master:
> > > >
> > > > N2350 > bootflow scan scsi
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > > >
> > > > next:
> > > >
> > > > N2350 > bootflow scan scsi
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> > >
> > > Can you debug to where these calls are so we can align these buffers?
> > > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
> >
> > I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.
>
> Looks like we probably need to align the buffer tempbuff.
>
> /drivers/scsi/scsi.c
> static int scsi_detect_dev(struct udevice *dev, int target, int lun,
>   struct blk_desc *dev_desc)
> {
> unsigned char perq, modi;
> lbaint_t capacity;
> unsigned long blksz;
> struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
> int count, err;
>
> pccb->target = target;
> pccb->lun = lun;
> pccb->pdata = (unsigned char *)&tempbuff;
> pccb->datalen = 512;
>
> If you look at the log I posted previously, this error shows up during
> "bootflow scan scsi".
>

Taking the hint from Simon. I turned on log_debug and can see where
the alignment is not correct. It is fs.c fs_read_alloc(). The
memalign() call here probably needs to be revised to take into
consideration ARCH_DMA_MINALIGN somehow? It is 64 for armv7.

diff --git a/fs/fs.c b/fs/fs.c
index 2b815b1db0..b70281532e 100644
--- a/fs/fs.c
+++ b/fs/fs.c
@@ -1019,9 +1019,12 @@ int fs_read_alloc(const char *fname, ulong
size, uint align, void **bufp)
        int ret;

        buf = memalign(align, size + 1);
+       log_debug("aligned buf addr 0x%x\n", (unsigned int)buf);
+
        if (!buf)
                return log_msg_ret("buf", -ENOMEM);
        addr = map_to_sysmem(buf);
+       log_debug("aligned buf sysmem addr 0x%x\n", (unsigned int)addr);

        ret = fs_read(fname, addr, 0, size, &bytes_read);
        if (ret) {

Please see the log below after the break.

All the best,
Tony

========

Log:

   bootmeth_try_file()    /boot/boot.scr - err=0
          fs_devread()  <2, 0, 1024>
          fs_devread()  <8, 0, 32>
          fs_devread()  <8216, 256, 128>

bootmeth_alloc_file()    - script file size 4b7

       fs_read_alloc() aligned buf addr 0x3eb4b608
       fs_read_alloc() aligned buf sysmem addr 0x3eb4b608
          fs_devread()  <12312, 0, 8>

<snip>

          fs_devread()  <551551112, 256, 128>
          fs_devread()  <551592072, 0, 1207>

scsi_read buffer addr=0x3eb4b608
scsi_read_ext: startblk 20e0a88a, blccnt 0 buffer 3EB4B608
scsi_exec pdata=0x3eb4b608
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ahci_scsi_exec pdata=0x3eb4b608
ata_scsiop_read_write
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
ata_scsiop_read_write exiting...
ahci_scsi_exec exiting...
scsi_read exiting...
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08

All the best,
Tony