am654_sdhci: mmc fail to send stop cmd

[Jan Kiszka]

On 23.07.20 05:25, Faiz Abbas wrote:
> Jan,
>
> On 21/07/20 10:52 pm, Jan Kiszka wrote:
>> On 21.07.20 19:03, Faiz Abbas wrote:
>>> Jan,
>>>
>>> On 21/07/20 12:06 pm, Jan Kiszka wrote:
>>>> On 21.07.20 01:23, Jaehoon Chung wrote:
>>>>> On 7/20/20 10:21 AM, Peng Fan wrote:
>>>>>> Hi Jan,
>>>>>>
>>>>>>> Subject: am654_sdhci: mmc fail to send stop cmd
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> on one device with one specific SD-card (possibly an aging one), I'm seeing
>>>>>>> frequent "mmc fail to send stop cmd" messages, followed by read errors
>>>>>>> when loading kernel and dtb. -ETIMEDOUT is returned by mmd_send_cmd.
>>>>>>> However, I can always resolve this by simply retrying the stop command like
>>>>>>> this:
>>>>>>>
>>>>>>> diff --git a/drivers/mmc/mmc.c b/drivers/mmc/mmc.c index
>>>>>>> f36d11ddc8..9019d9f2ed 100644
>>>>>>> --- a/drivers/mmc/mmc.c
>>>>>>> +++ b/drivers/mmc/mmc.c
>>>>>>> @@ -406,7 +406,11 @@ static int mmc_read_blocks(struct mmc *mmc, void
>>>>>>> *dst, lbaint_t start,  #if !defined(CONFIG_SPL_BUILD) ||
>>>>>>> defined(CONFIG_SPL_LIBCOMMON_SUPPORT)
>>>>>>>                 pr_err("mmc fail to send stop cmd\n");  #endif
>>>>>>> -            return 0;
>>>>>>> +            pr_err("retrying...\n");
>>>>>>> +            if (mmc_send_cmd(mmc, &cmd, NULL)) {
>>>>>>> +                pr_err("failed again\n");
>>>>>>> +                return 0;
>>>>>>> +            }
>>>>>>>             }
>>>>>>>         }
>>>>>>>
>>>>>>>
>>>>>>> Hardware is our IOT2050, baseline is today's master (1c4b5038afcc) with
>>>>>>> board-enabling and a bunch of patches from your tree [1]. However, already
>>>>>>> 4d6da10ce611 exposes the problem.
>>>>>>>
>>>>>>> What could cause this?
>>>>>>
>>>>>> Where the timeout happen in driver?
>>>>>>
>>>>>> Did you try enlarge the timeout value?
>>>>>
>>>>> how about adding SDHCI_QUIRK_WAIT_SEND_CMD?
>>>>
>>>> I tried that already, but the result was even worse, a non-working mmc.
>>>>
>>>>> And as Peng's comment, It needs to find where return error in driver code.
>>>>>
>>>>
>>>> As written in my other reply:
>>>> https://gitlab.denx.de/u-boot/u-boot/-/blob/f12341a9529540113f01989149bbbeb68662a829/drivers/mmc/sdhci.c#L385
>>>> Thus, it's reported by the hw.
>>>>
>>>
>>> Its a command timeout for which we cannot program a higher timeout.
>>>
>>> Can you send a full failure log?
>>>
>>
>> [unrelated fsbl, spl stuff]
>>
>> U-Boot 2020.07-00883-g4d6da10ce6-dirty (Jul 20 2020 - 06:30:08 +0200)
>>
>> Model: Siemens IOT2050 Advanced Base Board
>> DRAM:  2 GiB
>> MMC:   sdhci at 4f80000: 1, sdhci at 04FA0000: 0
>> Loading Environment from SPI Flash... SF: Detected w25q128 with page size 256 Bytes, erase size 64 KiB, total 16 MiB
>> OK
>> In:    serial
>> Out:   serial
>> Err:   serial
>> Hit any key to stop autoboot:  0
>> stat: 18000
>> stat: 18000
>> stat: 208000
>> switch to partitions #0, OK
>> mmc1(part 0) is current device
>> ** No partition table - mmc 1 **
>> switch to partitions #0, OK
>> mmc0 is current device
>> Scanning mmc 0:1...
>> Found U-Boot script /boot/boot.scr
>> 784 bytes read in 2 ms (382.8 KiB/s)
>> ## Executing script at 83000000
>> 65329 bytes read in 11 ms (5.7 MiB/s)
>> stat: 18000
>> mmc fail to send stop cmd, -110
>> retrying...
>> 17113096 bytes read in 1409 ms (11.6 MiB/s)
>> Moving Image from 0x80080000 to 0x80200000, end=812c0000
>> ## Flattened Device Tree blob at 82000000
>>     Booting using the fdt blob at 0x82000000
>>     Loading Device Tree to 00000000fdf0f000, end 00000000fdf21f30 ... OK
>>
>> [kernel boot]
>>
>> The diff I'm carrying on top of [1] is below.
>>
>>> Also, does the same card + board combination work in kernel? That should help us point to hardware vs U-boot.
>>>
>>
>> The same card on the same board works without complaints with the kernel
>> driver (5.8-rc5 at the moment). Even more strange, the same card a
>> different board (IOT2050 Basic, some SoC series, slightly different
>> type) does not throw those errors with the same U-Boot.
>
> Was this card working with an older U-boot version and only failing in mainline?
>

Good point: Just tested our legacy firmware that was based on
https://git.ti.com/cgit/processor-sdk/processor-sdk-u-boot/log/?h=029e4c009aaeaee2d06aa8271dbd3a9e73a28aa7
(https://github.com/siemens/meta-iot2050/blob/master/recipes-bsp/u-boot/u-boot-iot2050-2019.01-ti-sdk.inc),
and it does not expose the issue so far. If I look at the transfer rate,
2.8 MiB/s with the old firmware vs. 11.x MiB/s with upstream, you
suggestion below may make the difference.

>>
>> Note that we are still carrying those clock swapping changes in [2].
>> I've also tried to remove it, but it has no impact on this issue.
>>
>
> One more thing to try is to reduce the speed mode to default as we are already gating frequency
> to 25 MHz. Can you modify the sdhci-caps-mask to the following for sdhci1?
>
> sdhci-caps-mask = <0x7 0x200000>;
>

Trying that out now...

Jan