U-Boot is broken on real N900 HW

[Pali Rohár]

On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
> On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár <pali at kernel.org> wrote:
> >
> > On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
> > > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
> > > > Hi,
> > > >
> > > > On 01/04/2020 00:42, Pali Rohár wrote:
> > > > > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
> > > > >> This patch series contain fixes for Nokia RX-51 board (aka N900).
> > > > >> After these changes it is possible to run U-Boot in qemu emulator again.
> > > > >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without
> > > > >> problem.
> > > > >
> > > > > But on real Nokia N900 device is U-Boot crashing in reboot loop.
> > > > >
> > > > > I do not have serial console for Nokia N900 to debug this issue, but
> > > > > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is
> > > > > that there is no crash and even no error in qemu emulator so I cannot
> > > > > debug this issue.
> > > > >
> > > > > First problem is around /* reset lp5523 led */ code in rx51.c. On real
> > > > > N900 device it generates repeating messages:
> > > > >
> > > > >   Check if pads/pull-ups of bus are properly configured
> > > > >   Timed out in wait_for_event: status=0000
> > > > >
> > > > > When I commented that few lines all these messages disappeared. So
> > > > > problem is with OMAP I2C.
> > > > >
> > > > > Second problem happen after misc_init_r() function finishes. U-Boot just
> > > > > prints on N900 screen message "data abort" and reboots. As I do not have
> > > > > serial console it is hard to debug. but I figured out that problem is in
> > > > > mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put
> > > > > debug info prior to mmc_set_ios() call and after it, and debug info
> > > > > prior was printed, not after.
> > > > >
> > > > > I remember that somebody had serial jig for Nokia N900, could somebody
> > > > > look at this reboot loop problem?
> > > > >
> > > > > And any idea how should be OMAP I2C configured in U-Boot to correctly
> > > > > work?
> > > > >
> > > > > Maybe I will try to find some time to git bisect which change broke
> > > > > U-Boot on real N900 hardware.
> > > >
> > > > Took latest u-boot master, applied patches and this is the result on
> > > > serial (first part is NOLO booting, I think that can be ignored) [1].
> > >
> > > ...
> > >
> > > > U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
> > > >
> > > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
> > > > Nokia RX-51 + LPDDR/OneNAND
> > > > I2C:   ready
> > > > DRAM:  256 MiB
> > > > NAND:  0 Bytes
> > >
> > > Looks like that something with NAND is broken.
> > >
> > > > MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
> > > > In:    vga
> > > > Out:   vga
> > > > Err:   vga
> > > > Timed out in wait_for_event: status=0100
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > Timed out in wait_for_event: status=0000
> > > > Check if pads/pull-ups of bus are properly configured
> > > > i2c_read (addr phase): pads on bus probably not configured (status=0x10)
> > > > i2c_write: timed out writig last byte!
> > >
> > > These i2c errors are caused by
> > >
> > >       /* reset lp5523 led */
> > >       i2c_set_bus_num(1);
> > >       state = 0xff;
> > >       i2c_write(0x32, 0x3d, 1, &state, 1);
> > >       i2c_set_bus_num(0);
> > >
> > > Is there anything which needs to be done to initialize i2c bus 1?
> > > Because this code is working fine on older U-Boot version.
> >
> > Above code worked fine for U-Boot 2013.04, but in git version from
> > January 2015 it prints above error messages.
> >
> > On on internet forums I see these error messages also from other OMAP3
> > board, e.g. beagle board.
> >
> > Has somebody some working OMAP3 board? And can test if it works with
> > recent version of U-Boot? I guess that above i2c problem would happen
> > also on other OMAP3 boards.
> 
> There was a conversion a while ago to dm_i2c, and I converted my board
> to support using the device tree during the SPL phase, and whenever I
> am aware any driver has driver model (DM) support, I try to convert my
> board.
> 
> I have a DM3730 and the last check I did was 2020.04-rc1, and it was working

Ok, so it either OMAP3430 specific problem or N900 board specific
problem. N900 does not use driver model.

Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and
it returned error.

If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors
and basically U-Boot stopped responding.

So by above observation it looks like I2C bus num 1 does not work, but
I2C bus num 0 works fine.

Do I need to call i2c_probe(...) before calling i2c_write(...)?

And is something special needed for initializing omap i2c bus num 1?
Because from my above observation it looks like that something is
missing for bus 1 which in older u-boot version was not needed.

> U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600)
> Trying to boot from MMC1
> spl_load_image_fat_os: error reading image args, err - -2
> 
> 
> U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
> 
> OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz
> Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit
> Logic DM37x/OMAP35x reference board + LPDDR/NAND
> DRAM:  256 MiB
> NAND:  512 MiB
> MMC:   OMAP SD/MMC: 0
> Loading Environment from NAND... OK
> OMAP die ID: 619e00029ff800000168300f1502501f
> Net:   eth0: ethernet at 08000000
> Hit any key to stop autoboot:  2
> 
> >
> > > Was something changed to OMAP i2c bus code in U-Boot?
> > >
> > > > OMAP die ID: 031400240000000004036ac10b01100f
> > > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
> > > >
> > >
> > > And then U-Boot freeze, right?
> > >
> > > Any idea how to debug this issue?
> > >
> > > On my N900 I'm getting "data abort" error on display and then instant
> > > reboot.
> >
> > It looks like that omap hs mmc code cause that freeze/reboot on real HW.
> > Was there some significat change to OMAP3 or omap hs mmc?
> 
> I booted by board from MMC as shown above.  I also use the pinctrl
> features from the device tree to mux the pins in U-Boot (not SPL), so
> my SPL only does manual muxing the essential components it needs
> during the SPL phase, and lets U-Boot do the rest.   I only  mention
> the pinmux because of message regarding pads/pull-ups.
> 
> adam

I debugged this problem more. I disabled all preboot commands to get
clean U-Boot setup. And it worked.

After I called any "mmc" command (e.g. "mmc info") I got that instant
board reboot. Preboot commands for n900 try to read some files from mmc,
so this is reason why it get into reboot loop.

Is there any reason why "mmc info" command can cause "data abort" and
instant reboot of board?

And do you know what is needed for proper initialization of omap mmc
controller for omap3 board? Because it looks like something fundamental
is missing.

Currently there are just calls for "omap_mmc_init()" functions and
"twl4030_power_mmc_init()" functions. See:
https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c