U-Boot is broken on real N900 HW

[Pali Rohár]

On Saturday 25 April 2020 07:00:58 Adam Ford wrote:
> On Sat, Apr 25, 2020 at 6:50 AM Pali Rohár <pali at kernel.org> wrote:
> >
> > On Saturday 25 April 2020 06:36:58 Adam Ford wrote:
> > > On Sat, Apr 25, 2020 at 5:42 AM Pali Rohár <pali at kernel.org> wrote:
> > > >
> > > > On Thursday 02 April 2020 20:42:31 Pali Rohár wrote:
> > > > > On Wednesday 01 April 2020 12:32:29 Merlijn Wajer wrote:
> > > > > > Hi,
> > > > > >
> > > > > > On 01/04/2020 00:42, Pali Rohár wrote:
> > > > > > > On Wednesday 01 April 2020 00:35:07 Pali Rohár wrote:
> > > > > > >> This patch series contain fixes for Nokia RX-51 board (aka N900).
> > > > > > >> After these changes it is possible to run U-Boot in qemu emulator again.
> > > > > > >> And U-Boot can boot kernel image from RAM, eMMC or OneNAND memory without
> > > > > > >> problem.
> > > > > > >
> > > > > > > But on real Nokia N900 device is U-Boot crashing in reboot loop.
> > > > > > >
> > > > > > > I do not have serial console for Nokia N900 to debug this issue, but
> > > > > > > seems that it is related to OMAP I2C and OMAP HS MMC code. Problem is
> > > > > > > that there is no crash and even no error in qemu emulator so I cannot
> > > > > > > debug this issue.
> > > > > > >
> > > > > > > First problem is around /* reset lp5523 led */ code in rx51.c. On real
> > > > > > > N900 device it generates repeating messages:
> > > > > > >
> > > > > > >   Check if pads/pull-ups of bus are properly configured
> > > > > > >   Timed out in wait_for_event: status=0000
> > > > > > >
> > > > > > > When I commented that few lines all these messages disappeared. So
> > > > > > > problem is with OMAP I2C.
> > > > > > >
> > > > > > > Second problem happen after misc_init_r() function finishes. U-Boot just
> > > > > > > prints on N900 screen message "data abort" and reboots. As I do not have
> > > > > > > serial console it is hard to debug. but I figured out that problem is in
> > > > > > > mmc_set_ios() function in mmc.c file. In function mmc_set_clock() I put
> > > > > > > debug info prior to mmc_set_ios() call and after it, and debug info
> > > > > > > prior was printed, not after.
> > > > > > >
> > > > > > > I remember that somebody had serial jig for Nokia N900, could somebody
> > > > > > > look at this reboot loop problem?
> > > > > > >
> > > > > > > And any idea how should be OMAP I2C configured in U-Boot to correctly
> > > > > > > work?
> > > > > > >
> > > > > > > Maybe I will try to find some time to git bisect which change broke
> > > > > > > U-Boot on real N900 hardware.
> > > > > >
> > > > > > Took latest u-boot master, applied patches and this is the result on
> > > > > > serial (first part is NOLO booting, I think that can be ignored) [1].
> > > > >
> > > > > ...
> > > > >
> > > > > > U-Boot 2020.04-rc4-00033-g7dbafe0634-dirty (Apr 01 2020 - 12:15:47 +0200)
> > > > > >
> > > > > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
> > > > > > Nokia RX-51 + LPDDR/OneNAND
> > > > > > I2C:   ready
> > > > > > DRAM:  256 MiB
> > > > > > NAND:  0 Bytes
> > > > >
> > > > > Looks like that something with NAND is broken.
> > > > >
> > > > > > MMC:   OMAP SD/MMC: 0, OMAP SD/MMC: 1
> > > > > > In:    vga
> > > > > > Out:   vga
> > > > > > Err:   vga
> > > > > > Timed out in wait_for_event: status=0100
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > Timed out in wait_for_event: status=0000
> > > > > > Check if pads/pull-ups of bus are properly configured
> > > > > > i2c_read (addr phase): pads on bus probably not configured (status=0x10)
> > > > > > i2c_write: timed out writig last byte!
> > > > >
> > > > > These i2c errors are caused by
> > > > >
> > > > >       /* reset lp5523 led */
> > > > >       i2c_set_bus_num(1);
> > > > >       state = 0xff;
> > > > >       i2c_write(0x32, 0x3d, 1, &state, 1);
> > > > >       i2c_set_bus_num(0);
> > > > >
> > > > > Is there anything which needs to be done to initialize i2c bus 1?
> > > > > Because this code is working fine on older U-Boot version.
> > > >
> > > > Above code worked fine for U-Boot 2013.04, but in git version from
> > > > January 2015 it prints above error messages.
> > > >
> > > > On on internet forums I see these error messages also from other OMAP3
> > > > board, e.g. beagle board.
> > > >
> > > > Has somebody some working OMAP3 board? And can test if it works with
> > > > recent version of U-Boot? I guess that above i2c problem would happen
> > > > also on other OMAP3 boards.
> > >
> > > There was a conversion a while ago to dm_i2c, and I converted my board
> > > to support using the device tree during the SPL phase, and whenever I
> > > am aware any driver has driver model (DM) support, I try to convert my
> > > board.
> > >
> > > I have a DM3730 and the last check I did was 2020.04-rc1, and it was working
> >
> > Ok, so it either OMAP3430 specific problem or N900 board specific
> > problem. N900 does not use driver model.
> 
> i have an OMAP3530 which is basically a 3430, and it works too.  I am
> guessing the issue is unique to the N900 or the fact that it's
> high-security.  Neither of my boards are HS parts.  They are both GP.

N900 is HS device, but I guess that should be caused by GP vs HS
difference. Working i2c bus 0 and non-working i2c bus 1 could not be
caused by GP vs HS difference. Also I guess that omap hs mmc would be
same on GP and HS boards.

> I would highly consider migrating to the driver model if you can.

Until it works correctly, I cannot do migration.

> Using the driver model, I was able to greatly reduce the size of board
> file and the manual initialization of drivers, because the DM does
> that for you.
> I also know they're advocating removing boards which do not comply
> with the DM requirements.
> 
> >
> > Before calling i2c_write(0x32, ...) I tried to call i2c_probe(0x32) and
> > it returned error.
> >
> > If I tried to call "i2c dev 1" in U-Boot console, I got tons of errors
> > and basically U-Boot stopped responding.
> >
> > So by above observation it looks like I2C bus num 1 does not work, but
> > I2C bus num 0 works fine.
> >
> > Do I need to call i2c_probe(...) before calling i2c_write(...)?
> >
> > And is something special needed for initializing omap i2c bus num 1?
> > Because from my above observation it looks like that something is
> > missing for bus 1 which in older u-boot version was not needed.
> >
> > > U-Boot SPL 2020.04-rc1-01140-ge1dff2d69e-dirty (Feb 15 2020 - 06:18:18 -0600)
> > > Trying to boot from MMC1
> > > spl_load_image_fat_os: error reading image args, err - -2
> > >
> > >
> > > U-Boot 2020.04-rc3-00184-g860eba6a8f-dirty (Mar 26 2020 - 05:43:04 -0500)
> > >
> > > OMAP3630/3730-GP ES1.2, CPU-OPP2, L3-200MHz, Max CPU Clock 1 GHz
> > > Model: LogicPD Zoom DM3730 Torpedo + Wireless Development Kit
> > > Logic DM37x/OMAP35x reference board + LPDDR/NAND
> > > DRAM:  256 MiB
> > > NAND:  512 MiB
> > > MMC:   OMAP SD/MMC: 0
> > > Loading Environment from NAND... OK
> > > OMAP die ID: 619e00029ff800000168300f1502501f
> > > Net:   eth0: ethernet at 08000000
> > > Hit any key to stop autoboot:  2
> > >
> > > >
> > > > > Was something changed to OMAP i2c bus code in U-Boot?
> > > > >
> > > > > > OMAP die ID: 031400240000000004036ac10b01100f
> > > > > > OMAP3530-HS ES3.1, CPU-OPP2, L3-165MHz, Max CPU Clock 600 MHz
> > > > > >
> > > > >
> > > > > And then U-Boot freeze, right?
> > > > >
> > > > > Any idea how to debug this issue?
> > > > >
> > > > > On my N900 I'm getting "data abort" error on display and then instant
> > > > > reboot.
> > > >
> > > > It looks like that omap hs mmc code cause that freeze/reboot on real HW.
> > > > Was there some significat change to OMAP3 or omap hs mmc?
> > >
> > > I booted by board from MMC as shown above.  I also use the pinctrl
> > > features from the device tree to mux the pins in U-Boot (not SPL), so
> > > my SPL only does manual muxing the essential components it needs
> > > during the SPL phase, and lets U-Boot do the rest.   I only  mention
> > > the pinmux because of message regarding pads/pull-ups.
> > >
> > > adam
> >
> > I debugged this problem more. I disabled all preboot commands to get
> > clean U-Boot setup. And it worked.
> >
> > After I called any "mmc" command (e.g. "mmc info") I got that instant
> > board reboot. Preboot commands for n900 try to read some files from mmc,
> > so this is reason why it get into reboot loop.
> >
> > Is there any reason why "mmc info" command can cause "data abort" and
> > instant reboot of board?
> >
> > And do you know what is needed for proper initialization of omap mmc
> > controller for omap3 board? Because it looks like something fundamental
> > is missing.
> >
> > Currently there are just calls for "omap_mmc_init()" functions and
> > "twl4030_power_mmc_init()" functions. See:
> > https://gitlab.denx.de/u-boot/u-boot/-/blob/master/board/nokia/rx51/rx51.c