Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Samuel Holland]

Hello,

I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
where the board fails to boot if I enable watchdog auto-start.

The call to get_timer() -> get_ticks() panics because no timer is
available. And since panic_finish() calls udelay(), this causes infinite
recursion of trying and failing to get the timer.

The issue is that the RISC-V architectural timer driver is bound in
arch_early_init_r, which is only called _after_ the first instance of
INIT_FUNC_WATCHDOG_RESET in the initcall list.

Below is a boot log where I commented out the calls to get_timer() and
time_after_eq() in watchdog_reset() in order to illustrate the problem.

I modified the log to add the initcall symbol names for clarity.

> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00 +0000)
> 
> DRAM:  512 MiB
> initcall: 000000005ff63916
> initcall: 000000005ff6391a
> initcall: 000000004a012a4e (initr_reloc_global_data)
> initcall: 000000004a012a88 (initr_barrier)
> initcall: 000000004a012a1c (initr_malloc)
> Pre-reloc malloc() used 0x720 bytes (1 KB)
> initcall: 000000004a01290e (log_init)
> initcall: 000000004a012a94 (initr_bootstage)
> initcall: 000000004a012a8c (initr_of_live)
> initcall: 000000004a012a06 (initr_dm)
> Binding device root_driver to driver root_driver
> Probing device root_driver with driver root_driver
> Binding device osc24M_clk to driver fixed_clock
> Binding device soc to driver simple_bus
> Binding device pinctrl at 2000000 to driver sunxi-pinctrl
> Binding device pinctrl at 2000000 to driver gpio_sunxi
> Binding device PA to driver gpio_sunxi
> Binding device PB to driver gpio_sunxi
> Binding device PC to driver gpio_sunxi
> Binding device PD to driver gpio_sunxi
> Binding device PE to driver gpio_sunxi
> Binding device PF to driver gpio_sunxi
> Binding device PG to driver gpio_sunxi
> Binding device i2c0-pb10-pins to driver pinconfig
> Binding device i2c2-pb0-pins to driver pinconfig
> Binding device mmc0-pins to driver pinconfig
> Binding device mmc1-pins to driver pinconfig
> Binding device rgmii-pe-pins to driver pinconfig
> Binding device spi0-pins to driver pinconfig
> Binding device spi1-pd-pins to driver pinconfig
> Binding device uart0-pb8-pins to driver pinconfig
> Binding device uart1-pg6-pins to driver pinconfig
> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
> Binding device i2s2-pb-pins to driver pinconfig
> Binding device i2s2-pb3-din-pin to driver pinconfig
> Binding device i2s2-pb4-dout-pin to driver pinconfig
> Binding device ledc-pc0-pin to driver pinconfig
> Binding device pwm0-pd16-pin to driver pinconfig
> Binding device pwm2-pd18-pin to driver pinconfig
> Binding device pwm7-pd22-pin to driver pinconfig
> Binding device spdif-pd22-pin to driver pinconfig
> Binding device clock-controller at 2001000 to driver sun50i_d1_ccu
> Binding device reset to driver sunxi_reset
> Binding device serial at 2500000 to driver ns16550_serial
> Binding device serial at 2500400 to driver ns16550_serial
> Binding device i2c at 2502000 to driver i2c_mvtwsi
> Binding device i2c at 2502800 to driver i2c_mvtwsi
> Binding device mmc at 4020000 to driver sunxi_mmc
> Binding device mmc at 4020000.blk to driver mmc_blk
> Binding device mmc at 4021000 to driver sunxi_mmc
> Binding device mmc at 4021000.blk to driver mmc_blk
> Binding device usb at 4100000 to driver sunxi-musb
> Binding device phy at 4100400 to driver sun4i_usb_phy
> Binding device usb at 4101000 to driver ehci_generic
> Binding device usb at 4101400 to driver ohci_generic
> Binding device usb at 4200000 to driver ehci_generic
> Binding device usb at 4200400 to driver ohci_generic
> Binding device ethernet at 4500000 to driver eth_sun8i_emac
> Binding device watchdog at 6011000 to driver sunxi_wdt
> initcall: 000000004a001914 (board_init)
> initcall: 000000004a04d814 (efi_memory_init)
> initcall: 000000004a012a9c (initr_binman)
> initcall: 000000004a012a90 (initr_dm_devices)

Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
enabled, but that still doesn't help because the timer is not bound yet.
In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
the board hangs if dm_timer_init() fails:

initcall sequence 000000005ffdac78 failed at call 000000004a012a06 (err=-19)
### ERROR ### Please RESET the board ###

> initcall: 000000004a01c4f4 (stdio_init_tables)
> initcall: 000000004a039ff8 (serial_initialize)
> Probing device pinctrl at 2000000 with driver sunxi-pinctrl
> Probing device soc with driver simple_bus
> Probing device clock-controller at 2001000 with driver sun50i_d1_ccu
> Probing device osc24M_clk with driver fixed_clock
> Probing device uart0-pb8-pins with driver pinconfig
> Probing device serial at 2500000 with driver ns16550_serial
> Probing device reset with driver sunxi_reset
> initcall: 000000004a0129ea (initr_announce)
> Now running in RAM - U-Boot at: 5ff51000
> initcall: 000000004a0367ce (initr_watchdog)
> Probing device watchdog at 6011000 with driver sunxi_wdt
> WDT:   Started with servicing (16s timeout)
> initcall: 000000004a012610 (init_func_watchdog_reset)

Here's where the timer needs to be available for watchdog auto-start to
work. An unpatched U-Boot would panic here.

> initcall: 000000004a012a84 (arch_initr_trap)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a000ed4 (arch_early_init_r)
> Binding device cpus to driver cpu_bus
> Binding device cpu at 0 to driver riscv_cpu
> Binding device riscv_timer to driver riscv_timer

But here is where the timer device is actually bound.

Do you have any suggestions for how best to fix this?

Some things I can think of:
 - Add an option to probe the CPU device early in initr_dm_devices()
 - Require the board to probe the CPU device from board_init()
 - Move arch_early_init_r() earlier in the initcall list
 - Skip restarting the watchdog until a timer is available (but this
   would not fix CONFIG_TIMER_EARLY)

Thanks,
Samuel

> Probing device cpus with driver cpu_bus
> Probing device cpu at 0 with driver riscv_cpu
> Binding device sbi-sysreset to driver sbi-sysreset
> initcall: 000000004a012a98 (power_init_board)
> initcall: 000000004a012610 (init_func_watchdog_reset)
> initcall: 000000004a0129ca (initr_mmc)
> MMC:   Probing device mmc0-pins with driver pinconfig
> Probing device mmc at 4020000 with driver sunxi_mmc
> Probing device pinctrl at 2000000 with driver gpio_sunxi
> Probing device PF with driver gpio_sunxi
> Need to init timer!
> Probing device riscv_timer with driver riscv_timer
> Timer is 000000005df34f80
> Read timer, got 0x17dacfb
> Timer is 000000005df34f80
> Read timer, got 0x17f63b8
> Probing device mmc1-pins with driver pinconfig
> Probing device mmc at 4021000 with driver sunxi_mmc
> Timer is 000000005df34f80
> Read timer, got 0x186be7e
> Timer is 000000005df34f80
> Read timer, got 0x188752e
> mmc at 4020000: 0, mmc at 4021000: 1
> initcall: 000000004a012970 (relocated to 000000005ff63970)
> Loading Environment from nowhere... OK
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a012912 (relocated to 000000005ff63912)
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
> initcall: 000000004a010938 (relocated to 000000005ff61938)
> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
> In:    serial at 2500000
> Out:   serial at 2500000
> Err:   serial at 2500000
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
> initcall: 000000004a012950 (relocated to 000000005ff63950)
> initcall: 000000004a012610 (relocated to 000000005ff63610)
> initcall: 000000004a012934 (relocated to 000000005ff63934)
> Net:   phy interface8
> Probing device rgmii-pe-pins with driver pinconfig
> Probing device ethernet at 4500000 with driver eth_sun8i_emac
> Timer is 000000005df34f80
> Read timer, got 0x1ae9538
> Timer is 000000005df34f80
> Read timer, got 0x1b04bed
> Timer is 000000005df34f80
> Read timer, got 0x1b202af
> Timer is 000000005df34f80
> Read timer, got 0x1b3b96e
> Timer is 000000005df34f80
> Read timer, got 0x1b5702c
> Timer is 000000005df34f80
> Read timer, got 0x1b726ec
> 
> Warning: ethernet at 4500000 (eth0) using random MAC address - 62:69:97:68:19:04
> eth0: ethernet at 4500000
> initcall: 000000004a01292a (relocated to 000000005ff6392a)
> Hit any key to stop autoboot:  0