Watchdog auto-start and TIMER_EARLY broken in RISC-V U-Boot

[Samuel Holland]

On 9/11/21 1:31 PM, Sean Anderson wrote:
> +CC Heinrich
> 
> Did you ever try booting with WDT on k210?
> 
> On 9/11/21 12:43 PM, Samuel Holland wrote:
>> Hello,
>>
>> I am porting U-Boot to the Allwinner D1 SoC, and I ran into an issue
>> where the board fails to boot if I enable watchdog auto-start.
>>
>> The call to get_timer() -> get_ticks() panics because no timer is
>> available. And since panic_finish() calls udelay(), this causes infinite
>> recursion of trying and failing to get the timer.
>>
>> The issue is that the RISC-V architectural timer driver is bound in
>> arch_early_init_r, which is only called _after_ the first instance of
>> INIT_FUNC_WATCHDOG_RESET in the initcall list.
>>
>> Below is a boot log where I commented out the calls to get_timer() and
>> time_after_eq() in watchdog_reset() in order to illustrate the problem.
>>
>> I modified the log to add the initcall symbol names for clarity.
>>
>>> U-Boot 2021.10-rc2-00278-ge2be1a426d6-dirty (Jan 01 1970 - 00:00:00
>>> +0000)
>>>
>>> DRAM:  512 MiB
>>> initcall: 000000005ff63916
>>> initcall: 000000005ff6391a
>>> initcall: 000000004a012a4e (initr_reloc_global_data)
>>> initcall: 000000004a012a88 (initr_barrier)
>>> initcall: 000000004a012a1c (initr_malloc)
>>> Pre-reloc malloc() used 0x720 bytes (1 KB)
>>> initcall: 000000004a01290e (log_init)
>>> initcall: 000000004a012a94 (initr_bootstage)
>>> initcall: 000000004a012a8c (initr_of_live)
>>> initcall: 000000004a012a06 (initr_dm)
>>> Binding device root_driver to driver root_driver
>>> Probing device root_driver with driver root_driver
>>> Binding device osc24M_clk to driver fixed_clock
>>> Binding device soc to driver simple_bus
>>> Binding device pinctrl at 2000000 to driver sunxi-pinctrl
>>> Binding device pinctrl at 2000000 to driver gpio_sunxi
>>> Binding device PA to driver gpio_sunxi
>>> Binding device PB to driver gpio_sunxi
>>> Binding device PC to driver gpio_sunxi
>>> Binding device PD to driver gpio_sunxi
>>> Binding device PE to driver gpio_sunxi
>>> Binding device PF to driver gpio_sunxi
>>> Binding device PG to driver gpio_sunxi
>>> Binding device i2c0-pb10-pins to driver pinconfig
>>> Binding device i2c2-pb0-pins to driver pinconfig
>>> Binding device mmc0-pins to driver pinconfig
>>> Binding device mmc1-pins to driver pinconfig
>>> Binding device rgmii-pe-pins to driver pinconfig
>>> Binding device spi0-pins to driver pinconfig
>>> Binding device spi1-pd-pins to driver pinconfig
>>> Binding device uart0-pb8-pins to driver pinconfig
>>> Binding device uart1-pg6-pins to driver pinconfig
>>> Binding device uart1-pg8-rts-cts-pins to driver pinconfig
>>> Binding device i2s2-pb-pins to driver pinconfig
>>> Binding device i2s2-pb3-din-pin to driver pinconfig
>>> Binding device i2s2-pb4-dout-pin to driver pinconfig
>>> Binding device ledc-pc0-pin to driver pinconfig
>>> Binding device pwm0-pd16-pin to driver pinconfig
>>> Binding device pwm2-pd18-pin to driver pinconfig
>>> Binding device pwm7-pd22-pin to driver pinconfig
>>> Binding device spdif-pd22-pin to driver pinconfig
>>> Binding device clock-controller at 2001000 to driver sun50i_d1_ccu
>>> Binding device reset to driver sunxi_reset
>>> Binding device serial at 2500000 to driver ns16550_serial
>>> Binding device serial at 2500400 to driver ns16550_serial
>>> Binding device i2c at 2502000 to driver i2c_mvtwsi
>>> Binding device i2c at 2502800 to driver i2c_mvtwsi
>>> Binding device mmc at 4020000 to driver sunxi_mmc
>>> Binding device mmc at 4020000.blk to driver mmc_blk
>>> Binding device mmc at 4021000 to driver sunxi_mmc
>>> Binding device mmc at 4021000.blk to driver mmc_blk
>>> Binding device usb at 4100000 to driver sunxi-musb
>>> Binding device phy at 4100400 to driver sun4i_usb_phy
>>> Binding device usb at 4101000 to driver ehci_generic
>>> Binding device usb at 4101400 to driver ohci_generic
>>> Binding device usb at 4200000 to driver ehci_generic
>>> Binding device usb at 4200400 to driver ohci_generic
>>> Binding device ethernet at 4500000 to driver eth_sun8i_emac
>>> Binding device watchdog at 6011000 to driver sunxi_wdt
>>> initcall: 000000004a001914 (board_init)
>>> initcall: 000000004a04d814 (efi_memory_init)
>>> initcall: 000000004a012a9c (initr_binman)
>>> initcall: 000000004a012a90 (initr_dm_devices)
>>
>> Here's where dm_timer_init() would be called if CONFIG_TIMER_EARLY was
>> enabled, but that still doesn't help because the timer is not bound yet.
>> In fact, CONFIG_TIMER_EARLY actually makes the situation worse, because
>> the board hangs if dm_timer_init() fails:
>>
>> initcall sequence 000000005ffdac78 failed at call 000000004a012a06
>> (err=-19)
>> ### ERROR ### Please RESET the board ###
> 
> I'm not sure what's going on here. Why does dm_timer_init get called
> from initr_dm?

dm_timer_init gets called from initr_dm_devices (not initr_dm) if
CONFIG_TIMER_EARLY is enabled.

> Why does it fail?

It fails because at this point there are no devices bound for the timer
uclass, so uclass_first_device_err returns -ENODEV.

Regards,
Samuel

> --Sean
> 
>>> initcall: 000000004a01c4f4 (stdio_init_tables)
>>> initcall: 000000004a039ff8 (serial_initialize)
>>> Probing device pinctrl at 2000000 with driver sunxi-pinctrl
>>> Probing device soc with driver simple_bus
>>> Probing device clock-controller at 2001000 with driver sun50i_d1_ccu
>>> Probing device osc24M_clk with driver fixed_clock
>>> Probing device uart0-pb8-pins with driver pinconfig
>>> Probing device serial at 2500000 with driver ns16550_serial
>>> Probing device reset with driver sunxi_reset
>>> initcall: 000000004a0129ea (initr_announce)
>>> Now running in RAM - U-Boot at: 5ff51000
>>> initcall: 000000004a0367ce (initr_watchdog)
>>> Probing device watchdog at 6011000 with driver sunxi_wdt
>>> WDT:   Started with servicing (16s timeout)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>
>> Here's where the timer needs to be available for watchdog auto-start to
>> work. An unpatched U-Boot would panic here.
>>
>>> initcall: 000000004a012a84 (arch_initr_trap)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a000ed4 (arch_early_init_r)
>>> Binding device cpus to driver cpu_bus
>>> Binding device cpu at 0 to driver riscv_cpu
>>> Binding device riscv_timer to driver riscv_timer
>>
>> But here is where the timer device is actually bound.
>>
>> Do you have any suggestions for how best to fix this?
>>
>> Some things I can think of:
>>   - Add an option to probe the CPU device early in initr_dm_devices()
>>   - Require the board to probe the CPU device from board_init()
>>   - Move arch_early_init_r() earlier in the initcall list
>>   - Skip restarting the watchdog until a timer is available (but this
>>     would not fix CONFIG_TIMER_EARLY)
>>
>> Thanks,
>> Samuel
>>
>>> Probing device cpus with driver cpu_bus
>>> Probing device cpu at 0 with driver riscv_cpu
>>> Binding device sbi-sysreset to driver sbi-sysreset
>>> initcall: 000000004a012a98 (power_init_board)
>>> initcall: 000000004a012610 (init_func_watchdog_reset)
>>> initcall: 000000004a0129ca (initr_mmc)
>>> MMC:   Probing device mmc0-pins with driver pinconfig
>>> Probing device mmc at 4020000 with driver sunxi_mmc
>>> Probing device pinctrl at 2000000 with driver gpio_sunxi
>>> Probing device PF with driver gpio_sunxi
>>> Need to init timer!
>>> Probing device riscv_timer with driver riscv_timer
>>> Timer is 000000005df34f80
>>> Read timer, got 0x17dacfb
>>> Timer is 000000005df34f80
>>> Read timer, got 0x17f63b8
>>> Probing device mmc1-pins with driver pinconfig
>>> Probing device mmc at 4021000 with driver sunxi_mmc
>>> Timer is 000000005df34f80
>>> Read timer, got 0x186be7e
>>> Timer is 000000005df34f80
>>> Read timer, got 0x188752e
>>> mmc at 4020000: 0, mmc at 4021000: 1
>>> initcall: 000000004a012970 (relocated to 000000005ff63970)
>>> Loading Environment from nowhere... OK
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a012912 (relocated to 000000005ff63912)
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a01c50c (relocated to 000000005ff6d50c)
>>> initcall: 000000004a010938 (relocated to 000000005ff61938)
>>> initcall: 000000004a0188bc (relocated to 000000005ff698bc)
>>> In:    serial at 2500000
>>> Out:   serial at 2500000
>>> Err:   serial at 2500000
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a0012dc (relocated to 000000005ff522dc)
>>> initcall: 000000004a012950 (relocated to 000000005ff63950)
>>> initcall: 000000004a012610 (relocated to 000000005ff63610)
>>> initcall: 000000004a012934 (relocated to 000000005ff63934)
>>> Net:   phy interface8
>>> Probing device rgmii-pe-pins with driver pinconfig
>>> Probing device ethernet at 4500000 with driver eth_sun8i_emac
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1ae9538
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b04bed
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b202af
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b3b96e
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b5702c
>>> Timer is 000000005df34f80
>>> Read timer, got 0x1b726ec
>>>
>>> Warning: ethernet at 4500000 (eth0) using random MAC address -
>>> 62:69:97:68:19:04
>>> eth0: ethernet at 4500000
>>> initcall: 000000004a01292a (relocated to 000000005ff6392a)
>>> Hit any key to stop autoboot:  0
>>
>>
>