How to debug HW startup?

[Mauro Condarelli]

Hi Stefan,
I managed to find ONE of the reasons why my ROM build didn't run:
I forgot to enable `CONFIG_BOARD_EARLY_INIT_F=y`.
I wanted, nonetheless, be prepared for further mishaps, but
I have some other problems (see below).


On 1/10/20 2:33 PM, Stefan Roese wrote:
> Hi Mauro,
>
> (added Weijie to Cc)
>
> On 10.01.20 10:06, Mauro Condarelli wrote:
>>
>>
>> On 1/10/20 7:31 AM, Stefan Roese wrote:
>>> Hi Mauro,
>>>
>>> On 09.01.20 18:28, Mauro Condarelli wrote:
>>>> I managed to brick my target.
>>>>
>>>> Situation:
>>>> I have a board with a paleolithic (1.1.3) version of u-boot.
>>>> I had been testing by loading in ram from USB:
>>>>       usb reset; fatload usb 0 80010000 u-boot.bin; go 80010000
>>>> and everything was ok.
>>>> I changed a few settings (both defconfigs are attached below)
>>>> and tried "the real thing"
>>>> Unfortunately reflashing the actual boot produced a brick.
>>>> It does not utter a single byte.
>>>
>>> Ugh. Too bad.
>> I know... :(
>>  
>>>> I will have to reflash the original using an external apparatus
>>>> (which I don't have here, so I'll have to take target to another
>>>> location, probably tomorrow morning), but question is:
>>>> how do I debug such a situation?
>>>
>>> To debug very early problems, I suggest to use the DEBUG_UART interface
>>> in U-Boot. I also used it quite a lot before - also on this platform.
>>>
>>> Please see:
>>>
>>> include/debug_uart.h:
>>>
>>>      debug_uart_init();
>>>      printhex8(0x01);
>>>      ...
Could You share a Linkit  _defconfig with early serial debug enabled?
I'm decidedly missing something as, even enabling

CONFIG_DEBUG_UART=y
CONFIG_DEBUG_UART_BASE=0x10000e00
CONFIG_DEBUG_UART_CLOCK=20
CONFIG_DEBUG_UART_SHIFT=2
CONFIG_DEBUG_UART_ANNOUNCE=y

I still have plenty of errors:

/home/mcon/vocore/__V2__/Buildroot/recov/host/bin/mipsel-linux-ld.bfd:
arch/mips/cpu/start.o: in function `wr_done':
(.text+0x650): undefined reference to `debug_uart_init'
/home/mcon/vocore/__V2__/Buildroot/recov/host/bin/mipsel-linux-ld.bfd:
(.text+0x654): undefined reference to `debug_uart_init'
/home/mcon/vocore/__V2__/Buildroot/recov/host/bin/mipsel-linux-ld.bfd:
board/vocore/vocore2/built-in.o: in function `board_early_init_f':
(.text.board_early_init_f+0x10): undefined reference to `printhex8'
/home/mcon/vocore/__V2__/Buildroot/recov/host/bin/mipsel-linux-ld.bfd:
common/built-in.o: in function `putc':
(.text.putc+0x18): undefined reference to `printch'
/home/mcon/vocore/__V2__/Buildroot/recov/host/bin/mipsel-linux-ld.bfd:
common/built-in.o: in function `puts':
(.text.puts+0x2c): undefined reference to `printch'


>>>
>>> When using UART2 on the MT7628 please make sure to configure the pin
>>> mux before using the debug uart. Otherwise nothing will get printed.
>> I do *not* do anything explicitly in my code, but I have stanzas in .dts
>> Is that supposed to be enough? (I attach my current .dts as I'm sorry to
>> say I don't really fully grok .dts and I'm merely copying stanzas
>> around).
>>
>> I have Your code in board/vocore/vocore2/board.c (attached), shouldn't
>> that be enough?
>
> Yes, this should be enough when using UART2 IIRC.
>
> BTW: I just noticed that the VoCore uses the MT7628 and the LinkIt and
> the GARDENA boards both use the MT7688. I don't remember the differences
> but I don't think this should be a problem.
AFAIK differences between MT7628 and MT7688 are restricted to WiFi
radio section (MT7628 is 2r2t while MT7688 is 1r1t); any addendum to
this I would like to know.
 
>>> BTW: This might also be a problem on your board, if you use UART2 and
>>> the muxing is not done no output will occur.
>> I understand (see above).
>>
>>>> What could I have done so wrong?
>>>>
>>>> As You can see I changed only a few settings:
>>>>
>>>> --- configs/vocore_vocore2-ram_defconfig    2020-01-09
>>>> 16:11:12.568096050 +0100
>>>> +++ configs/vocore_vocore2_defconfig    2020-01-09 16:07:10.528267378
>>>> +0100
>>>> @@ -1,9 +1,12 @@
>>>>    CONFIG_MIPS=y
>>>> -CONFIG_SYS_TEXT_BASE=0x80010000
>>>> +CONFIG_SYS_TEXT_BASE=0x9c000000
>>>>    CONFIG_ENV_SIZE=0x00001000
>>>>    CONFIG_NR_DRAM_BANKS=1
>>>>    CONFIG_ARCH_MTMIPS=y
>>>>    CONFIG_BOARD_VOCORE2=y
>>>> +CONFIG_BOOT_ROM=y
>>>> +CONFIG_ONBOARD_DDR2_SIZE_1024MBIT=y
>>>> +CONFIG_ONBOARD_DDR2_CHIP_WIDTH_16BIT=y
>>>>    CONFIG_MIPS_BOOT_FDT=y
>>>>    CONFIG_ENV_VARS_UBOOT_CONFIG=y
>>>>    CONFIG_SYS_BOOT_GET_CMDLINE=y
>>>> ... in a way that's very similar to boards based on the same SoC
>>>> (linkit-smart-7688 and gardena-smart-gateway-mt7688).
>>>>
>>>> In the ancient u-boot I had to remove a header from the RAM
>>>> version, but this was not needed with current u-boot.
>>>>
>>>> Did I forget some step?
>>>
>>> Did you never program U-Boot into SPI NOR before on your VoCore2?
>> Yes.
>
> Autsch.
>
>> Up to now I've been using the RAM-version and loaded it from my old
>> (paleolithic 1.1.3) vendor-provided u-boot.
>> Note: I have modified and reflashed *that* u-boot several times, so I
>> was kind of confident I could do without much problem.
>> Of course it's fully possible some initialization done by old code is
>> missing in the new one.
>>
>>> Which binary did you program?
>> I flashed "u-boot.bin" which looks like a copy of "u-boot-dtb.bin";
>
> Yes, both images are identical.
>
>> this is exactly the same file I used for my RAM-based tests
>> (after switching _defconfig and recompiling, of course).
>
> Do you have a list of differences of the LinkIt MT7688 and the VoCore2
> board? Like DDR2 setup, UART etc?
>  
>>> How do the fist line look like? Here my output:
>>>
>>> $ hexdump -n 256 u-boot.bin
>>> 0000000 013f 1000 4800 4080 0000 0000 0000 0000
>>> 0000010 0000 0000 0000 0000 0000 0000 0000 0000
>>> *
>>> 0000100
>> Mine is quite similar:
>> $ hexdump -n 256 u-boot.bin
>> 0000000 013f 1000 4800 4080 0000 0000 0000 0000
>> 0000010 0000 0000 0000 0000 0000 0000 0000 0000
>> *
>> 0000100
>
> Ok, this is not a problem. The image is the "correct" one.
>  
>> I'm about to go where there's a nailbed to reflash SPI NOR
>> "from outside"; I plan to read back what's on flash before
>> putting back the "old" u-boot to see if something went wrong
>> while flashing (but I doubt it).
>>
>> Problem is how to proceed.
>> Old code did a lot of hard-coded initialization (non-DT-based)
>> which I don't (explicitly) do here (including a long RAM calibration
>> I didn't even try to understand).
>
> The RAM code is nearly identical. The current mainline code is "cloned"
> from the MediaTek (paleolithic) version. So I don't suspect a problem
> here. This will change btw with the new code from Weijie. He re-wrote
> the early init code including the DDR/DDR2 code and its calibration.
> I already tested this code on the Linkit board and it worked there
> without any issues.
>
> Still the current DDR2 code "should" work on your board as well.
>
>> I will bring back a certain number of working modules, so I will
>> have a certain number of "tries" before I need to go back for
>> hard reflashing; I should try to minimize commuting ;)
>
> Right. Again, I suggest to use the DEBUG UART for early debugging.
I'm having a few problems in that department...

> You might also want to give the new version / patches from Weijie
> a try. He added SPL support and a "cleaner" init code for this SoC.
I'm interested in giving it a spin (and help debugging on another HW,
if needed), but I would like to have a solid base from where to move,
changing too many things at once is rarely a good strategy ;)

> Perhaps this helps as well.
>
> Thanks,
> Stefan
>
Thanks
Mauro