[U-Boot] [linux-sunxi] A10 bring up and DRAM configuration procedure

Mon Apr 4 14:05:32 CEST 2016

On Sun, Apr 03, 2016 at 07:11:21PM +0300, Siarhei Siamashka wrote:
> Hi Piotr,

Hi Siarhei,
thanks for comprehensive reply.

> > Other question is how to proceed from this point ? I have datasheet for DDR3,
> > but number of possible parameters to tweak seems to be overwhelming. Also as
> > sunxi wiki claim timing from datasheet have to be converted to number of cycles
> > - I'm not sure how to approach that. Is there any guide where to start ?
> 
> Well, the standard JEDEC timings are already tabulated in 
> 
>     http://git.denx.de/?p=u-boot.git;a=blob;f=board/sunxi/dram_timings_sun4i.h
> 
> Some DRAM chips may have better timings than mandated by the standard,
> but this is more like performance tuning. Either way, these timings are
> a "digital" thing, they specify how many cycles to wait between sending
> different commands to the DRAM chip. And they are rather unlikely to be
> related to your reliability problems.
> 
> It is much more likely that you have "analog" problems and the signal
> gets distorted too badly on the way from the DRAM controller in the
> SoC to the DRAM chips. From what I heard, the DDR3 tracks routing is a
> major PITA, one needs to get both the tracks length and the tracks
> impedance matched within certain tolerance levels. As you are dealing
> with a custom hardware design, then maybe the DDR3 routing is a little
> bit screwed up?

You're right. I think this is the reason, because of that first thing I
tried when I saw corrupted serial output was decreasing DRAM_CLK.

> 
> The DRAM controller configuration registers allow to configure the
> delays and drive/termination impedance to correct the PCB imperfection
> to some extent. In practice, all the "professionally" designed Allwinner
> devices work with the same generic Allwinner blessed DRAM settings,
> because the PCB routing is apparently usually done and tested to work
> fine with these default settings. Also reducing the DRAM clock speed
> usually improves reliability. It was believed that any A10/A13/A20
> device should work stable if we clock the DRAM down to 360MHz.
> 
> On the hardware side, you can maybe check the ZQ and RZQ resistors
> (one connected to the SoC and also one extra resistor for each DRAM
> chip). They should at least exist and be preferably 240 ohm. Some board
> manufacturers tune the nominal value of these resistors, apparently
> as a way to fine tune the reliability. That's the configuration knob
> that is easily available to board manufacturers (they don't dare to
> touch the DRAM settings in U-boot and probably don't know how to do
> this).

I have schematics and confirmed that there is 240 ohm (%1 tolerance)
resistor although I cannot be sure about its quality. So there are
resistors on ZQ from DRAM die an SZQ (AA7) from SoC side. If I will get
to wall I will probably check if those have good enough quality and
maybe replace it. Is there any guide what can be used or I should just
try slightly higher/lower and see if situation change ?

> 
> One more thing is the VDD-DLL voltage. It is served from DCDC3 (AXP209
> PMIC) and is usually set to 1.250V by default. Increasing this voltage
> usually improves the DRAM reliability and allows it to be clocked
> higher. You can try to increase this voltage to 1.300V to check
> if this helps. Don't go beyond 1.325V on Allwinner A10 though.
> 
> > sunxi wiki describe calibration, when it is possible to run system, but what
> > can be done in situation when U-Boot cannot reach that state ?
> 
> The whole idea of this calibration process is to find good ZQ and TPR3
> settings. The settings are assumed to be good if the board is able to
> pass the lima-memtester reliability test at a higher DRAM clock speed
> without failing. We exploit the fact that lima-memster is by far more
> sensitive to DRAM misconfiguration than the usual software. So the
> borderline unreliable DRAM settings may not exhibit any visible bad
> effects (the system is able to boot, the usual linux software works
> fine too), but easily detected by lima-memtester. Because the system
> is sufficiently usable at such borderline unstable DRAM settings, we
> can run automation scripts to do a bruteforce search, which is probing
> different TPR3 values without human supervision. That's what is
> explained at:
> 
>     https://linux-sunxi.org/A10_DRAM_Controller_Calibration
> 
> But in your case, you can try different ZQ and TPR3 settings manually
> in the U-Boot defconfig and instead of running lima-memtester (which
> first needs a bootable userland) just monitor how far the system is
> able to boot as an alternative selection criteria for picking better
> configuration values. If you manage to boot the system up to a
> semi-working userland, then you can switch to using lima-memtester
> for the final tuning and validation.

I applied above suggestions those were very helpful. In addition, for
further reference, to tweak ZQ for A10 configuration requires enabling
ODT. So I applied those options:

CONFIG_AXP_DCDC3_VOLT=1300
CONFIG_DRAM_ZQ=123
CONFIG_DRAM_ODT_EN=y

And now I sporadically can start kernel. Unfortunately sunxi-3.4 crash
on:

mm/page_alloc.c:2317 __alloc_pages_nodemask+0x374/0x7b8()

and there are also couple BUGs:

BUG: Bad page state in process swapper  pfn:4ebb7

I wonder if tweaking ZQ can help with that. I will try and let you know.
If you have any suggestions related to above

> 
> > If anyone more experienced can advise what can be done at this point I would
> > appreciate.
> 
> Maybe ask the hardware people if they have done the DDR3 routing right?
> Also if it is just a single board, then it might be not misconfigured,
> but defective. Trying another sample could be a good idea.

It would be hard to verify hardware people answer. I think all points to
wrong DRAM wiring. Hardware people obviously didn't have previous
experience with Allwinner (ie. IDDET for USB OTG was not connected and
other minor problems). Are there any guides for DRAM wiring for
Allwinner ?

I have 2 boards and tried both although I need more statistics to gain
confidence in results.

> 
> > [1] http://www.micron.com/~/media/Documents/Products/Data%20Sheet/DRAM/DDR3/1Gb_DDR3_SDRAM.pdf
> > [2] http://git.denx.de/?p=u-boot.git;a=blob;f=lib/crc32.c;h=97592124867abb815d576899f8789545c3aef1aa;hb=HEAD#l200
> > [3] https://gist.github.com/pietrushnic/ea41a4ae38b7af8a6fa456113ce19af8
> > [4] http://www.denx.de/wiki/view/DULG/UBootCrashAfterRelocation
> > [5] https://gist.github.com/pietrushnic/d6a3d6c0b4a20a3226dfb67c4d129142
> 
> -- 
> Best regards,
> Siarhei Siamashka

-- 
Best Regards,
Piotr Król
Embedded Systems Consultant
http://3mdeb.com | @3mdeb_com