Zynq MPSoC Dynamic DDR DIMM Configuration support

[Jorge Ramirez-Ortiz, Foundries]

On 13/05/21, Michal Simek wrote:
> Hi,
> 
> On 5/13/21 9:24 AM, Jorge Ramirez-Ortiz, Foundries wrote:
> > On 13/05/21, Michal Simek wrote:
> >> Hi Jorge,
> >>
> >> On 5/12/21 11:21 PM, Jorge Ramirez-Ortiz, Foundries wrote:
> >>> Hi Michal
> >>>
> >>> We are doing some work on an MPSoC UZ3EG platform part of which
> >>> requires us to replace FSBL with SPL.
> >>
> >> Just for curiosity what's the reason for this requirement that fsbl is
> >> not enough?
> > 
> > we prefer to have a common boot strategy on all the boards we support
> > (whether it is from NXP, Xilinx, ST or TI); it just simplifies the OTA
> > process (including their firmwares); and of course our meta layers.
> > 
> >>
> >>>
> >>> It seems the actual boot process is becoming an issue on these SoCs;
> >>> currently, 1) we embed the PMU firmware on SPL so the bootrom can
> >>> extract it and program it;
> >>
> >> Actually it is not working like this. PMU FW is own part of boot.bin and
> >> it is not embed in SPL.
> > 
> > yes, absolutely (boot.bin is what we have been flashing), holding SPL and FW 
> > 
> >>
> >>
> >>> 2) then SPL configures the PMU using a
> >>> platform specific binary that gets built also with SPL; 
> >>
> >> PMU cfg object.
> > 
> > yep, pm_cfg_obj.c
> > 
> >>
> >>
> >> and finally,
> >>> 3) SPL sets up the DDR using its psu_init_gpl.c settings (also board
> >>> specific, part of the XSA).
> >>
> >> yes.
> >>
> >>>
> >>> It is this final step in the boot sequence that is being broken by the
> >>> Dynamic DDR DIMM configuration feature [1]
> >>>
> >>> [1] https://www.xilinx.com/support/answers/75768.html
> >>
> >> This was developed for zcu102 and maybe others boards which have DIMM
> >> modules where origin part were EOL.
> > 
> > right.
> > 
> > but notice that there is also support on u-boot for
> > altera/imx/marvell/microchip; so just wondering if we should add
> > drivers/ddr/xilinx to this list.  
> 
> Also SPD code is around too.
> 
> 
> 
> >>
> >>
> >>> Are you aware of any work in progress to support this? Any thoughts on
> >>> how to work around it and train the DDR? will the functionality
> >>> required to implmenet Dynamic DDR DIMM configuration be added as a
> >>> separate file to the XSA tarball or will we need to do some native
> >>> implementation in SPL?
> >>
> >> I am not aware about any work on SPL side to support this. IIRC FSBL
> >> didn't have generic DDR configuration. It was only by reading SPD and
> >> aligned some parameters but it is quite a long time I have looked at it
> >> last time.
> > 
> > ok
> > 
> >>
> >>
> >>> Becase without a change in the last link in the process chain
> >>> described earlier (calls to psu_init()), DDR just wont be accessible
> >>> to U-BOOT or OP-TEE.
> >>>
> >>> In our case, we were able to boot from QSPI, boot SPL (in OCM), have
> >>> SPL unpack and validate the FIT image, execute TF-A(in OCM), but then
> >>> any jumps to OP-TEE or U-BOOT would obviously not progress since the
> >>> DDR wasnt properly trained/initialized.
> >>>
> >>> so, any thoughts or plans you can share?
> >>
> >> The question is why you need this feature to be there. It is not working
> >> for every DIMM module. And normally if you have custom boards you need
> >> just one DDR configuration (or limited number based on HW versions) and
> >> for that there is really no need to waste your boot up time on it.
> > 
> > not sure what you mean. we need this feature because it adds the
> > expected flexibility to the bootloader. Sure, we can hardcode DDR
> > configurations but why should we when it can be resolved by
> > software. Am I missing your point?
> 
> Ok. One is support for evaluation boards and the second for a product.
> In product you don't need this because I would say most of projects are
> not using DIMMs.
> If you want to support generic evaluations boards then of course it can
> be done.
> 
> > 
> >> You can add multiple configurations to psu_init_gpl() and based on any
> >> information (board rev/pins/etc) decided which one should be used.
> > 
> > so what you are suggesting is that we customize psu_init_gpl() to the
> > target (ie, have an updated xsa file with whatever config we need for
> > this system).  
> 
> In products yes.
> 
> > 
> > what I fail to grasp is why we can't take a step forward and do what
> > we do for other architectures in u-boot. and what fsbl already does by
> > config.
> > 
> > I mean, why not? do you foresee any integration issues with the
> > current bootflow?
> 
> Not really if you are willing to write that code just go for it.
> I think it is definitely feasible to do so but I haven't seen any
> algorithm to do it in generic way even there must be one behind in
> xilinx tools. There are a lot of types of memories which you can
> configure in xilinx design tools but it won't be any problem to limit it
> to some.

ok. unfortunately I am not sure when - we have tight deadlines on this
product so we will just update psu_init().

In fact, and this is relevant as well to keep in mind for a future
TO-DO list, the issue we were having was not due to the dynamic
configuration of DDR but to the fact that the DDR on this product is
ECC (and ECC, when enabled in the DDR controller must be initialized
_after_ psu_init() has run - which is something that FSBL does but SPL
doesnt).

So we will just disable ECC for the time being and will send a patch
once we come around to do it (or with a bit of luck someone will beat
me to it and I will just pick it up from the next u-boot release)

> 
> Thanks,
> Michal
>