[PATCH v3 1/2] mach-k3: add runtime memory carveouts for MMU table

[Kumar, Udit]

On 6/18/2025 5:22 PM, Anshul Dalal wrote:
> On Wed Jun 18, 2025 at 3:56 PM IST, Udit Kumar wrote:
>> On 6/17/2025 7:28 PM, Anshul Dalal wrote:
>>> In u-boot we only provide a single MMU table for all k3 platforms,
>>> this does not scale for devices with reserved memory outside the range
>>> 0x9e780000 - 0xa0000000 (eg j722s[1]) or for devices with < 2GiB of
>>> memory (eg am62-SIP with 512MiB of RAM).
>> please avoid giving reference to internal HACKs
>>
> Got it :)
>
>>> To properly configure the MMU on various k3 platforms, the
>>> reserved-memory regions need to be queried at runtime from the
>>> device-tree and the MMU table should be updated accordingly.
>>>
>>> This patch adds the required fixups to the MMU table (during proper
>>> U-boot stage) by marking the reserved regions as non cacheable and
>>> keeping the remaining area as cacheable.
>>>
>>> For the A-core SPL, the 128MiB region starting from SPL_TEXT_BASE
>>> is marked as cacheable i.e 0x80080000 to 0x88080000.
>>>
>>> The 128MiB size is chosen to allow for future use cases such as falcon
>>> boot from the A-Core SPL which would require loading kernel image from
>>> the SPL stage. This change also ensures the reserved memory regions that
>>> all exist past 0x88080000 are non cacheable preventing speculative
>>> accesses to those addresses.
>>>
>>> [1]:
>>> https://git.ti.com/cgit/ti-u-boot/ti-u-boot/tree/arch/arm/mach-k3/arm64/arm64-mmu.c?h=ti-u-boot-2025.01-next#n54
>>>
>>> Signed-off-by: Anshul Dalal <anshuld at ti.com>
>>> ---
>>> Changes for v3:
>>>    - Remove unused memory regions in SPL's map
>>>    - Add runtime addition of MMU entry for the framebuffer in SPL
>>>    - Refactor k3_mem_map_init to use standard u-boot APIs
>>>    - Unmap reserved-memory regions instead of keeping them uncached
>>> v2: https://lore.kernel.org/u-boot/20250610160833.1705534-1-anshuld@ti.com/
>>>
>>> Changes in v2:
>>> - Removed dependency to:
>>>     https://lore.kernel.org/u-boot/20250522150941.563959-1-anshuld@ti.com/
>>>
>>> v1: https://lore.kernel.org/u-boot/20250602120054.1466951-1-anshuld@ti.com/
>>> ---
>>>    arch/arm/mach-k3/arm64/arm64-mmu.c     | 191 +++++++++++++++++++++++--
>>>    arch/arm/mach-k3/include/mach/k3-ddr.h |   9 ++
>>>    board/ti/common/k3-ddr.c               |  10 ++
>>>    3 files changed, 197 insertions(+), 13 deletions(-)
>>>
> [snip]
>
>>> +int k3_mem_map_init(void)
>>> +{
>>> +	fdt_addr_t mem_base;
>>> +	fdt_size_t mem_size;
>>> +	struct fdt_resource dt_reserved[K3_MMU_REGIONS_COUNT],
>>> +		coalesced[K3_MMU_REGIONS_COUNT];
>>> +	int k3_map_idx = -EINVAL, ret, nodeoffset, subnode;
>>> +	void *blob = (void *)gd->fdt_blob;
>>> +	unsigned int carveout_len, i, j;
>>> +
>>> +	ret = fdt_fixup_reserved(blob, "tfa", CONFIG_K3_ATF_LOAD_ADDR, 0x80000);
>>> +	if (ret) {
>>> +		pr_err("%s: Failed to fixup reserved node for tfa [%d]\n",
>>> +		       __func__, ret);
>>> +		return ret;
>>> +	}
>> [..]+
>> +	k3_mem_map[k3_map_idx] = (const struct mm_region){ 0 };
>> I assume you are looping over reserved -memory and adding to map..
>>
>> what if , we need some reserved memory as non-cached.
>>
>>
> Yes, here I loop over all the reserved regions and add the carveouts to
> the map after coalescing. So far the reserved regions are kept unmapped
> but that causes issues with remoteproc, I'll fix that in the next
> revision by marking them as non-cacheable instead.

you should check wrt device capability, If device is coherent then it 
can be as is

If device is non-coherent (we observed in J722s, still to confirm this) 
    then node should be

non-cacheable


>
>>> +
>>> +	return 0;
>>> +}
>>> diff --git a/arch/arm/mach-k3/include/mach/k3-ddr.h b/arch/arm/mach-k3/include/mach/k3-ddr.h
>>> index 39e6725bb9b..0b164ebf5e6 100644
>>> --- a/arch/arm/mach-k3/include/mach/k3-ddr.h
>>> +++ b/arch/arm/mach-k3/include/mach/k3-ddr.h
>>> @@ -8,10 +8,19 @@
>>>    
>>>    #include <spl.h>
>>>    
>>> +/* Number of mappable regions in the MMU page table */
>>> +#define K3_MMU_REGIONS_COUNT 32
>>> +
>> why 32 ?
> I just took it as a worst case scenario with assuming > 10
> reserved-memory nodes with no coalescing and two entries per node (one
> for the carveout and another for the non-cached entry as per above).
> The table size is 1KiB at the moment and I think it's a reasonable
> compromise.
>
>>>    int dram_init(void);
>>>    int dram_init_banksize(void);
>>>    
>>>    void fixup_ddr_driver_for_ecc(struct spl_image_info *spl_image);
>>>    void fixup_memory_node(struct spl_image_info *spl_image);
>>>    
>>> +/*
>>> + * Modifies the MMU memory map based on DDR size and reserved-memory
>>> + * nodes in DT
>>> + */
>>> +int k3_mem_map_init(void);
>>> +
>>>    #endif /* _K3_DDR_H_ */
>>> diff --git a/board/ti/common/k3-ddr.c b/board/ti/common/k3-ddr.c
>>> index a8425da8de5..ee882f62109 100644
>>> --- a/board/ti/common/k3-ddr.c
>>> +++ b/board/ti/common/k3-ddr.c
>>> @@ -7,6 +7,7 @@
>>>    #include <dm/uclass.h>
>>>    #include <k3-ddrss.h>
>>>    #include <spl.h>
>>> +#include <mach/k3-ddr.h>
>>>    
>>>    #include "k3-ddr.h"
>>>    
>>> @@ -14,6 +15,15 @@ int dram_init(void)
>>>    {
>>>    	s32 ret;
>>>    
>>> +	if (IS_ENABLED(CONFIG_ARM64) && xpl_phase() != PHASE_SPL) {
>>> +		ret = k3_mem_map_init();
>>> +		if (ret) {
>>> +			printf("%s: Error fixing up MMU memory map: %d\n",
>>> +			       __func__, ret);
>>> +			return ret;
>> If can not set MMU table, then should be Hang , no ?
> That I have deferred to the caller which is initcall in our case which
> will hang the board if dram_init fails.

ok


>
>>> +		}
>>> +	}
>>> +
>>>    	ret = fdtdec_setup_mem_size_base_lowest();
>>>    	if (ret)
>>>    		printf("Error setting up mem size and base. %d\n", ret);