[PATCH v2 3/7] riscv: Use a valid bit to ignore already-pending IPIs

[Leo Liang]

On Mon, Sep 14, 2020 at 10:22:59AM -0400, Sean Anderson wrote:
> Some IPIs may already be pending when U-Boot is started. This could be a
> problem if a secondary hart tries to handle an IPI before the boot hart has
> initialized the IPI device.
> 
> This commit introduces a valid bit so secondary harts know when and IPI
> originates from U-boot, and it is safe to use the IPI API. The valid bit is
> initialized to 0 by board_init_f_init_reserve. Before this, secondary harts
> wait in wait_for_gd_init.
> 
> Signed-off-by: Sean Anderson <seanga2 at gmail.com>
> Reviewed-by: Bin Meng <bin.meng at windriver.com>
> Reviewed-by: Rick Chen <rick at andestech.com>
> ---
> 
> Changes in v2:
> - Use a valid bit instead of addr to validate IPIs
> 
>  arch/riscv/include/asm/smp.h |  7 +++++++
>  arch/riscv/lib/smp.c         | 16 ++++++++++++++--
>  2 files changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
> index 1b428856b2..2dae0800ce 100644
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -18,14 +18,21 @@
>   * IPI data structure. The hart ID is inserted by the hart handling the IPI and
>   * calling the function.
>   *
> + * @valid is used to determine whether a sent IPI originated from U-Boot. It is
> + * initialized to zero by board_init_f_alloc_reserve. When U-Boot sends its
> + * first IPI, it is set to 1. This prevents already-pending IPIs not sent by
> + * U-Boot from being taken.
> + *
>   * @addr: Address of function
>   * @arg0: First argument of function
>   * @arg1: Second argument of function
> + * @valid: Whether this IPI is valid
>   */
>  struct ipi_data {
>  	ulong addr;
>  	ulong arg0;
>  	ulong arg1;
> +	unsigned int valid;
>  };
>  
>  /**
> diff --git a/arch/riscv/lib/smp.c b/arch/riscv/lib/smp.c
> index ab6d8bd7fa..8f33ce1fe3 100644
> --- a/arch/riscv/lib/smp.c
> +++ b/arch/riscv/lib/smp.c
> @@ -54,7 +54,13 @@ static int send_ipi_many(struct ipi_data *ipi, int wait)
>  		gd->arch.ipi[reg].arg0 = ipi->arg0;
>  		gd->arch.ipi[reg].arg1 = ipi->arg1;
>  
> -		__smp_mb();
> +		/*
> +		 * Ensure valid only becomes set when the IPI parameters are
> +		 * set. An IPI may already be pending on other harts, so we
> +		 * need a way to signal that the IPI device has been
> +		 * initialized, and that it is ok to call the function.
> +		 */
> +		__smp_store_release(&gd->arch.ipi[reg].valid, 1);
>  
>  		ret = riscv_send_ipi(reg);
>  		if (ret) {
> @@ -83,7 +89,13 @@ void handle_ipi(ulong hart)
>  	if (hart >= CONFIG_NR_CPUS)
>  		return;
>  
> -	__smp_mb();
> +	/*
> +	 * If valid is not set, then U-Boot has not requested the IPI. The
> +	 * IPI device may not be initialized, so all we can do is wait for
> +	 * U-Boot to initialize it and send an IPI
> +	 */
> +	if (!__smp_load_acquire(&gd->arch.ipi[hart].valid))
> +		return;
>  
>  	smp_function = (void (*)(ulong, ulong, ulong))gd->arch.ipi[hart].addr;
>  	invalidate_icache_all();

Reviewed-by: Leo Liang <ycliang at andestech.com>