[U-Boot] setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?

Thu Sep 7 06:55:22 UTC 2017

On Thu, 2017-09-07 at 06:45 +0000, Mingkai Hu wrote:
> > -----Original Message-----
> > From: Mingkai Hu
> > Sent: Wednesday, September 06, 2017 5:37 PM
> > To: 'Joakim Tjernlund' <Joakim.Tjernlund at infinera.com>; Roy Zang
> > <roy.zang at nxp.com>; York Sun <york.sun at nxp.com>
> > Cc: u-boot at lists.denx.de
> > Subject: RE: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: Joakim Tjernlund [mailto:Joakim.Tjernlund at infinera.com]
> > > Sent: Tuesday, September 05, 2017 8:45 PM
> > > To: Mingkai Hu <mingkai.hu at nxp.com>; Roy Zang <roy.zang at nxp.com>;
> > 
> > York
> > > Sun <york.sun at nxp.com>
> > > Cc: u-boot at lists.denx.de
> > > Subject: Re: setup of PEX_GCLK_RATIO in E500 CPUs(P2010) missing ?
> > > 
> > > On Wed, 2017-08-30 at 15:25 +0000, York Sun wrote:
> > > > On 08/30/2017 06:05 AM, Joakim Tjernlund wrote:
> > > > > On Tue, 2017-08-29 at 17:33 +0000, York Sun wrote:
> > > > > > +Roy Zang to comment on PCIe clock source
> > > > > > 
> > > > > > On 08/29/2017 10:06 AM, Joakim Tjernlund wrote:
> > > > > > > On Tue, 2017-08-29 at 15:43 +0000, York Sun wrote:
> > > > > > > > On 08/29/2017 06:21 AM, Joakim Tjernlund wrote:
> > > > > > > > > On Tue, 2017-08-29 at 12:47 +0200, Joakim Tjernlund wrote:
> > > > > > > > > > As we are looking at PCI stuff ATM I would like to ask
> > > > > > > > > > about PEX_GCLK_RATIO in E500 CPUs. I cannot find this is
> > > > > > > > > > setup at all for E500 but I THINK this is required.
> > > > > > > > > > 
> > > > > > > > > > In 83xx one do:
> > > > > > > > > > get_clocks();
> > > > > > > > > > /* Configure the PCIE controller core clock ratio */
> > > > > > > > > > out_le32(hose_cfg_base + PEX_GCLK_RATIO,
> > > > > > > > > 
> > > > > > > > > A bit strange with out_le32 instead of out_be32 ?
> > > > > > > > > 
> > > > > > > > > > (((bus ? gd->arch.pciexp2_clk : gd->arch.pciexp1_clk) /
> > > > > > > > > > 1000000) * 16) / 333); udelay(1000000);
> > > > > > > > > > 
> > > > > > > > > > Any clues?
> > > > > > > > > > 
> > > > > > > > > >     Jocke
> > > > > > > > > 
> > > > > > > > > Seems like only 83xx is setting this parameter.
> > > > > > > > > 
> > > > > > > > > Anyhow, I put together this patch:
> > > > > > > > > 
> > > > > > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > > > > > @@ -322,6 +322,10 @@ void fsl_pci_init(struct
> > > > > > > > > pci_controller *hose, struct fsl_pci_info *pci_info)
> > > > > > > > > 
> > > > > > > > >            pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > > > > > 
> > > > > > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > > > > > +       pci_hose_write_config_dword(hose, dev, 0x440, (gd-
> > > > 
> > > > bus_clk / 333333333) * 16);
> > > > > > > > > +       /* udelay(1000000) needed here ?*/
> > > > > > > > > +
> > > > > > > > >            block_rev = in_be32(&pci->block_rev1);
> > > > > > > > >            if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > > > > > >                    pi = &pci->pit[2];      /* 0xDC0 */
> > > > > > > > > 
> > > > > > > > > Any chance this will work for all supported FSL PCIe controllers?
> > > > > > > > > 
> > > > > > > > 
> > > > > > > > Jocke,
> > > > > > > > 
> > > > > > > > You don't need to program this register if the actual PCIe
> > > > > > > > clock is the same as default. Since SerDes reference clock
> > > > > > > > has to be 100MHz for PCIe protocol, my guess is the PCIe
> > > > > > > > clock is always correct. The bus_clk you are referring is
> > > > > > > > not the PCIe clock. Again, I am not a PCIe expert, I could
> > > > > > > > be wrong. Since PCIe (SerDes) has been working on multiple
> > > > > > > > platforms, I guess the
> > > 
> > > clock is correct.
> > > > > > > 
> > > > > > > I don't think so. Here is what T1042 says about this:
> > > > > > > 
> > > > > > > The PCI Express controller clock frequency is one-half the
> > > > > > > platform
> > > 
> > > clock frequency.
> > > > > > > 
> > > > > > > The PCI Express controller core clock ratio register is used
> > > > > > > to program the ratio of the actual PCI Express controller
> > > > > > > clock frequency to the default controller core frequency ( 333 MHz ).
> > > > > > > This is required only when a PCI Express controller clock
> > > > > > > frequency
> > > 
> > > other than the default 333 MHz has to be used.
> > > > > > > As an example of programming PEX_GCLK_RATIO, consider the case
> > > > > > > where the actual PCI Express controller clock is 250 MHz, the
> > > > > > > ratio of the actual clock to the default clock ( 333 MHz) is
> > > > > > > 3:4. that is, the default core clock has to be multiplied by
> > > > > > > the ratio (3/4, which is equivalent to 12/16). So the register
> > > > > > > has to be
> > > 
> > > programmed with the decimal numerator value 12 or 0x0000_000C.
> > > > > > > 
> > > > > > > Our CCB is 250 MHz so this should be set to 0xc.
> > > > > > > 
> > > > > > > I found this on Google too:
> > > > > > > The PEX controller hardware requires for timing tuning in
> > > > > > > order to operate properly at given CCB/2 clock frequency.
> > > > > > > PEX_GCLK_RATIO register controls this tuning with 333MHz/16
> > > > > > > granularities. So for given CCB frequency Fccb [MHz] we must
> > > > > > > write to
> > > 
> > > PEX_GCLK_RATIO nearest integer of ((Fccb/2)/(333/16)). So for Fccb=375
> > > we have to write 18.
> > > > > > 
> > > > > > Jocke,
> > > > > > 
> > > > > > Basically I agree with you, if the clock is different from
> > > > > > default, you need to set the ratio. T104x indeed uses 1/2
> > > > > > platform clock as PCIe clock. I am not sure if P2010 uses the
> > > > > > same way. Let me add Roy to this thread.
> > > > > > 
> > > > > > Roy, how do we get PCIe clock for P1/T1 (and other mpc85xx) SoCs?
> > > > > > Don't we need to set PEX_GCLK_RATIO register? It is not even in
> > > > > > the
> > > 
> > > header file.
> > > > > 
> > > > > Patch was off, this works:
> > > > > diff --git a/drivers/pci/fsl_pci_init.c
> > > > > b/drivers/pci/fsl_pci_init.c index 52792dcd59..e7f2499784 100644
> > > > > --- a/drivers/pci/fsl_pci_init.c
> > > > > +++ b/drivers/pci/fsl_pci_init.c
> > > > > @@ -322,6 +322,11 @@ void fsl_pci_init(struct pci_controller
> > > > > *hose, struct fsl_pci_info *pci_info)
> > > > > 
> > > > >          pci_setup_indirect(hose, cfg_addr, cfg_data);
> > > > > 
> > > > > +       /* Configure the PCIE controller core clock ratio */
> > > > > +       pci_hose_write_config_dword(hose, dev, 0x440,
> > > > 
> > > > pci_hose_write_config_dword() writes to config space, doesn't it?
> > > > 0x440 is not in config space.
> > > > 
> > > > > +                                   ((gd->bus_clk / 1000000) * 16/2) / 333);
> > > > > +       /* udelay(1000000) needed here ?*/
> > > > 
> > > > Let us not to add delay if we don't know yet.
> > > > 
> > > > > +
> > > > >          block_rev = in_be32(&pci->block_rev1);
> > > > >          if (PEX_IP_BLK_REV_2_2 <= block_rev) {
> > > > >                  pi = &pci->pit[2];      /* 0xDC0 */
> > > > > 
> > > > > 
> > > > > Any news on this ?
> > > > > I know both T1042 and P2010 has the same setting/formula, a good
> > > > > start :)
> > > > 
> > > > +Mingkai to comment on PCIe clock
> > > > 
> > > > York
> > 
> > This is the clock to PCIe controller which is not different from the clock to
> > PCIe SerDes lane.
> > My understanding is that it should be configured according to the platform
> > frequency.
> > But there is no specific and consistent description for all the platforms. I am
> > checking with the design team to see if there is hardware fixed clock to PCIe
> > controller for the platforms which don't have clear description. Will keep you
> > all updated.
> > 
> 
> This ratio will be used by GPEX only to calculate the actual timer values to be used for Ack Latency and
> Replay timeout values and the FC update timer value under traffic condition. It's applicable for all PowerPC
> controllers except T4 and T2 family, so would you please help to update the patch to set the register
> according to platform clock (or platform/2, need to check chip RM) excluding T4/T2 platform?

I think NXP is in a better position to adjust the patch w.r.t actual PCIe clock. I just
want to known, for now, what PCIe clock is for T1042 and P2010, CCB or CCB/2, so we can set
the correct value and continue with our trouble shooting.

T1040RM actually has this writing in 28.11.19:
The PCI Express controller clock frequency is one-half the platform clock frequency.
....
That indicates the PCIe clock is CCB/2. P2010 does not have that text. 
Please advice.

 Jocke