PFE subsystem bring up on S32G2

[aaronbaranoff]

I am looking for a simple example of how to power up the PFE sub system and setting up the appropriate clocks. I have a driver that I have written that should work on a native A53 (no Linux).  The clocks never come up. I know that prior to me working on this logic I could not get the PFE registers to even read now they read but not everything is working and I have traced it down to this.

Here is the basic code to init the subsystem (this is very fragmented and not clearly explain by NXP) so any source example would be appreciated.

pfe_init(void)

{

    /* Enable */

    /* Setup DMA coherency */

    volatile uint32_t tries;

    RDC.RD2_CTRL_REG.B.RD2_CTRL_UNLOCK = 1;

    RDC.RD2_CTRL_REG.B.RD2_INTERCONNECT_INTERFACE_DISABLE = 0;

    tries = 0;

    while (RDC.RD2_STAT_REG.B.RD2_INTERCONNECT_INTERFACE_DISABLE_REQ_ACK_STAT != 0 )

    {

        tries++;

        if (tries > MAX_TRIES)

        {

            break;

        }

    }

    tries = 0;

    while (RDC.RD2_STAT_REG.B.RD2_INTERCONNECT_INTERFACE_DISABLE_STAT != 0)

    {

        tries++;

        if (tries > MAX_TRIES)

        {

            break;

        }

    }

    /**((volatile unsigned int *)0x4007ca00) = 0x3f; */

    S32G_GPR.PFE_COH_EN.B.DDR = true;

    S32G_GPR.PFE_COH_EN.B.HIF0 = true;

    S32G_GPR.PFE_COH_EN.B.HIF1 = true;

    S32G_GPR.PFE_COH_EN.B.HIF2 = true;

    S32G_GPR.PFE_COH_EN.B.HIF3 = true;

    S32G_GPR.PFE_COH_EN.B.UTIL = true;

    /* Setup EMAC modes -RGMII (ALL PFES)*/

    S32G_GPR.PFE_EMACX_INTF_SEL.B.EMAC0 = 2;

    S32G_GPR.PFE_EMACX_INTF_SEL.B.EMAC1 = 2;

    S32G_GPR.PFE_EMACX_INTF_SEL.B.EMAC2 = 2;

    /* Enable power for PFE */

    S32G_GPR.PFE_PWR_CTRL.B.PWRACK = 0;

    S32G_GPR.PFE_PWR_CTRL.B.PWRCLAMP = 0;

    S32G_GPR.PFE_PWR_CTRL.B.PWRDWN = 0;

    S32G_GPR.PFE_PWR_CTRL.B.PWRISO = 0;

    S32G_GPR.GENCTRL1.B.CTRL = 1;

    /* Setup & Enable the PFE Sys Clock */

    /* Setup and Enable the PFE PE Clock */

    S32G_GPR.LLCE_CTRL.B.EXEVENT = 0;

    S32G_GPR.LLCE_CTRL.B.LINDIVSEL = 0;

    S32G_GPR.LLCE_CTRL.B.LLCE_HTM0_TRC_DIS = 0;

    S32G_GPR.LLCE_CTRL.B.LLCE_HTM1_TRC_DIS = 0;

    S32G_GPR.LLCE_CTRL.B.LLCE_LSPI_TRIG_IN = 0;

    S32G_GPR.LLCE_CTRL.B.PFE_2_LLCE_INT_HS_BP = 0;

    S32G_GPR.GENCTRL1.B.CTRL = 1;

    /* Setup EMAC Tx/RX clocks */

    S32G_GPR.GENCTRL0.B.ACCEL_FLXNC_RD0_IDLEREQ = 0; /* normal operation */

    S32G_GPR.GENCTRL0.B.ACCEL_FLXNC_REF_DIV = 0; /* Divide by 2 */

    S32G_GPR.GENCTRL0.B.USBPWRFLT = 0;    

    tries = 0;    

    do {

        MC_ME.PRTN2_COFB0_CLKEN.B.REQ0 = 1;

        MC_ME.PRTN2_COFB0_CLKEN.B.REQ1 = 1;

        MC_ME.PRTN2_COFB0_CLKEN.B.REQ2 = 1;

        MC_ME.PRTN2_COFB0_CLKEN.B.REQ3 = 1;

        Write_Control_Key();

        MC_ME.PRTN2_PCONF.B.OSSE = 1;

        MC_ME.PRTN2_PCONF.B.PCE = 1;

        MC_ME.PRTN2_PUPD.B.OSSUD = 1;

        MC_ME.PRTN2_PUPD.B.PCUD = 1;

        /* Tell hardware to check above registers for changes */

        Write_Control_Key();

        tries++;

        if (tries > MAX_TRIES)

        {

            break; /* code dies here - the clocks never get enabled
                      therefore it does not come out of reset */

        }

    } while (MC_ME.PRTN2_COFB0_STAT.R != 0xf);

    /* Put PFE in reset */

    MC_RGM.PRST_0[2].PRST_0.B.PERIPH_0_RST = 1;

    tries = 0;

    while (MC_RGM.PSTAT_0[2].PSTAT_0.B.PERIPH_0_STAT != 1)

    {

        tries++;

        if (tries > MAX_TRIES)

        {

            break;

        }

    }    

    tries = 0;

    do {

        S32G_GPR.GENCTRL1.B.CTRL = 0;

        /* Take the PFE out of reset */

        MC_RGM.PRST_0[2].PRST_0.B.PERIPH_0_RST = 0;

        MC_ME.PRTN2_PCONF.B.OSSE = 1;

        MC_ME.PRTN2_PCONF.B.PCE = 1;

        MC_ME.PRTN2_PUPD.B.OSSUD = 1;

        MC_ME.PRTN2_PUPD.B.PCUD = 1;

        /* Tell hardware to check above registers for changes */

        Write_Control_Key();

        tries++;

        if (tries > MAX_TRIES)

        {

            break;

        }

    } while (MC_RGM.PSTAT_0[2].PSTAT_0.B.PERIPH_0_STAT == 1);

    MC_ME.PRTN2_PCONF.B.OSSE = 0;

    MC_ME.PRTN2_PCONF.B.PCE = 1;

    MC_ME.PRTN2_PUPD.B.OSSUD = 1;

    MC_ME.PRTN2_PUPD.B.PCUD = 1;

    /* Tell hardware to check above registers for changes */

    Write_Control_Key();

    tries = 0;

    while (MC_RGM.PSTAT_0[2].PSTAT_0.B.PERIPH_0_STAT != 0)

    {

        tries++;

        if (tries > MAX_TRIES)

        {

            break;

        }        

    }

}

[aaronbaranoff]

None of the registers in the Red area are NOT accessible (readback 0 always). I can’t talk to them and your PFE code does not seem to use them (otherwise I would expect them to have non-zero values), so that seems to confirm the DMA interface is missing on the instantiations of the GMACs used for the PFE. So, the only way to send raw packets out of the PFE MACs is via the Slave interface (not documented by NXP) or the Debug Interface (which the registers are documented but not the sequence). Please Help and confirm or correct my suspicions.

[aaronbaranoff]

What time zone are you in maybe when can work interactively.

[aaronbaranoff]

Also added S32G_GPR

[aaronbaranoff]

Still no luck. PFE Partition does not fully initialize. The block clocks do not enable. There is no clear documentation on the expected sequence. The turn-on process is in Figure 148 of the S32G2 Reference Manual but it does not mention the block clocks (PRTN2_COFB0_STAT or PRTN2_COFB0_CLKEN). I am hoping some other user who has had luck with this can post an example bit of code. It comes out of reset and the PCS flag says the clocks are enabled but the block clocks don't get enabled but, not clue as to why. I even trying taking the exterenal clocks out of the picture. by using the FIRC. 

After setting up (PFE_MACx_RX_CLK, PFE_MACx_TX_CLK, PFE_PE_CLK) to used the default FIRC. PFE partition still comes out of reset but the block clocks being enabled is ignored. I have tried that moving the block clock enable before and after the rest of the is PFE partition out of reset also no impact.

I have tried in EL1 and EL3 modes. I have even trying ignoring the PRTN2_COFB0_STAT regardless of my attempts to enable via  PRTN2_COFB0_CLKEN REQs if I ignore the status that then I can read the PFE registers but writing to some registers fail and the behavior is not correct. See Table 195. 

I am using the RDB2 board (untill our board is ready) but have not had any luck. 

[aaronbaranoff]

I even tried pushing this to the very early stages in the bootloader and had the same results. 

MC_ME.PRTN2_COFB0_STAT.R is 0 (it should 0xf)...

WHY?!?

[alejandro_e]

Hello @aaronbaranoff,

As mentioned, it seems that the fact that MC_ME.PRTN2_COFB0_STAT.R is 0 is a know issue with only a workaround for now, skipping the check. The next thing I can think of to find the root cause is to compare your registers to mine, Please send a register dump of the MC_ME whole register rage, just before configuring PRTN2_COFB0_STAT, that way I can compare it to one of the available examples.

Thanks

[aaronbaranoff]

Even the AUTOSAR code reports issues with similar things but no solution or clarity.

[aaronbaranoff]

What I am asking is not driver specific. This is a simple register (reference manual) question.

I am trying to get Blocks 0-3 to have their clocks enabled in PRTN2_COFB0_STAT.  Even if I wanted to use your driver per 47.4 I need to (Enable PFE COFB clocks in MC_ME as described in the "MC_ME partition 2 mapping" section in this document.)

This is basic sequence but it does not work and the datasheet does not give any insight as to anything else that is needed. The partition comes out of reset and Figure 148 in the Reference Manual does not cover enabling the block clocks inside the partition. 

MC_ME.PRTN2_COFB0_CLKEN.B.REQ0 = 1;
MC_ME.PRTN2_COFB0_CLKEN.B.REQ1 = 1;
MC_ME.PRTN2_COFB0_CLKEN.B.REQ2 = 1;
MC_ME.PRTN2_COFB0_CLKEN.B.REQ3 = 1;

MC_ME.PRTN2_PCONF.B.OSSE = 1;
MC_ME.PRTN2_PUPD.B.OSSUD = 1;
MC_ME.PRTN2_PCONF.B.PCE = 1;
MC_ME.PRTN2_PUPD.B.PCUD = 1;

/* Tell hardware to check above registers for changes */
Write_Control_Key();

After this I read PRTN2_COFB0_STAT but the status never shows the clocks enabled. 

[alejandro_e]

Hello @aaronbaranoff,

Thanks for the reference on the AUTOSAR driver bug, I checked internally the change related to that bug fix, the function that got changed is Power_Ip_MC_ME_ConfigureCOFB(), you can see the final implementation in the RTD 5.0.0 QLP3. For what I understand, the problem is not in the writing part of the process, but when reading to verify, you will see in the function that when POWER_IP_MC_ME_PRTN2_COFB0_STAT_RESERVED is defined ad STD_ON the validation for PartitionIndex is skipped. Checking the configuration files for any S32G you will see POWER_IP_MC_ME_PRTN2_COFB0_STAT_RESERVED is true, for example: Resource_TS_T40D11M50I0R0\resource\CORTEXM_S32G2XXM7_s32g274a_bga525.properties:

what I understand is that there is a known issue with that register validation and for now skipping it is the know workaround.

[aaronbaranoff]

I have replicated NXP's code but added checking. The PFE subsystem comes out of reset but the block level clocks do not enable. If I ignore the status the behavior is that I don't get link status although I can read many of the PFE GMAC registers when I write to the DMA registers the values don't stick.

I believe that the status is a clue but get NO insight as to why. I have enabled the clocks and the pin mux prior to the PFE init.

Are there any other rules about being in EL1 vs EL3 mode or similiar I don't see any in the documention.

[alejandro_e]

Hello @aaronbaranoff,

Can you elaborate on this statement?  "but the block level clocks do not enable. If I ignore the status the behavior is that I don't get link status although I can read many of the PFE GMAC registers when I write to the DMA registers the values don't stick."

I have checked the documentation and just as you, I did not see anything specific considering EL3 vs EL0, however, I would recommend using EL0 to avoid any issue accessing the memory regions. 

[alejandro_e]

Hello @aaronbaranoff,

Thanks for reaching out to us again. At the movement I cannot share any resource that I have not shared before, for instance, the reference manual, FCI API documentation and code or code samples. Since the SW support for the A53 is very focused on Linux and the PFE being such a complicated module, I cannot offer you much support by looking at your program, I can suggest the same I always do, to check the already offered implementations and adapt them to your needs.

I can check internally if there is any plan to release more documents related to the PFE initialization, however I cannot ensure there will be anything helpful for your case.

I will get back to you

[aaronbaranoff]

Attached are the registers dumbs (in excel from the debugger)

[alejandro_e]

Hello @aaronbaranoff,

Sorry for such a delay in my reply. I was finally able to obtain the differences in the registers just before configuring the  PRTN2_COFB0_CLKEN register, you can find them in the xlsx file attached, you will find the differences marked with a red "X", each register dump is in a different page.

My registers were taken in this context:

[aaronbaranoff]

Thank you for the diffs. The only surprise was the CLKENs for the blocks inside the PFE were not set in the code you. Is that a known issue? I guess it is ignored or not needed. 

0x0000000F

EL1N:0x0000000040088530

00000000

0x00000000

X

OK, my PFE GMAC has come out of reset and I have valid link status. However, I have one more issue. All the registers outside of the DMA read and write, I can see our writes written as expected. But the DMA ones (TX or RX) seemed to allow us to write (no errors) but the value does not stick (it reads back 0). Is there something else that I am missing. It is like there is an extra external step I need to get the PFE GMAC to come to life that is not documented.

[aaronbaranoff]

It behaves as if the internal clocks are not correct. But since the Subsystem bug does not let us read the status it is hard to tell. I have verified the PIn Mux is setup to the correct pins and the clocks look like they are setup correctly. 

[alejandro_e]

Hello @aaronbaranoff,

You mean the registers in the PFE EMAC correct? this is, the same registers described in the GMAC reference manual.

If so, can you share a dump of all the registers, so that I can compare them with the registers state of the demo app just before sending.

Thanks in advance. 

[aaronbaranoff]

Here is the registers just prior to me not being able to successfully write to DMA_CH0_RxDesc_Ring_Length and to DMA_CH0_RxDesc_List_Address. When I write to them and read back the value it reads back 0 even though I write a valid address.

I however can sucessfuly get the link status, write and readback registers like the MAC address. 

This code is virtually identical to the code we are sucessfully using control the GMAC. But, on the PFE_MAC2 it does not allow use to read and write to certain registers. (mostly DMA related). In addition to the register comparison. Are the any other things outside of the PFE_MAC2 needed to enable its DMA (addition clocks or resets)? 

So in my code. I added checks and the following fail.

• DMA_CH0_TxDesc_List_Address  (failed readback)
• DMA_CH0_TxDesc_Ring_Length  (failed readback)
• DMA_CH0_RxDesc_List_Address  (failed readback)
• DMA_CH0_RxDesc_Ring_Length  (failed readback)
• MTL_TxQ0_Operation_Mode (is correct value but I can’t modify it)
• MTL_RxQ0_Operation_Mode (is correct value but I can’t modify it)
• MAC_RxQ_Ctrl0 (does not allow accept the write values)

[aaronbaranoff]

Also DMA_SysBus_Mode (failed readback). It is like an internal clock is not configured. But, I have not way of checking that since the PRTN2_COFB0_STAT always reports 0.

[aaronbaranoff]

Could you also compare my MC_CGM_2 registers to yours. I Think there is some clock that is missing and since all the PFE clocks are in that section.