Vybrid DMA0, ENET0 vs DMA1, ENET1 and CA5 vs CM4

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Vybrid DMA0, ENET0 vs DMA1, ENET1 and CA5 vs CM4

3,782 Views
scottthompson
Contributor II

Hello,

We are running Linux 3.13.9 on CA5 core and MQX 4.2.x on CM4 core.  Originally, we allowed Linux to take ETH0 and DMA0 (and DMAMUX0/1) and left ETH1 and DMA1 to CM4.  We are using an external 7-port Ethernet switch configured as dual-RMII PHY and using the two RMII connections on the Vybrid to connect to this switch.  MDC/MDIO is connected to RMII0 only.

In response to AN4947 (in addition to the Vybrid RM), to reduce latency the CM4 core should be assigned ETH0 and DMA0 and the CA5 should take ETH1 and DMA1.  I was able to swap the DMA channels okay, and even got ETH1 to boot properly in Linux (but a few errors saying the MAC was invalid--but am able to ping and communicate).

In the device tree, I disabled ETH0 but maintained the IOMUXC assignments since they are being used with weak pull-ups to configure the RCON registers for booting.  I also disabled the MDC/MDIO IOMUXC in the device tree because I plan on using MQX to manage the switch now...  Although Linux boots and appears to operate properly using ETH1, MQX hangs on the first access to ETH0 in the MAC driver, as shown below.

ECR = ENET_ECR_RESET_MASK;

/* wait until the initialization cycle completes */


while ((macnet_ptr->ECR & ENET_ECR_RESET_MASK) && (timeout<MACNET_RESET_TIMEOUT)){

ECR & ENET_ECR_RESET_MASK), ENETERR_INIT_FAILED);

The macnet_ptr variable points to 0x400D0000 which is ETH0.  DS-5 and DSTREAM report in the debugger that the registers in the ENET_MemMapPtr are "unavailable" but I'm not sure if that's just a cache issue while drilling into the BSP with caches still turned on with DSTREAM...  The ETH0 driver is configured for direct connection to the switch.

My goal is to try and get Linux to work with ETH1 and MQX to work with ETH0 as well as MDC/MDIO connected to RMII0 but controlled by MQX.

In the configuration that "works", Linux is driving ETH0/DMA1 but MDC/MDIO is still connected to RMII0.

Has anyone been able to get Linux to use (only) ETH1 and MQX to use ETH0?  We've generally had to completely split our peripheral assignments as we found that when Linux takes ahold of a device, MQX hangs when trying to access the peripheral's memory region.  I'm guessing that Linux somehow still has control of ETH0 even though it's been configured to use ETH1.

At the end of the day, Linux = ETH1/DMA1 and MQX = ETH0/DMA0.  So far, I am able to get the DMA swapped out okay.  BTW our Vybrid SoMs have the PHYs removed.

Thanks,

Scott

Labels (4)
0 Kudos
7 Replies

1,814 Views
timesyssupport
Senior Contributor II

Hi Scott,

                As far as I checked I could not see any configuration options that you may have missed. As you said you can swap the DMA channels and move on.

Thanks,

Timesys support.

0 Kudos

1,814 Views
timesyssupport
Senior Contributor II

Hi Scott,

                 My apologies, I could not reproduce this issue.The issue with  peripherals lock up when accessed in MQX (if they’ve been initialized in Linux) is something that I have not exposed. I will try and discuss with our senior engineers and come back to you.

Thanks,

Timesys support.

0 Kudos

1,814 Views
timesyssupport
Senior Contributor II

Hi Scott,

                 I am currently looking at this issue  I will come back to you with my findings at the earliest.  In the mean time can I know the reason to do above mentioned changes. Can you also include your work order which helps debug the problem efficiently.

Thanks,

Timesys support.

0 Kudos

1,814 Views
scottthompson
Contributor II

Last one:

Investigation may show that our biggest improvement may be due to swapping DMA. SAI channels are on AIPS0 as well as DMAMUX0, DMAMUX1. Also, this peripheral bridge is coupled via a port splitter which connects to the 64 kB TCM.

I think in hindsight that swapping Ethernet ports is probably not going to do a whole lot of anything for us. DDR access is from Cortex-A5 side only now. Both ETH0 and ETH1 go through AIPS1. A quick-and-dirty NIC mapping isn’t showing anything outstanding besides the DMA and RAM reassignments…

If you can think of any configuration options I may have missed, or have some insights into why peripherals lock up when accessed in MQX if they’ve been initialized in Linux, then please feel free to post them; otherwise, I will just go ahead and close this discussion as solved by swapping our DMA channels.

Thanks,

Scott

0 Kudos

1,814 Views
scottthompson
Contributor II

Hi,

After re-reading through the AN and reworking our memory requirements, it appears that it would be prudent to re-map our architecture with the NIC-301 in mind and clearly identify which master/slave ports connect to which memory. We may have mis-read the document and assumed that CM4 needed to be attached to ETH0 and DMA0. Our improved DMA access may have been due to us relocating our RAM to a different slave port.

0 Kudos

1,814 Views
scottthompson
Contributor II

Hello,

We are processing real-time audio data (TDM-4/32-bit/48 ksps) in Cortex-M4 core using DMA to handle the I2S interface. Both Cortex-A5 and Cortex-M4 cores connect to dedicated RMII ports: CA5 core (Linux) was going to handle switch management functions (recall we are connected to a Marvell 7-port switch) while the CM4 core handled the real-time audio. Both cores are able to output Ethernet audio frames (we are implementing AVB); CA5 audio frames were designed for pre-recorded announcements, for example, or tones and didn’t require the real-time capacity that we need from the CM4 core.

It was observed in the CM4 core that when Linux was initially at rest, audio would flow properly (real-time). As the CA5 core was loaded down with processing (not using shared resources, except for some DDR access on one go), it was observed that this impacted the CM4 core in its ability to handle the real-time audio.

Since both cores share the NIC-301, further searching revealed the mentioned application note (AN4947: Understanding Vybrid Architecture). The application note mentions severe latency penalties when accessing the NIC through various non-ideal muxing. It mentions that the CM4 core should ideally access DMA0, DCU0, USB0, ENET0, etc. and that the CA5 core should point to DMA1, DCU1, USB1, ENET1, etc. Further digging through the Vybrid RM shows that this should be the ideal method to reduce latency through the NIC. I haven’t delved too deeply into the exact master/slave NIC relationships but it warranted investigation, in particularly the DMA access, since our real-time audio is reliant on an efficient DMA engine.

We swapped out DMA resources between Linux and MQX and observed reduced execution times in our main CM4 code; this seemed to begin to validate our task of moving forward with recommendations in AN4947. Toward this end, I reworked the device tree to point Linux to ETH1 and MQX to ETH0.

With careful patching of the device tree, I was able to (finally—Linux was a learning curve for me!) get Linux to boot using ETH1, but the boot-up diagnostic messages indicated that there may be some oddities happening (for example, I’d get a message saying something like invalid MAC address 00:00:00:00:00:00 and it assigning a random MAC address). At the end of the day, I was able to ping the CA5 when it was configured for ETH1. Interestingly, our assigned MAC address for this core never propagates down where it is reflected in an ‘ifconfig’ display (i.e., MAC addresses programmed in uBoot don’t propagate through when using ETH1, but they do/did when using ETH0).

In MQX, when attempting to configure its Ethernet driver for ETH0 (instead of ETH1) . Upon any access to ETH0 registers (e.g., RESET MAC), MQX simply hangs—no ISR warning, no hard fault, no description of why it hangs.

This ‘hanging’ in MQX was observed when attempting to access any peripherals that Linux had originally claimed (e.g., I2C). This is the reason that we explicitly mapped out peripherals that Linux could have, and peripherals that MQX could have. We thought this was a suitable solution to trying to attempt Core Semaphore access, especially since we defined at a system level which core was doing what.

So the current configuration is (not using USB):

CA5: Linux 3.13.9; ETH0; DMA1; DMAMUX2, DMAMUX3

CM4: MQX 4.2.0.1; ETH1; DMA0; DMAMUX0, DMAMUX1

Since we are using AVB for our real-time audio over network, we thought it was important to also swap out the Ethernet ports to line up with the reference manual and application note to yield our best performance.

Please bear with me, I’m a real-time embedded guy so Linux is fresh for me, but I’m able to understand enough to patch the device tree and compile the kernel to yield a working build).

I started with TimeSys a few years ago when we first got started with our proof-of-concept but a co-worker configured the work order (in conjunction with our internal Linux kernel) so I don’t have that information handy—sorry.

I hope this sufficiently answers the “why” of our intention.

Thanks in advance for any insights into mapping CA5 to ETH1. I think the lock-up in MQX is simply Linux somehow holding onto ETH0 resources somewhere, even though they are not explicitly configured—if that helps for a starting point…

As an aside, the SMI interface is on RMII0 but is configured in MQX outside of Linux. This interface works and is currently being used with Marvell’s DSDT driver suite. This interface will NOT change if/when Linux changes to ETH1 and MQX changes to ETH0.

Cheers,

Scott T.

0 Kudos

1,814 Views
jamesbone
NXP TechSupport
NXP TechSupport

timesyssupport​, can you please take a look

0 Kudos