Broadcast/Multicast Storm issue on P1010 with Marvell 88E1512 PHY

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Broadcast/Multicast Storm issue on P1010 with Marvell 88E1512 PHY

1,171 Views
kantamnagesh
Contributor I

I am using the Freescale P1010 Processor and Marvell 88E1512 PHY for Ethernet with Linux 3.18.9-rt4 kernel;

While performing Cyber Security tests using Achilles-box, we have found certain issues with IP Multicast Storm (generates identical, valid IP packets and multicasts them over the link at a specific rate) and UDP Broadcast Storm (generates identical, valid UDP packets and broadcasts them over the link at a specific rate), thus it was observed that the above threshold rate (above 90Mbps), the TCP and UDP ports went down and it requires a hard reboot to bring them back to the normal state.

Immediately after the broadcast storm starts, the message "NOHZ: local_softirq_pending 102" appears 10 times on the console. The reason it "only" appears 10 times is because the kernel limits the number of times it writes this message to 10.

About 11 seconds after the first "NOHZ: local_softirq_pending 102" message, a page allocation failure is reported. This means that system memory is depleted. Analysis of the call stack reveals that it is the driver (drivers/net/ethernet/freescale/gianfar.c and net/core/skbuff.c) for the Ethernet port, more precisely the soft-irq task which handles incoming frames, that fails to obtain more memory to receive buffers. Before the broadcast storm starts, there is more than 200MB of free memory, and all this is gone in 11 seconds. This renders the system unable to continue normal operation, even after the broadcast storm stops.

There seems to be a weakness in how Linux handles incoming Ethernet frames, leaving it vulnerable to storm attacks where all available memory is consumed, leaving the system in an unstable state after the attack is over.

Could anyone have faced such similar issue and does anybody have knowledge on how to fix / suggestion to find out why a broadcast/multicast storm is able to consume almost all CPU power of the P1010 processor.

Labels (1)
Tags (1)
0 Kudos
3 Replies

871 Views
alexander_yakov
NXP Employee
NXP Employee

We support only Linux kernel from our Linux SDK. You mentioned Linux 3.18.9-rt4 kernel, it is not from our SDK, right?
Using Linux SDK1.9, Linux 3.12.37-rt51+g43cecda, and Spirent Testcenter to generate 100% load 1Gbps broadcast traffic - no issue observed.


Have a great day,
Alexander
TIC

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

0 Kudos

872 Views
kantamnagesh
Contributor I

Thanks for the information,
I have chosen Linux 3.18.9-rt4 version because Linux SDK1.x was not supported in some of our peripherals,e.g (Marvell 88E6046 Ethernet switch drivers). I will explore further and update on same.

Best Regards,

Nagesh

0 Kudos

872 Views
kantamnagesh
Contributor I

Thanks for the information Alexander,

I will explore the changes and update on same.

Best Regards,Nagesh

On Wednesday, October 24, 2018, 1:38:34 AM GMT+5:30, alexander.yakovlev <admin@community.nxp.com> wrote:

|

NXP Community

|

Re: Broadcast/Multicast Storm issue on P1010 with Marvell 88E1512 PHY

reply from alexander.yakovlev in PowerQUICC Processors - View the full discussion

We support only Linux kernel from our Linux SDK. You mentioned Linux 3.18.9-rt4 kernel, it is not from our SDK, right?

Using Linux SDK1.9, Linux 3.12.37-rt51+g43cecda, and Spirent Testcenter to generate 100% load 1Gbps broadcast traffic - no issue observed.

Have a great day,

Alexander

TIC

 

0 Kudos