FM Error in Linux with Topaz Hypervisor

[paterb]

I am running separate Linux instances on all 4 cores of a T1042 processor on top of the Topaz Hypervisor.  While running a test which was receiving Ethernet traffic on 1 port and forwarding out another port, which then got received in a different core (there was also minimal Macless traffic between the cores), I got the following error:

In Linux on Core 0:  cpu0/0: ! CRITICAL FM Error [CPU00, drivers/net/ethernet/freescale/fman/Peripherals/FM/fm.c:4358 FM_ErrorIsr]: Invalid State;

On Hypervisor:  [0] Error interrupt reflected, error-int=1, guest=p0-linux

Any ideas what could cause an error like this?

[yipingwang]

Hello Pat Erb,

Communication between two Hypervisor Linux partitions through DPAA Macless is a typical use case.

The TX frame queues from one partition should be the RX frame queues from the other partition.

Please refer to the following dts definition.

dpa-ethernet@10 {

compatible = "fsl,p4080-dpa-ethernet", "fsl,dpa-ethernet-macless";

fsl,qman-frame-queues-rx = <0x4000 8>;

fsl,qman-frame-queues-tx = <0x4008 8>;

local-mac-address = [02 00 c0 a8 6f fe];

fsl,bman-buffer-pools = <&bp10>;

};

bp10: buffer-pool@10 {

compatible = "fsl,b4080-bpool", "fsl,bpool";

fsl,bpid = <10>;

fsl,bpool-ethernet-cfg = <0 0x80 0 1728 0 0x80000000>;

fsl,bpool-ethernet-seeds;

};

Device tree from the second partition:

dpa-ethernet@10 {

compatible = "fsl,p4080-dpa-ethernet", "fsl,dpa-ethernet-macless";

fsl,qman-frame-queues-rx = <0x4008 8>;

fsl,qman-frame-queues-tx = <0x4000 8>;

local-mac-address = [02 00 c0 a8 79 fe];

fsl,bman-buffer-pools = <&bp10>;

};

bp10: buffer-pool@10 {

compatible = "fsl,b4080-bpool", "fsl,bpool";

fsl,bpid = <10>;

fsl,bpool-ethernet-cfg = <0 0x80 0 1728 0 0x80000000>;

};

In the above configuration there are 8 TX frame queues and 8 RX frame queues used in interchangeable roles in each partition. Both interfaces share one statically defined buffer pool.

In addition, which fmc policy file did you use? Would you please provide some detailed steps to help us to reproduce your problem?

Have a great day,
Yiping

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[paterb]

Yiping,

I understand how the shared MAC works, but wasn't sure if this error we saw was on the Shared MAC port or on a regular Ethernet port, as both were being used at the time of the error.  Unfortunately, we have not been able to reproduce the issue yet. 

Can you confirm this error is on the shared MAC port?

Here is our fmc policy files:

<netpcd xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

  xsi:noNamespaceSchemaLocation="xmlProject/pcd.xsd" name="example"

  description="HV2p Shared-Mac Configuration">

  <distribution name="eth_dist_2">

  <queue count="1" base="0x350"/>

  </distribution>

  <policy name="shared_mac_policy_2">

  <dist_order>

  <distributionref name="eth_dist_2"/>

  </dist_order>

  </policy>

   <distribution name="eth_dist_3">

      <queue count="1" base="0x3d0"/>

   </distribution>

   <policy name="shared_mac_policy_3">

      <dist_order>

         <distributionref name="eth_dist_3"/>

      </dist_order>

   </policy>

   <distribution name="eth_dist_4">

      <queue count="1" base="0x450"/>

   </distribution>

   <policy name="shared_mac_policy_4">

      <dist_order>

         <distributionref name="eth_dist_4"/>

      </dist_order>

   </policy>

</netpcd>

<netpcd xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"                  

  xsi:noNamespaceSchemaLocation="xmlProject/pcd.xsd" name="example" 

  description="FMAN Tester configuration">

  <classification name="eth_dest_clsf">

  <key>

  <fieldref name="ethernet.dst"/>

  </key>

  <entry>

  <data>0x0001af079b85</data>

  <queue base="0x210"/>

  </entry>

  <entry>

  <data>0x0200C0A8A1FE</data>

  <queue base="0x220"/>

  </entry>

  <action condition="on-miss" type="distribution" name="garbage_dist"/>

  </classification>

  <classification name="arp_clsf">

  <key>

  <fieldref name="arp.p_dst"/>

  </key>

  <entry>

  <data>0xC0A80162</data>

  <mask>0xFFFFFFFF</mask>

  <queue base="0x220"/>

  </entry>

  </classification>

  <distribution name="eth_dist">

  <queue count="1" base="0x230"/>

  <action type="classification" name="eth_dest_clsf"/>

  </distribution>

  <distribution name="arp_dist">

  <queue count="1" base="0x230"/>

  <protocols>

                <protocolref name="arp"/>

        </protocols>

  <action type="classification" name="arp_clsf"/>

  </distribution>

  <distribution name="garbage_dist">

  <queue count="1" base="0x230"/>

  </distribution>

  <policy name="shared_mac_policy">

  <dist_order>

  <distributionref name="arp_dist"/>

  <distributionref name="eth_dist"/>

  </dist_order>

  </policy>

</netpcd>

<cfgdata>

  <config>

  <engine name="fm0">

  <port type="1G" number="0" policy="shared_mac_policy_2"/>

  <port type="1G" number="1" policy="shared_mac_policy_3"/>

  <port type="1G" number="4" policy="shared_mac_policy_4"/>

      </engine>

  </config>

</cfgdata>

Pat