T1024 10G port not consistently up

[ramkrishnan]

We have configured the T1024 to be in serdes protocol 0x99. With the 10G port attached to a Marvell Tranceiver 88X222M. Marvell has a training script which we run using the mdio commands in uboot and the problem is that the port passes traffic roughly around 3 out of 10 times (the fact that it works 3 times I would think indicates that the hardware path seems to be fine).  

I have forced the link to be up in the uboot for the 10G link. On the Marvell tranceiver it shows that the host side and port side links are all up.

But if I run a ping after a few tries it gives the errors that it runs out of txbds. I dump out the registers FE4E0200 to see if any packets are being transmitted and the counts do not increment. But in the cases it does work. I see the counters going up in FE4E0200 and FE4E0100.

The serdes registers seem to be all correct with configuration for XFI 10G. We tested all the clocks and they all seem to be fine.

Is there anything else we can look at the T1024 to try to isolate where the issue might be.

Thank you,

Ram Krishnan

[ramkrishnan]

There seems to be a workaround , though it does not explain it. 

After the Training of the 88X2222P is done and the ping fills up the buffer descriptor queue, set the loopback bits on the SERDES on the QorIQ and then take it out of the Loopback. Things seems to works and traffic starts flowing thru after that.

Specifically in this case the register was

FE0EA83C (Defined in Section 30.4.24 of the T1024 RM) bits 2-3 to 01 and then set it to 00. As soon as you set it to 01, the packets gets flushed out to the destination and when you set it back to 00 you can ping the destination.

Not exactly sure why that would work. But it seems to point to some initialization on the QorIQ for the Serdes Lanes or maybe the loopback affects the mac , not entirely sure yet. Could somebody shed some light on this ?

Thanks,

Ram