Hi,
We have encountered an issue with IPSec. When using IPSec to receive downlink packets, after a period of time, the core layer cannot receive the packets.
We are attempting to debug this issue
I conducted the following checks (before and after the problem occurred, the rate of downlink packet filling remained unchanged)
1. Check if there are no data packets on the network port. Through packet capture inspection, it was found that the network port can receive downstream ESP packets
2. Check the rx port of fman and find that the number of counter values increasing within 1 second before and after the problem is similar
3. Check the esp interrupt and performance monitor registers, and after checking the issue, the decryption times in 1 second have increased by about the same amount
4. I tried to check the FMan offline port, but the counter was always 0 and there was no valid information available
May I ask if there is any other information that can help us debug the issue of the core layer not receiving packets?
Hi, We have encountered an issue with IPSec. When using IPSec to receive downlink packets, after a period of time, the core layer cannot. Check Here
Let me be more clear about your status,
1. Can I know IPsec run in user space or kernel space? DPDK in user space?
2. Fman rx port can continue receive ESP packets, right? Did you know where these received ESP packets go?
3. Can you see 'ob_bytes_encrypted' and 'ib_bytes_decrypted' number increase when in problematic setup?
You can have a try below commands to debug,
1. If you know to-SEC fqid, you can query this FQ to check tis status, e.g fqid 318 is to SEC,
root@:~# echo 318 > /sys/kernel/debug/qman/query_fq_np_fields
root@:~# cat /sys/kernel/debug/qman/query_fq_np_fields
root@:~# cat /sys/kernel/debug/qman/query_fq_fields
2. Stop traffic, check Bman buffer counter with below command,
cat /sys/devices/ffe000000.soc/ffe31a000.bman/pool_count/?
Hi ,
The previous reply was nested in too many layers, so a new reply is being opened at this layer.
Regarding the previous question, my test results indicate that it is not a problem with SEC decryption, what should I check next? Thank you.
Good to know JR mode is used and SEC work well for IPsec decryption.
If SEC can return decrypted packet successful, can core receive these packets from SEC?
If core can receive these packets, can customer trace these packets? Be dropped when route to output port?
Sorry, but the only thing I can confirm so far is that no packets are received on the USER side; is there any way to count the packets between SEC and core?
You could check in two ways,
1. add printk in kernel caam driver, caam_jr_dequeue() in caam_jr_dequeue drivers\crypto\caam\jr.c
2. check JR interupt count increase or not
root@localhost:~# cat /proc/interrupts | grep jr
77: 13 0 0 0 GICv2 103 Level 1710000.jr
78: 0 0 0 0 GICv2 104 Level 1720000.jr
79: 0 0 0 0 GICv2 105 Level fsl-jr0
root@localhost:~#
thanks, we will try then.
1.ipsec in kernel , i did not use DPDK
2.Fman rx port can continue receive ESP packets, these ESP packets should enter the SEC for decryption
3.Only tested the downlink, ib_ bytes_ decrypted number consistently increase when in problematic setup
For your suggestion for debug
1.I have checked fpids, but did not see SEC's pid , just get Rx PCD,Rx PCD High Priority, RX default and RX error. May I ask if there are any other methods to determine the SEC's pid ?
2.i will try , thank you
Please refer to the following update from the AE team.
Can I know which LSDK customer use?
76: 125 0 0 0 GICv2 103 Level 1710000.jr
77: 3113307 0 0 0 GICv2 104 Level 1720000.jr
78: 781273 0 0 0 GICv2 105 Level 1730000.jr
root@localhost:~#
root@localhost:~# cat /proc/interrupts | grep QMan
32: 0 0 0 0 GICv2 216 Level QMan portal 3
33: 0 0 0 0 GICv2 218 Level QMan portal 2
34: 0 0 0 0 GICv2 220 Level QMan portal 1
35: 50 0 0 0 GICv2 222 Level QMan portal 0
If customer use JR mode, they needn't query FQ status.
caam_jr_dequeue() in driver/crypto/caam/jr.c userstatus = caam32_to_cpu(jr_outentry_jrstatus(jrp->outring,
hw_idx));
"userstatus" should be zero if SEC run successful. Please check this status when using JR mode.
As my previous answer, did customer find any error message when SEC finish ipsec decryption?
For JR mode, they can check in linux caam driver, in caam_jr_dequeue() in driver/crypto/caam/jr.c
userstatus = caam32_to_cpu(jr_outentry_jrstatus(jrp->outring,
hw_idx));
"userstatus" should be zero if SEC run successful. Please check this status when using JR mode.
1. Yes,I used JR mode
2."userstatus" is zero when IPsec downlink data impassability, so SEC run successful.
3.You asked me if there is anything other than a "userstatus" error message when the SEC completes the ipsec decryption?
You can check if interrupts number increase for jr, it means JR mode is enabled.
Or you can check whether "CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API_QI=y" enabled in your kenrel config. 'y' means QI mode is enabled. Meanwhile "CONFIG_CRYPTO_DEV_FSL_CAAM_CRYPTO_API=n", JR mode should be disabled if QI mode enabled.