Issue in Freescale network driver

[abhijit_thorat]

I am using QorIQ® T2080 development board for my project. 

I have prepared yocto setup to built required images using following packages:

1. QorIQ Linux SDK v2.0 PPC64E6500 Yocto Cache.iso
2. QorIQ Linux SDK v2.0 SOURCE.iso
3. QorIQ Linux SDK v2.0-1703.tar.bz2

When I run my custom application on ssh I see different kind of issues in freescale driver.

---------------------------------------------------------------------------------------------------------------------------------------

CPU: 5 PID: 0 Comm: swapper/5 Tainted: G B W O 4.1.35-rt41 #45
Call Trace:
[c0000000fff9b3f0] [c00000000077a664] .dump_stack+0xac/0xec (unreliable)
[c0000000fff9b480] [c0000000000d8988] .bad_page+0x13c/0x190
[c0000000fff9b520] [c0000000000dc5c4] .get_page_from_freelist+0x450/0x8b4
[c0000000fff9b670] [c0000000000dcd98] .__alloc_pages_nodemask+0x19c/0x9c0
[c0000000fff9b810] [c00000000066458c] .__alloc_page_frag+0x134/0x1dc
[c0000000fff9b8b0] [c0000000006682f0] .netdev_alloc_frag+0x34/0x60
[c0000000fff9b930] [c0000000004c7f04] ._dpa_bp_add_8_bufs+0x80/0x2fc
[c0000000fff9ba20] [c0000000004c81e0] .dpaa_eth_refill_bpools+0x60/0x84
[c0000000fff9bab0] [c0000000004c6780] .priv_rx_default_dqrr+0x80/0x1c0
[c0000000fff9bb60] [c00000000062b184] .qman_p_poll_dqrr+0x1b0/0x2b8
[c0000000fff9bc20] [c0000000004c6af0] .dpaa_eth_poll+0x2c/0x7c
[c0000000fff9bcb0] [c000000000676cd8] .net_rx_action+0x23c/0x33c
[c0000000fff9bdb0] [c00000000003d12c] .__do_softirq+0x198/0x314
[c0000000fff9beb0] [c00000000003d7d8] .irq_exit+0xb8/0xe4
[c0000000fff9bf20] [c000000000005ad0] .__do_irq+0x64/0xa8
[c0000000fff9bf90] [c000000000012fc0] .call_do_irq+0x14/0x24
[c0000000f10ff9c0] [c000000000005b98] .do_IRQ+0x84/0x118
[c0000000f10ffa50] [c00000000001793c] exc_0x500_common+0xfc/0x100
--- interrupt: 501 at .book3e_idle+0x24/0x4c
LR = .book3e_idle+0x24/0x4c
[c0000000f10ffd40] [c0000000000091b4] .arch_cpu_idle+0x34/0xa0 (unreliable)
[c0000000f10ffdb0] [c0000000000749f0] .cpu_startup_entry+0x204/0x2ac
[c0000000f10ffeb0] [c00000000001e898] .start_secondary+0x3f8/0x538
[c0000000f10fff90] [c00000000000046c] start_secondary_prolog+0x10/0x14

------------------------------------------------------------------------------------------------------------------------------

 Unable to handle kernel paging request for data at address 0x10000007ee50080
Faulting instruction address: 0xc0000000006bd3d0
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=24 CoreNet Generic
Modules linked in: ter_ib(O)
CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 4.1.35-rt41 #80
task: c0000001f10fccd0 ti: c0000001fe7ac000 task.ti: c0000001f1100000
**bleep**: c0000000006bd3d0 LR: c0000000004e0360 CTR: c0000000002dd8b0
REGS: c0000001fe7af690 TRAP: 0300 Tainted: G O (4.1.35-rt41)
MSR: 0000000080029000 <CE,EE,ME> CR: 28000088 XER: 00000000
DEAR: 010000007ee50080 ESR: 0000000000000000 SOFTE: 1
GPR00: c0000000004e0360 c0000001fe7af910 c000000000c7fe00 c00000002e3a2200
GPR04: c0000001f17e1000 c00000002e3a2200 000000000000008e 010000007ee4cec0
GPR08: 0000000000000000 00000000000031c0 00000000000000d6 010000007ee50f3f
GPR12: 0000000048000082 c00000003fff56c0 0000000000004163 c00000002e3a2200
GPR16: 80000800809f13d0 0000000000000018 c000000000c87080 00000001fd9c7000
GPR20: 000000010006d86a c0000001fe525680 000000007ee4cec0 000000007ee4cec0
GPR24: 0000000000000000 c0000001f17e1700 c0000001f17e1000 800003fffff2697c
GPR28: c00000007ef790d8 800003fffff275b0 000000007ee4cec0 c00000007ee4cec0
**bleep** [c0000000006bd3d0] .eth_type_trans+0x54/0x160
LR [c0000000004e0360] ._dpa_rx+0x1b4/0x784
Call Trace:
[c0000001fe7af910] [c0000001f1425f00] 0xc0000001f1425f00 (unreliable)
[c0000001fe7af990] [c0000000004e0360] ._dpa_rx+0x1b4/0x784
[c0000001fe7afab0] [c0000000004dd99c] .priv_rx_default_dqrr+0xac/0x1c0
[c0000001fe7afb60] [c00000000064c238] .qman_p_poll_dqrr+0x1b0/0x2b8
[c0000001fe7afc20] [c0000000004ddce0] .dpaa_eth_poll+0x2c/0x7c
[c0000001fe7afcb0] [c000000000698024] .net_rx_action+0x23c/0x33c
[c0000001fe7afdb0] [c00000000003d12c] .__do_softirq+0x198/0x314
[c0000001fe7afeb0] [c00000000003d7d8] .irq_exit+0xb8/0xe4
[c0000001fe7aff20] [c000000000005ad0] .__do_irq+0x64/0xa8
[c0000001fe7aff90] [c000000000012fc0] .call_do_irq+0x14/0x24
[c0000001f11039c0] [c000000000005b98] .do_IRQ+0x84/0x118
[c0000001f1103a50] [c00000000001793c] exc_0x500_common+0xfc/0x100
--- interrupt: 501 at .book3e_idle+0x24/0x4c
LR = .book3e_idle+0x24/0x4c
[c0000001f1103d40] [c0000000000091b4] .arch_cpu_idle+0x34/0xa0 (unreliable)
[c0000001f1103db0] [c000000000074930] .cpu_startup_entry+0x204/0x2ac
[c0000001f1103eb0] [c00000000001e898] .start_secondary+0x3f8/0x538
[c0000001f1103f90] [c00000000000046c] start_secondary_prolog+0x10/0x14
Instruction dump:
b12300ae 409d0028 81030074 394afff2 91430070 7f885040 7d000026 5508f7fe
0b080000 38c6000e f8c300c0 79290420 <7d4748ae> 7d074a14 794607e1 4082003c

----------------------------------------------------------------------------------------------------------------------------------------

Unable to handle kernel paging request for data at address 0x1000001ec041000
Faulting instruction address: 0xc000000000680f64
Oops: Kernel access of bad area, sig: 11 [#1]
SMP NR_CPUS=24 CoreNet Generic
Modules linked in: ter_ib(O)
CPU: 1 PID: 12 Comm: rcuc/1 Tainted: G O 4.1.35-rt41 #59
task: c0000001f1120d90 ti: c0000001fe7b0000 task.ti: c0000001f1124000
**bleep**: c000000000680f64 LR: c0000000006811c4 CTR: c0000000002dcb50
REGS: c0000001fe7b3820 TRAP: 0300 Tainted: G O (4.1.35-rt41)
MSR: 0000000080029000 <CE,EE,ME> CR: 24a2be24 XER: 00000000
DEAR: 01000001ec041000 ESR: 0000000000000000 SOFTE: 1
GPR00: c0000000006811c4 c0000001fe7b3aa0 c000000000c7dc00 c0000001ec4da200
GPR04: 00000001ec04109a 40ffffffffffff66 0000000000000001 0000000000000000
GPR08: 01000001ec041000 0000000000070000 0000000000000000 000000000000b78a
GPR12: 0000000024adbe24 c00000003fff56c0 0000000000000003 0000000000000100
GPR16: 0000000000000002 0000000000000018 c000000000c83080 00000001fd9c9000
GPR20: 0000000100002ec9 c000000000b5e140 000000000000012c 0000000000000001
GPR24: 0000000000000000 0000000000000001 ffffffffffffff80 0000000040000000
GPR28: c0000001fe525680 80000800809f12c0 01000001ec041000 c0000001ec4da200
**bleep** [c000000000680f64] .skb_release_data+0x64/0x150
LR [c0000000006811c4] .consume_skb+0x34/0x80
Call Trace:
[c0000001fe7b3aa0] [c0000001fe7b3b30] 0xc0000001fe7b3b30 (unreliable)
[c0000001fe7b3b30] [c0000000006811c4] .consume_skb+0x34/0x80
[c0000001fe7b3bb0] [c0000000004dd4fc] .priv_tx_conf_default_dqrr+0x7c/0x1a4
[c0000001fe7b3c40] [c00000000064b4d8] .qman_p_poll_dqrr+0x1b0/0x2b8
[c0000001fe7b3d00] [c0000000004dcf80] .dpaa_eth_poll+0x2c/0x7c
[c0000001fe7b3d90] [c0000000006972c4] .net_rx_action+0x23c/0x33c
[c0000001fe7b3e90] [c00000000003d12c] .__do_softirq+0x198/0x314
[c0000001fe7b3f90] [c000000000012f9c] .call_do_softirq+0x14/0x24
[c0000001f11279b0] [c000000000005e60] .do_softirq_own_stack+0x4c/0x90
[c0000001f1127a40] [c00000000003ce28] .do_softirq.part.15+0x50/0x70
[c0000001f1127ac0] [c00000000003cef8] .__local_bh_enable_ip+0xb0/0xd0
[c0000001f1127b30] [c00000000008b8ac] .rcu_cpu_kthread+0xcc/0x734
[c0000001f1127c60] [c00000000005c53c] .smpboot_thread_fn+0x268/0x270
[c0000001f1127d10] [c0000000000586e4] .kthread+0xf0/0x110
[c0000001f1127e30] [c000000000000998] .ret_from_kernel_thread+0x58/0xc0
Instruction dump:
3d200001 61290001 418200d0 38de0020 7c0004ac 7ce03028 7ce93850 7ce0312d
40a2fff4 7c0004ac 2f870000 409e00d4 <7d2850ae> 7c7d1b78 3be00000 2f890000
---[ end trace 3f8d58e3cb691b64 ]---

----------------------------------------------------------------------------------------------------------------------------------

Kernel configuration for freescale driver : 

CONFIG_FSL_SDK_DPAA_ETH=y
# CONFIG_FSL_DPAA_HOOKS is not set
# CONFIG_FSL_DPAA_MACSEC is not set
# CONFIG_FSL_DPAA_CEETM is not set
CONFIG_FSL_DPAA_OFFLINE_PORTS=y
CONFIG_FSL_DPAA_ADVANCED_DRIVERS=y
CONFIG_FSL_DPAA_GENERIC_DRIVER=y
# CONFIG_FSL_DPAA_ETH_JUMBO_FRAME is not set
# CONFIG_FSL_DPAA_TS is not set
# CONFIG_FSL_DPAA_1588 is not set
CONFIG_FSL_DPAA_ETH_USE_NDO_SELECT_QUEUE=y
CONFIG_FSL_DPAA_ETH_REFILL_THRESHOLD=512
CONFIG_FSL_DPAA_CS_THRESHOLD_1G=0x06000000
CONFIG_FSL_DPAA_CS_THRESHOLD_10G=0x10000000
CONFIG_FSL_DPAA_INGRESS_CS_THRESHOLD=0x10000000
CONFIG_FSL_DPAA_ETH_DEBUGFS=y
# CONFIG_FSL_DPAA_ETH_DEBUG is not set
# CONFIG_FSL_DPAA_DBG_LOOP is not set

CONFIG_FSL_DPAA_ETH_MAX_BUF_COUNT=2048

---------------------------------------------------------------------------------------------------------------------------------

I made MAX BUF COUNT to 2048 from default value 128.

It just postpone the timing of issue. But i can still see crashes after sometime. 

Anyone knows what could be the reason or other changes required in kernel configuration?

[yipingwang]

Please build Linux Kernel with the default Kernel configuration file.

$ bitbake virtual/kernel -c cleansstate

$ bitbake virtual/kernel

Then try uImage generated with the default Linux Kernel configuration file on your target board to check whether the crash problem remains.

If the crash problem remains, please verify DDR controller configuration. Under u-boot, please use mtest command to test DDR memory.

Please use QCVS DDRv tool to connect to your target board do DDR configuration validation and optimization.