While developing a solution based on the board
we had a problem.
The board is used as a communication device between 2 hosts.
It is connected to one host via usb3 and using the Gadget module, we create several virtual network adapters (NCM) which are used to transfer data.
The second host is connected to it via 6 usb network adapters (2 different manufacturers) which are connected to the board via a usb hub.
The problem is in connection to the second host. Some network adapters stop receiving traffic or all of them disappear from the system or the board totally hangs up. The problem is observed regardless of the load, using only 3 adapters and transferring traffic at 40Mbps (bidirectional).
We tried to stop using external usb hubs but this did not help. We also tried to use usb-c port instead of usb-a. We also changed connection to host 1 from usb to ethernet (enet1 on the board) in case it affects the work of usb controller on the whole. But that didn't solve the problem either.
The firmware is in use:
NXP i.MX Release Distro 5.10-hardknott (hardknott)
imx-image-core
Can you please advise how to fix the problem?
This is what is displayed in the console during the problem:
[ 489.469553] xhci-hcd xhci-hcd.1.auto: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 4
[ 489.480086] xhci-hcd xhci-hcd.1.auto: Looking for event-dma 000000004400e000 trb-start 000000004400da80 trb-end 000000004400da80 seg-start 000000004400d000 seg-end 000000004400dff0
[ 489.496174] xhci-hcd xhci-hcd.1.auto: URB req 32768 and actual 4294949904 transfer length mismatch
[ 489.505157] xhci-hcd xhci-hcd.1.auto: URB req 32768 and actual 4294964408 transfer length mismatch
[ 575.213815] ------------[ cut here ]------------
[ 575.218466] NETDEV WATCHDOG: eth3 (r8152): transmit queue 0 timed out
[ 575.225015] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:485 dev_watchdog+0x300/0x310
[ 575.233281] Modules linked in: fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine rng_core authenc libdes imx8_media_dev(C) secvio flexcan caam error can_dev crct10dif_ce fuse
[ 575.251259] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G C 5.10.52-lts-5.10.y+ga11753a89ec6 #1
[ 575.260826] Hardware name: NXP i.MX8MPlus EVK board (DT)
[ 575.266140] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--)
[ 575.272151] pc : dev_watchdog+0x300/0x310
[ 575.276163] lr : dev_watchdog+0x300/0x310
[ 575.280171] sp : ffff800011d4bd90
[ 575.283486] x29: ffff800011d4bd90 x28: ffff0000c7930280
[ 575.288805] x27: 0000000000000004 x26: ffff0000c7811480
[ 575.294125] x25: 0000000000000140 x24: 00000000ffffffff
[ 575.299440] x23: ffff0000c78113dc x22: 0000000000000000
[ 575.304758] x21: ffff800011b36000 x20: ffff0000c7811000
[ 575.310074] x19: 0000000000000000 x18: 0000000000000030
[ 575.315389] x17: 0000000000000000 x16: 0000000000000000
[ 575.320705] x15: ffff800011b427b8 x14: ffffffffffffffff
[ 575.326021] x13: ffff800011b51780 x12: 0000000000000771
[ 575.331334] x11: 000000000000027b x10: ffff800011ba9780
[ 575.336653] x9 : 00000000fffff000 x8 : ffff800011b51780
[ 575.341973] x7 : ffff800011ba9780 x6 : 0000000000000003
[ 575.347290] x5 : 0000000000000000 x4 : 0000000000000000
[ 575.352605] x3 : 0000000000000000 x2 : 0000000000000100
[ 575.357921] x1 : 5fe28801cf9bbe00 x0 : 0000000000000000
[ 575.363238] Call trace:
[ 575.365691] dev_watchdog+0x300/0x310
[ 575.369359] call_timer_fn.constprop.0+0x24/0x80
[ 575.373979] __run_timers.part.0+0x1f0/0x224
[ 575.378248] run_timer_softirq+0x3c/0x7c
[ 575.382173] efi_header_end+0x124/0x290
[ 575.386011] irq_exit+0xdc/0xfc
[ 575.389155] __handle_domain_irq+0x80/0xe0
[ 575.393255] gic_handle_irq+0xc0/0x140
[ 575.397005] el1_irq+0xcc/0x180
[ 575.400149] cpuidle_enter_state+0x130/0x2f0
[ 575.404424] cpuidle_enter+0x38/0x50
[ 575.408003] do_idle+0x218/0x2a0
[ 575.411233] cpu_startup_entry+0x28/0x70
[ 575.415158] rest_init+0xd8/0xe8
[ 575.418389] arch_call_rest_init+0x10/0x1c
[ 575.422489] start_kernel+0x4ac/0x4e4
[ 575.426152] ---[ end trace 5f4df7325a953a5e ]---
[ 575.430893] r8152 2-1.3:1.0 eth3: Tx timeout
[ 575.440589] r8152 2-1.3:1.0 eth3: Tx status -2
[ 575.447483] r8152 2-1.3:1.0 eth3: Tx status -2
[ 575.452828] r8152 2-1.3:1.0 eth3: Tx status -2
[ 575.458495] r8152 2-1.3:1.0 eth3: Tx status -2
[ 577.633164] usb 2-1.3: reset SuperSpeed Gen 1 USB device number 7 using xhci-hcd
[ 638.006098] Process accounting resumed
[ 640.471001] Process accounting resumed
one more
[ 1129.236927] xhci-hcd xhci-hcd.1.auto: xHCI host not responding to stop endpoint command.
[ 1129.245029] xhci-hcd xhci-hcd.1.auto: USBSTS:
[ 1129.257456] xhci-hcd xhci-hcd.1.auto: xHCI host controller not responding, assume dead
[ 1129.265566] xhci-hcd xhci-hcd.1.auto: HC died; cleaning up
[ 1129.271586] usb 1-1: USB disconnect, device number 2
[ 1129.276607] r8152 1-1.2:1.0 eth4: Stop submitting intr, status -108
[ 1129.276620] r8152 1-1.3:1.0 eth3: Stop submitting intr, status -108
[ 1129.276629] r8152 1-1.4:1.0 eth5: Stop submitting intr, status -108
[ 1129.276638] r8152 1-1.1.3.4:1.0 eth6: Stop submitting intr, status -108
[ 1129.278339] r8152 2-1.1.2.4:1.0 eth7: Tx status -108
[ 1129.278350] r8152 2-1.1.2.4:1.0 eth7: Tx status -108
[ 1129.278356] r8152 2-1.1.2.4:1.0 eth7: Tx status -108
[ 1129.278362] r8152 2-1.1.2.4:1.0 eth7: Tx status -108
[ 1129.324125] usb 1-1.1: USB disconnect, device number 3
[ 1129.329301] usb 1-1.1.2: USB disconnect, device number 5
[ 1129.335120] usb 2-1: USB disconnect, device number 2
[ 1129.340152] usb 2-1.1: USB disconnect, device number 3
[ 1129.345614] usb 1-1.1.3: USB disconnect, device number 7
[ 1129.350979] usb 2-1.1.1: USB disconnect, device number 4
[ 1129.356326] usb 1-1.1.3.4: USB disconnect, device number 9
[ 1129.362022] ax88179_178a 2-1.1.1:1.0 eth2: unregister 'ax88179_178a' usb-xhci-hcd.1.auto-1.1.1, ASIX AX88179 USB 3.0 Gigabit Ethernet
[ 1129.374848] br600: port 1(eth6) entered disabled state
[ 1129.380552] device eth6 left promiscuous mode
[ 1129.384983] kauditd_printk_skb: 5 callbacks suppressed
[ 1129.384985] audit: type=1700 audit(1641892634.982:19): dev=eth6 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 1129.401562] br600: port 1(eth6) entered disabled state
[ 1129.422861] r8152 2-1.1.2.4:1.0 eth7: Get ether addr fail
[ 1129.428308] r8152 2-1.1.2.4:1.0 eth7: Promiscuous mode enabled
[ 1129.435194] ax88179_178a 2-1.1.1:1.0 eth2: Failed to read reg index 0x0002: -19
[ 1129.442569] ax88179_178a 2-1.1.1:1.0 eth2: Failed to write reg index 0x0002: -19
[ 1129.450535] br200: port 1(eth2) entered disabled state
[ 1129.456200] device eth2 left promiscuous mode
[ 1129.460608] br200: port 1(eth2) entered disabled state
[ 1129.460918] audit: type=1700 audit(1641892635.058:20): dev=eth2 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 1129.526483] ax88179_178a 2-1.1.1:1.0 eth2 (unregistered): Failed to write reg index 0x0002: -19
[ 1129.535443] usb 1-1.2: USB disconnect, device number 4
[ 1129.540790] ax88179_178a 2-1.1.1:1.0 eth2 (unregistered): Failed to write reg index 0x0001: -19
[ 1129.549515] ax88179_178a 2-1.1.1:1.0 eth2 (unregistered): Failed to write reg index 0x0002: -19
[ 1129.559011] br400: port 1(eth4) entered disabled state
[ 1129.564794] device eth4 left promiscuous mode
[ 1129.569589] usb 2-1.1.2: USB disconnect, device number 5
[ 1129.574961] usb 2-1.1.2.4: USB disconnect, device number 7
[ 1129.580497] br400: port 1(eth4) entered disabled state
[ 1129.580509] audit: type=1700 audit(1641892635.178:21): dev=eth4 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 1129.617262] br700: port 1(eth7) entered disabled state
[ 1129.622916] device eth7 left promiscuous mode
[ 1129.627357] br700: port 1(eth7) entered disabled state
[ 1129.627472] audit: type=1700 audit(1641892635.222:22): dev=eth7 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 1129.710593] usb 1-1.3: USB disconnect, device number 6
[ 1129.716525] br300: port 1(eth3) entered disabled state
[ 1129.722375] device eth3 left promiscuous mode
[ 1129.727013] br300: port 1(eth3) entered disabled state
[ 1129.727027] audit: type=1700 audit(1641892635.322:23): dev=eth3 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 1129.733463] usb 2-1.1.3: USB disconnect, device number 6
[ 1129.825564] usb 1-1.4: USB disconnect, device number 8
[ 1129.831352] br500: port 1(eth5) entered disabled state
[ 1129.837089] device eth5 left promiscuous mode
[ 1129.841516] br500: port 1(eth5) entered disabled state
[ 1129.841536] audit: type=1700 audit(1641892635.438:24): dev=eth5 prom=0 old_prom=256 auid=4294967295 uid=0 gid=0 ses=4294967295
Stopping Session c1 of user root.
Hi @AlexeyK,
Well, we never tried this amount of USB devices connected to the same i.MX8M Plus and use it as ETH connections, as this is a very specific scenario we would try to reproduce by our side, so we will need some things from your side,
1. Could you share the specs of the USB hub you are using? The network adapter will be useful too.
2. The process you use to create the virtual network adapter on our board.
As for the moment, the only thing that comes to my mind is that the USB port is not delivering enough power to support all the HUB's you are trying to use, but as I mentioned before, we will test it.
Regards,
Israel.
Hi, here's the requested data
HUB: ATolla 207 USB 3.0 Hub (μPD720210) with external power supply, so the lack of power is probably out of the question
network adapter models:
Tripp-Lite USB to GigE adapter model U336-000-R
https://www.amazon.com/gp/product/B0871ZHCKK/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1
Cablecreation cd0026 USB to GigE
Script to create a virtual network adapter:
# go to configfs directory for USB gadgets
CONFIGFS_ROOT=/sys/kernel/config # adapt to your machine
cd "${CONFIGFS_ROOT}"/usb_gadget
# create gadget directory and enter it
mkdir g1
cd g1
# USB ids
echo 0x1d6b > idVendor
echo 0x104 > idProduct
# USB strings, optional
mkdir strings/0x409 # US English, others rarely seen
echo "ConDev" > strings/0x409/manufacturer
echo "ModemBoard" > strings/0x409/product
# create the (only) configuration
mkdir configs/c.1 # dot and number mandatory
fun=ncm
for i in $(seq 0 0); do
mkdir functions/$fun.usb$i
ln -s functions/$fun.usb$i/ configs/c.1/
done
# binding the gadget to an USB host controller
# ls /sys/class/udc to see available UDCs
# for the testing system we are using virtual UDC
echo $(ls /sys/class/udc/) > UDC