Possible recursive locking detected when driving iw611 in imx6ull

[Gloss]

Hi everyone,

    I got trouble when I was using an iMX6ULL #EVK Board which runing linux kernel5.4.70 to drive a sdio wifi6 module IW611.The picture shows how I connect module and EVK.
How XM553 connects to EVK
When I did insmod with mlan.ko and moal.ko,the possible recursive locking detected occured:

[10474.002926] mlan: loading out-of-tree module taints kernel.
[10474.149936] wlan: Loading MWLAN driver
[10474.155481] wlan: Register to Bus Driver...
[10474.170832] vendor=0x0471 device=0x0205 class=0 function=1
[10474.177198] Attach moal handle ops, card interface type: 0x109
[10474.188339] rps set to 0 from module param
[10474.194801] No module param cfg file specified
[10474.201592] SDIO: max_segs=128 max_seg_size=65535
[10474.206363] rx_work=0 cpu_num=1
[10474.213197] Attach mlan adapter operations.card_type is 0x109.
[10474.224106] wlan: Enable TX SG mode
[10474.227668] wlan: Enable RX SG mode
[10474.250191] Request firmware: nxp/sd_w61x_v1.bin.se
[10474.646393] Wlan: FW download over, firmwarelen=621088 downloaded 544888
[10476.100699] WLAN FW is active
[10476.103880] on_time is 10476084502206
[10476.140473] VDLL image: len=76200
[10476.145066] fw_cap_info=0x487cff03, dev_cap_mask=0xffffffff
[10476.154421] uuid: e84f49b3327c56d4b5166031890cfbfc
[10476.159274] max_p2p_conn = 8, max_sta_conn = 16
[10476.274754] Register NXP 802.11 Adapter mlan0
[10476.290557] wlan: uap%d set max_mtu 2000
[10476.328245] Register NXP 802.11 Adapter uap0
[10476.350479] wlan: version = SDIW612---18.99.2.p19.17-MM6X18398.p9-(FP92)
[10476.368101] Set REG 0x90002328: 0x13d57 slew_rate=3
[10476.437051] wlan: Register to Bus Driver Done
[10476.469751] wlan: Driver loaded successfully
[10477.489812] wlan: mlan0 START SCAN
[10483.518779] wlan: SCAN COMPLETED: scanned AP count=52
[10483.614522] wlan: HostMlme mlan0 send auth to bssid 44:XX:XX:XX:c6:bd
[10483.636752] mlan0:
[10483.636793] wlan: HostMlme Auth received from 44:XX:XX:XX:c6:bd
[10483.744330] wlan: HostMlme mlan0 Connected to bssid 44:XX:XX:XX:c6:bd successfully
[10483.843767] IPv6: ADDRCONF(NETDEV_CHANGE): mlan0: link becomes ready

[10516.303345] ============================================
[10516.308670] WARNING: possible recursive locking detected
[10516.313997] 5.4.70-gaba74e2 #1 Tainted: G O
[10516.319492] --------------------------------------------
[10516.324813] kworker/u3:1/555 is trying to acquire lock:
[10516.330050] d8edb2d0 (&(&mlock->lock)->rlock){....}, at: moal_spin_lock+0x10/0x1c [moal]
[10516.338527]
but task is already holding lock:
[10516.344373] d8f44a10 (&(&mlock->lock)->rlock){....}, at: moal_spin_lock+0x10/0x1c [moal]
[10516.352730]
other info that might help us debug this:
[10516.359267] Possible unsafe locking scenario:

[10516.365195] CPU0
[10516.367650] ----
[10516.370105] lock(&(&mlock->lock)->rlock);
[10516.374302] lock(&(&mlock->lock)->rlock);
[10516.378498]
*** DEADLOCK ***

[10516.384425] May be due to missing lock nesting notation

[10516.391224] 3 locks held by kworker/u3:1/555:
[10516.395589] #0: d8f25ca0 ((wq_completion)MOAL_WORK_QUEUE){+.+.}, at: process_one_work+0x1f4/0x824
[10516.404582] #1: d9aebf24 ((work_completion)(&handle->main_work)){+.+.}, at: process_one_work+0x1f4/0x824
[10516.414176] #2: d8f44a10 (&(&mlock->lock)->rlock){....}, at: moal_spin_lock+0x10/0x1c [moal]
[10516.422957]
stack backtrace:
[10516.427336] CPU: 0 PID: 555 Comm: kworker/u3:1 Tainted: G O 5.4.70-gaba74e2 #1
[10516.435871] Hardware name: Freescale i.MX6 Ultralite (Device Tree)
[10516.442273] Workqueue: MOAL_WORK_QUEUE woal_main_work_queue [moal]
[10516.448504] [<c01134f0>] (unwind_backtrace) from [<c010d1d0>] (show_stack+0x10/0x14)
[10516.456279] [<c010d1d0>] (show_stack) from [<c0dbd490>] (dump_stack+0xd4/0x108)
[10516.463614] [<c0dbd490>] (dump_stack) from [<c0193fe4>] (__lock_acquire+0x79c/0x23ec)
[10516.471465] [<c0193fe4>] (__lock_acquire) from [<c0196520>] (lock_acquire+0xcc/0x278)
[10516.479314] [<c0196520>] (lock_acquire) from [<c0ddf638>] (_raw_spin_lock_irqsave+0x40/0x54)
[10516.487985] [<c0ddf638>] (_raw_spin_lock_irqsave) from [<bf0bae58>] (moal_spin_lock+0x10/0x1c [moal])
[10516.497655] [<bf0bae58>] (moal_spin_lock [moal]) from [<bf009048>] (wlan_prepare_cmd+0x60/0x840 [mlan])
[10516.507353] [<bf009048>] (wlan_prepare_cmd [mlan]) from [<bf036a90>] (wlan_send_addba+0x120/0x154 [mlan])
[10516.517184] [<bf036a90>] (wlan_send_addba [mlan]) from [<bf026358>] (wlan_wmm_process_tx+0x12d8/0x1340 [mlan])
[10516.527445] [<bf026358>] (wlan_wmm_process_tx [mlan]) from [<bf002d7c>] (mlan_main_process+0x708/0xa78 [mlan])
[10516.537852] [<bf002d7c>] (mlan_main_process [mlan]) from [<bf09d9c0>] (woal_main_work_queue+0x9c/0x154 [moal])
[10516.548114] [<bf09d9c0>] (woal_main_work_queue [moal]) from [<c0159318>] (process_one_work+0x2a8/0x824)
[10516.557539] [<c0159318>] (process_one_work) from [<c01598c4>] (worker_thread+0x30/0x50c)
[10516.565652] [<c01598c4>] (worker_thread) from [<c01617bc>] (kthread+0x134/0x184)
[10516.573067] [<c01617bc>] (kthread) from [<c01010b4>] (ret_from_fork+0x14/0x20)
[10516.580302] Exception stack(0xd9aebfb0 to 0xd9aebff8)
[10516.585371] bfa0: 00000000 00000000 00000000 00000000
[10516.593565] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[10516.601756] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000

The clock of SDIO automatically set to 50MHz.And actually running at 44MHz.

Can anyone help me?If I should offer more information to further deal with this issue,please tell me.

[Christine_Li]

Hi， @Gloss 

Thanks for creating case to us and hope you are doing well.

I received your reported case and right now I am thinking this new release driver may not adaptive with this Linux Kernel version. But this is only my suspect point, right now I am also not sure. 

Because you are using our 6ULL and IW611, I think I can try to reproduce this issue.

And then if I can reproduce, then I will use another Linux kernel 6.1.36 to see what happened.

Once have any updates, I will keep you posted.

Best regards,

Christine.

[Christine_Li]

Hi, @Gloss 

I have finished cross-compiling the Wi-Fi driver based on I.MX6ULL with kernel 5.4.70.

Can you please let me know your loading Wi-Fi driver parameters or commands? So that I can try with same steps with you? I think we are close to the same environment.

Best regards,

Christine.

[Gloss]

Hi,@Christine_Li 
Thank you for your reply.This is my script which is used to mount the module,and than assciated with ap.
Best regards,
Gloss Tsai.

[Christine_Li]

Hi, @Gloss 

Thanks for providing me your mount_net.sh. 

I tested on my side with 6ULL+IW612+Linux kernel 5.4.70. Can not reproduce your issue on my side.

Please see attachment for my setting up and logs.

I want to confirm with you below queries:

1.Which dtb are you using? 

fdt_file=imx6ull-14x14-evk-btwifi.dtb  or  fdt_file=imx6ull-14x14-evk.dtb?

2. "iw mlan0 set power_save off" is it necessary for you? Can you please have a try without this command?

3.Have you replace the zImage with the one you compiled with Linux Kernel 5.4? Or you are using the one default in the pre-built image package of I.MX6ULL?

4.Please check your Hardware connections and configurations, such as the VIO and VSD_IO. On my side, I am using IW612-EVK, connect it with SDIO(SD card interface, not Micro-SD interface) and change Jumper J12 to 3.3V working voltage.

5. Please provide your /etc/wpa_supplicant.conf to check further.

6.Do you connect Antenna to your IW611? If no, can you please connect Antenna to have a try?

I worry, if tries more than 5 times to connect with AP, but still failed, there might have some problem.

7.Can you please have a try with sduart_nw61x_v1.bin.se instead of your sd_w61x_v1.bin.se?

Best regards,

Christine.

[Gloss]

1.我使用的不是NXP官方制作的评估板，使用的设备树应该不是这两者。当前wifi模块虽然警告递归锁死锁，但能够正常运行，没有什么明显的错误现象。请问这个设备树的选择会影响什么内容？

2.是的，关闭低功耗是有必要的。开启低功耗模式，会导致延迟不可控，最大达到4s，平均200ms不到。关闭低功耗模式，能控制最大延迟在100ms左右，平均8ms。

3.如下图

4.我使用的是micro-SD的接口。

5.如下
ctrl_interface=/var/run/wpa_supplicant GROUP=netdev
ctrl_interface_group=0
update_config=1
country=CN
#ap_scan=1
network={
key_mgmt=NONE
#ssid="HUAWEI-TEST"
ssid="Xiaomi_Test_5G"
#priority=1
}
network={
key_mgmt=NONE
ssid="HUAWEI-TEST"
#ssid="Xiaomi_Test_5G"
#priority=2
}

6.是的，有天线。不过是否有天线都能正常连接AP。

7.尝试了，不过结果是一样的。log在附件。顺便提醒您一下，possible recursive locking detected并不是挂载后立即发生的，有时需要等待一段时间。

[Christine_Li]

1.我使用的不是NXP官方制作的评估板，使用的设备树应该不是这两者。当前wifi模块虽然警告递归锁死锁，但能够正常运行，没有什么明显的错误现象。请问这个设备树的选择会影响什么内容？

我使用的不是NXP官方制作的评估板

==>您是指：I.MX 6ULL 吗？还是IW611？可以麻烦您提供一下详细信息吗？

请问这个设备树的选择会影响什么内容？

==》设备树的选择就是您配置的Device  tree，我之所以使用fdt_file=imx6ull-14x14-evk-btwifi.dtb 是因为，这个dtb里默认已经配置好识别SDIO接口的Wi-Fi 卡，我使用的是IW612的u-SD的模组，直接插到i.MX 6ULL的SD卡槽里即可直接使用。

如下是dts 里跟SDIO接口的关键部分，可以供您参考

{
modem_reset: modem-reset {
compatible = "gpio-reset";
reset-gpios = <&gpio_spi 4 GPIO_ACTIVE_LOW>;
reset-delay-us = <1000>;
#reset-cells = <0>;
};

usdhc1_pwrseq: usdhc1_pwrseq {
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_wifi>;
compatible = "mmc-pwrseq-simple";
reset-gpios = <&gpio5 1 GPIO_ACTIVE_LOW>;
};
};

&iomuxc {
pinctrl_wifi: wifigrp {
fsl,pins = <
MX6UL_PAD_SNVS_TAMPER1__GPIO5_IO01 0x03029
>;
};
};

&usdhc1 {
#address-cells = <1>;
#size-cells = <0>;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_usdhc1>;
no-1-8-v;
non-removable;
pm-ignore-notify;
mmc-pwrseq = <&usdhc1_pwrseq>;
cap-power-off-card;
/delete-property/ wakeup-source;
/delete-property/ enable-sdio-wakeup;

brcmf: bcrmf@1 {
reg = <1>;
compatible = "brcm,bcm4329-fmac";
};
};

2.可以麻烦您关闭低功耗，试一下，是否还会有类似报错吗？

3.看了您的wpa_supplicant.conf 没发现什么问题

Best regards，

Christine.

[Gloss]

您是指：I.MX 6ULL 吗？还是IW611？可以麻烦您提供一下详细信息吗？

==》IMX6ULL的评估板是创龙生产的，IW611模块是azurewave生产的XM553，模拟sd卡接口的PCB是我们自己设计的。我们自制的u-sd XM553也能够做到即插即用，估计设备树没有问题。

可以麻烦您关闭低功耗，试一下，是否还会有类似报错吗？

==》以前没有关闭低功耗的时候也是有出现过该错误。

Best regards，

Gloss.

[Christine_Li]

Hi, @Gloss 

1.请问您手上还有我们其他Wi-Fi Chip的module吗？能否帮忙试试，别的module是否也有类似问题？

2.这个只是一个Warning，6ULL+IW611 同样的driver 和 FW，在同样的Linux kernel版本上我们没法复现出来，可能比较难排查，我只能内部沟通一下，看是否还有其他的排查方法，或者问问这个warning是否要紧。

Best regards，

Christine.

[Christine_Li]

Hi, @Gloss 

非常抱歉，由于相同环境下我们无法复现您的问题，而且该问题仅是一个warning，如不影响您目前的使用，建议您先忽略该warning. 

如果后续发现影响您Wi-Fi 的正常功能，您再提交case，我们再来跟踪，好不好？

Best regards，

Christine.

[Gloss]

好的，感谢您的支持，我会在出现明显异常的时候重新提起该case。

[Christine_Li]

Hi, @Gloss 

好的，感谢理解。

如果后续出现明显异常，您可以重新提交一个新case。

Best regards,

Christine.

[Christine_Li]

Hi, @Gloss 

Forgot to remind you one point, please add some sleep time in your script after each command.

Because for some commands, such as insmod and wpa_supplicant connect command, it needs several seconds to finish.

Here give you an example for your reference:

=========

insmod mlan.ko
sleep 1
insmod moal.ko drv_mode=3 cal_data_cfg=none fw_name=nxp/sd_w61x_v1.bin.se
sleep 3
ifconfig mlan0 up

sleep 1
#iw mlan0 set power_save off
wpa_supplicant -i mlan0 -D nl80211 -c /etc/wpa_supplicant.conf -B
sleep 3
iw mlan0 info

sleep 1
udhcpc -i mlan0
sleep 3
ifconfig

============

Best regards,

Christine.