88W8997 module command timeout issue (interface PCIE+UART, Host: iMX8MQ) - reopen

[yao_feng]

The FW crashed issue is still exist in Generic_PCIE-WLAN-UART-BT-8997-LNX_6_6_3-IMX8-16.92.21.p119.2-16.92.21.p119.2-MM6X16437.P3-GPL.

follow previous case:

https://community.nxp.com/t5/Wireless-Connectivity/88W8997-module-command-timeout-issue-interface-PC...

and reopen by this case.  

@ArthurC
@Christine_Li
88W8997 

[Christine_Li]

Hi, @ArthurC 

Thanks for your feedback.

I checked your wifi_mod_para.conf and dmesg logs, it only include Wi-Fi part, not include Bluetooth part.

Are you using pcie8997_wlan_v4.bin and uartuart8997_bt_v4.bin in parallel?

Yes, you are right, our default baudrate is 115200, not 3M, May I know how do you load Bluetooth driver and FW? and how to change from 115200 to 3M?

Yesterday, I tried with combo FW and BT working in 115200 baud rate and Wi-Fi working in 2.4G STA mode to do throughput iperf test, didn't reproduce.

After your remind, I prefer to do same test as your test environment.

Best regards

Christine.

[Christine_Li]

Hi, @ArthurC 

Just to double confirm, in our Q1-24 release, it includes Wi-Fi only FW(pcie8997_wlan_v4.bin) and BT only FW(uartuart8997_bt_v4.bin).

You only upgrade Wi-Fi only FW from previous one(16.92.21.p84.4) to Q1-24(16.92.21.p119.2), and still using the old BT only FW(16.92.21.p84.4),Am I right?

Best regards,

Christine

[ArthurC]

Hello @Christine_Li ,

No, we use Wi-Fi & BT FW Both in Q1-24(16.92.21.p119.2) and report issue.

We means the Wi-Fi test firmware + BT  FW(16.92.21.p84.4) working well with tiny BT disconnections before.

[Christine_Li]

Hi, @ArthurC 

Can you please help to try with below combination:

Q1-24 WiFi only firmware(16.92.21.p119.2) + BT old FW(16.92.21.p84.4) ?

We prefer to know whether this combination is OK and narrow down the root cause.

At the same time, as we talked in Teams meeting, I will continue to test on my local side:

1.more test based on combo FW in Linux kernel 5.15.71 as shown to you

2.have a try with parallel FW in Linux kernel 6.1.22 (Wi-Fi & BT FW Both in Q1-24(16.92.21.p119.2))

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

After test the combination about Q1-24 WiFi only firmware(16.92.21.p119.2) + BT old FW(16.92.21.p84.4),

the result is negative. 

The issue is still occured.

[Christine_Li]

Hi, @ArthurC 

From the shared logs we can see the "Timeout cmd id (250.844023) = 0x107" and we can see you are using the older driver "MM5X16366.p5"

Can you please try with below latest driver+FW combination once it will helps to narrow down the issue

FW version = 16.92.21.p119.2

Driver version = MM6X16437.P3-GPL

Meanwhile we are also in-process to recreate the issue locally.

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

Please help to provide the version git tag about Driver version = MM6X16437.P3-GPL in GitHub - nxp-imx/mwifiex: WiFi extensions we will built-in and test here.

[Christine_Li]

Hi, @ArthurC 

Please use this tag: lf-6.6.3_1.0.0

This Driver version = MM6X16437.P3-GPL is built based on LNX_6_6_3.

Best regards,

Christine.

[Christine_Li]

Hi, @ArthurC 

We have tried to validate the issue with/without drvdbg=0xa0037 and we are not able faced the device hang and command timeout 0x107 issue.

Can you reproduce this issue based on our latest Driver:  Driver version = MM6X16437.P3-GPL and latest FW: FW version = 16.92.21.p119.2  ?

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li ,

Yes, we will test Driver version = MM6X16437.P3-GPL and latest FW: FW version = 16.92.21.p119.2.

It is ok in previous test, Driver version = MM6X16437.P3-GPL and latest FW: FW version = debug test.

[Christine_Li]

Hi, @ArthurC 

Thanks for your feedback.

From your test results, I can doubt that the issue is from our FW side, not driver side.

But please also help to verify the combination of :

Driver version = MM6X16437.P3-GPL and latest FW: FW version = 16.92.21.p119.2. 

to verify our suspect and narrow down root cause.

I will feedback your latest test results to our internal team.

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

The tast result about  Driver version = MM6X16437.P3-GPL and latest FW: FW version = 16.92.21.p119.2.  is still negative.

The dmesg log is as attached.

Looks like the FW is in unexspectable state caused driver running in abnormal state.

The iperf3 test connection is broken.

[Christine_Li]

Hi, @ArthurC 

Thanks for providing the test results.

But from the given logs, I saw you were still using this driver: MM5X16366.p5

[ 212.043821] wlan: version = PCIE8997--16.92.21.p119.2-MM5X16366.p5-GPL-(FP92)

Can you please help to double confirm?

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

Yes, it is latest driver, "lf-6.6.3_1.0.0" .

We built driver by nxp-wlan-sdk in yocto. 

We modified the driver source branch to be "lf-6.6.3_1.0.0" & proper SRCREV.

But we don't know why the version string is not updated.

We truly found the driver action is different.

[Christine_Li]

Hi, @ArthurC 

Just for your information, to eliminate your confusion and considerations about whether is a new issue on the Q1-2024 official release. I checked again the original logs in previous thread and our Q1-2024 official release note, also checked with our internal team.

We confirmed that now is a new issue.

Previous logs in the "1" folder you provided, shows that it is a wakeup card timeout issue.

Our Q1-2024 release note:

Now is a "Timeout cmd id (250.844023) = 0x107"

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

Thank you for your help.

We looking forward the bug fixed version firmware.

[Christine_Li]

Hi, @ArthurC 

1. Is there any updates on the test result of combination of :

latest Driver: Driver version = MM6X16437.P3-GPL

latest FW: FW version = 16.92.21.p119.2?

2.About the system hang issue with drvdbg=0xa0037 or drvdbg=0x20037, our internal team is observing the "ddrc freq set to high bus mode"/"ddrc freq set to low bus mode" kind of error during traffic running on i.MX8MQ while device is hang, Can you please confirm at your end you are also facing similar error? Then we can say this will be the same issue we seen locally. 

Because from the shared system hang logs, I am not able to see this kind of errors.

3.Regarding Dump files:

 From the test if you observed the "Timeout cmd id (250.844023) = 0x107" issue with/without drvdbg, you can take a manual dump using the below commands.

- echo "debug_dump" >/proc/mwlan/adapter0/config

- cat /proc/mwlan/adapter0/drv_dump > file_drv_dump

- cat /proc/mwlan/adapter0/fw_dump > file_fw_dump

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

3.Regarding Dump files:

 From the test if you observed the "Timeout cmd id (250.844023) = 0x107" issue with/without drvdbg, you can take a manual dump using the below commands.

- echo "debug_dump" >/proc/mwlan/adapter0/config

- cat /proc/mwlan/adapter0/drv_dump > file_drv_dump

- cat /proc/mwlan/adapter0/fw_dump > file_fw_dump

-> Files are as attached.

[Christine_Li]

Hi, @ArthurC 

Thanks for your quick reply.

So this attached logs is based on Driver version = MM6X16437.P3-GPL +  FW version = 16.92.21.p119.2, right?

At least, from your side, you think it should be Driver version = MM6X16437.P3-GPL + FW version = 16.92.21.p119.2, based on Linux 6.6.3. Right?

Best regards,

Christine.

[ArthurC]

Hello @Christine_Li,

So this attached logs is based on Driver version = MM6X16437.P3-GPL +  FW version = 16.92.21.p119.2, right?

-> Yes.

At least, from your side, you think it should be Driver version = MM6X16437.P3-GPL + FW version = 16.92.21.p119.2, based on Linux 6.6.3. Right?

-> Yes, BTW, the driver source is version Linux 6.6.3. but our working OS is still in Yocto Linux 5.15.71.

[Christine_Li]

Hi, @ArthurC 

OK. Got it.

Thanks for your clarification.

We will analyze the current logs, try to find something through current logs.

Once have any updates, I will let you know.

Best regards,

Christine.