88W8997 module command timeout issue (interface PCIE+UART, Host: iMX8MQ) - reopen

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

88W8997 module command timeout issue (interface PCIE+UART, Host: iMX8MQ) - reopen

25,652 Views
yao_feng
Contributor III

The FW crashed issue is still exist in Generic_PCIE-WLAN-UART-BT-8997-LNX_6_6_3-IMX8-16.92.21.p119.2-16.92.21.p119.2-MM6X16437.P3-GPL.

follow previous case:

https://community.nxp.com/t5/Wireless-Connectivity/88W8997-module-command-timeout-issue-interface-PC...

and reopen by this case.  

@ArthurC
@Christine_Li
88W8997 

0 Kudos
Reply
169 Replies

5,906 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

I received our internal team's feedback as below:

========

From the  latest comment i can understand that the customer see the issue with below combinations.

Driver version = MM6X16437.P3-GPL

FW version = 16.92.21.p119.2

From the shared customer logs i can see the customer is still using "MM5X16366.p5" older driver, If dmesg logs shows the driver print then it will be confirm may be customer is using the older driver version, it may happen due to their yocto setup discrepancy.

But still i will check at my end and double confirm with the results.

==========

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

5,901 Views
ArthurC
Contributor III

Hello @Christine_Li ,

 

Is the firmware crashed by driver?

 

Is there any change list between driver in Version "MM6X16437.P3-GPL" & "MM5X16366.p5" ?

 

What's the critical cause about issue occured?

0 Kudos
Reply

5,896 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

The root cause of this issue we are still under tracking.

I am not sure it is from driver side or FW side.

We will share with you once we have any updates.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

5,863 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

We tried by check out branch lf-6.6.3_1.0.0 from https://github.com/nxp-imx/mwifiex and we verified locally we are getting correct FW and driver version.

 

wlan: version = PCIE8997--16.92.21.p119.2-MM6X16437.p3-GPL-(FP92)

 

Can you please double check with latest driver and firmware?

And for sure, at the same time, I will push our internal team to output the analysis of current provided logs.

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

5,452 Views
ArthurC
Contributor III

Hello @Christine_Li ,

The system hang caused by 88W8997 is still occured with drvdbg=0xa0037 in PCIE8997--16.92.21.p119.2-MM6X16437.p3-GPL-(FP92).

 

Please find attached dmesg log. it recorded until system hang. Working a live about 1000 ms only.

 

Tags (1)
0 Kudos
Reply

5,443 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

Thanks for your feedback.

I have forwarded this results to our internal team.

But at the same time, can we firstly remove this parameter(drvdbg=0xa0037) and only verify the original "Timeout cmd id (250.844023) = 0x107" issue?

I know you might have the concern that once "Timeout cmd id (250.844023) = 0x107" occur, our internal team will still request to provide full dmesg logs with this parameter(drvdbg=0xa0037). I also have same concern with you.But it is better we focus on the original issue firstly.

At the same time, I will also discuss with our internal team about the system hang issue.

Is it ok for you to continue the test without drvdbg=0xa0037?

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

5,437 Views
ArthurC
Contributor III

Hello @Christine_Li,

 

Yes, sure.

We are on testing without parameter(drvdbg=0xa0037), currently.

0 Kudos
Reply

1,642 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

Based on your verification, Q2-24 release + driver patch(for 0x107 scan timeout issue), there is no FW crash issue(because of 0x107 scan timeout error) occurred.
Shared driver patch will be included in our Q3-24 MM official release.

According to our communication through email, can we close this case for now and track the disconnection issue on a new case?

We are always recommended to track one issue through one case according to our company's policy. Also for better future reference.


Yao will hep to create a new case to continue tracking the mentioned disconnection issue(based on Q2-24 release + driver patch) after you agreed.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

1,656 Views
ArthurC
Contributor III

Hello @Christine_Li

 

As the following we tested:

 

1. The 88W8997 working 2.4G Hz only with NetworkManager (nmcli) is stable for iperf3 test over 1 month no disconection occured.

 

2. The 88W8997 working 5G Hz only with NetworkManager (nmcli) is stable for iperf3 test over 1 month no disconection occured.

 

3. The 88W8997 working 5G Hz  & 3M Bluetooth A2DP audio playing with NetworkManager (nmcli) is stable for iperf3 test over 1 month no disconection occured.

 

4. The 88W8997 working 2.4G Hz  & 3M Bluetooth A2DP audio playing with NetworkManager (nmcli) is unstable for iperf3 test disconection occured.

 

With these result we don't think there is issue caused on NetworkManager (nmcli).

 

If there is SW compatiable issue for 88W8997 module with NetworkManager (nmcli).

 

Is 88W8997 module working well on any platform with UBUNTU system?

 

Because UBUNTU is using NetworkManager (nmcli).

 

0 Kudos
Reply

1,673 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

For 0x107 command timeout

  • Based on the your feedback we can conclude the issue 0x107 cmd timeout is resolved with Q2-24+shared test PATCH. Because with default driver load parameters, there is no FW crash issue, only meet disconnection and reconnection issue.

 

For disconnection issue

  • We already explained this is due to the nmcli not because of the NXP DRIVER/FW, Please refer below image .
  • New Activation request from networkmanager caused the disconnect.
  • So driver had triggered the disconnection as per the command received from networkmanager(nmcli).

 But above is based on previous logs on Q1-24 release, for the new Q2-24 release + patch verification, we need related logs to confirm whether is same as before.

 

Suggestion to the you for disconnection issue

  • We suggest to test your scenario with the open source wpa_supplicant, instead of nmcli.
  • Because the NXP official release(Q2-24) is already tested with wpa_supplicant, and we could not observed any disconnection issue.
  • I understand that in your project you do not want to replace nmcli with wpa_supplicant, but this suggestion is only used for debug purpose.

About your request of sharing driver patch only includes useful logs, not print too many logs

  • I discussed with our internal team, currently we could not share this kind of driver patch because we need to firstly check the logs based on Q2-24 release + patch and see why have this disconnection issue. Otherwise, it is difficult for us to know which part of logs we need to add in driver patch. Hope you can understand our point.
  • If there is still some confusion or misunderstandings, please feel free to contact me through Microsoft Teams(christine.li@nxp.com) for quick communication.

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

1,827 Views
ArthurC
Contributor III

Hello @Christine_Li,

 

As our log showing, there is disconnect occured & re-connection showing in nmcli logs because the Wi-Fi communcation is broken & nmcli will do reconnection by itself.

 

The problem is that why driver make nmcli judging Wi-Fi communcation broken, isn't it?

 

And there is no any log print in driver about 2.4G Hz Wi-Fi working with Bluetooth,  isn't it?

 

So, it is not making sense about based on current nmcli log & dmesg to judge the issue caused from nmcli.

 

And it is waste our time to keep providing the useless logs without any key log printing changes in driver.

 

Please help the provide other driver debug log print change patch to analysis.

 

The mass raw printing will cause our system broken. 

 

0 Kudos
Reply

1,844 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi,  @ArthurC 

Thanks for your feedback. Please see below inline replies:

Why is NXP making the auto_fw_reload enabled(default)? 

[Christine]: This parameter is used when firmware crashed if meet some unknown errors, and works as a reset strategy so that Wi-Fi can continue to work.

 

Is there known expected issue to NXP's product needed to re-init the FW?

[Christine]:If there is any known issue, we will fix it in the newer firmware and mentioned in our release note.

 

We didn't do any nmcli command to reconnect during iperf3 test. Why should we do this thing to break our test? 

[Christine]:I can understand that this action from nmcli to do disconnect and reconnect is not your expected behavior, or not do it deliberately. But from given logs, it shows that this is from nmcli side. Sometimes some app will do some thing in the background or be effected by some thing else. 

 

How do NXP make sure NXP's driver & FW is OK? 

[Christine]: There might be some misunderstandings between us. We are not saying that NXP's driver & FW is totally OK without any issue. Just mentioned, the disconnected and reconnection issue is not from NXP's driver and FW side, and we concluded this from previous given logs. But we admit that the scan timeout 0x107 error is from NXP side, so we request you to test with our new release + our patch. And for the new release + patch, we did test locally, we didn't see this scan timeout 0x107 error.

 

Why the latest driver is more stable than previous one? 

[Christine]: We always fix some issues reported by our internal test team or reported by some of our customer in the newer release. Or add some features in the new release. So in theory, newer release is more recommended to customer. 

 

Why issue will not be occurred in 5G Hz Wi-Fi? 

[Christine]: It should because BT is working in 2.4G Hz.  But we need logs to show where we have to improve in 2.4G Hz.

 

Why issue is not occurred in bcm4356a2?

[Christine]:  I can understand your situation and your concerns. With same test environment and same test steps, issue is not seen on other products. But to identify why it doesn't see on other products and why it only happens on our product, we need logs to compare. We suspect there is not same behavior from nmcli even you do same thing in front, but there might some thing else effect nmcli to have different behaviors.   

 

Is this NXP's service quality?

[Christine]: I have to say sorry and apologize if my support is not satisfied you. But as mentioned several times, we need logs to find evidence to show it is our issue then we can do something from our side. I can also understand that there are so many things to improve on our products, but we have to improve them through comparing and logs. Also thank you so much for the long-term corporation on this case. If I am a customer using this product, this kind of using experience can also not make me satisfied, but please also understand our situation, without logs showing, we will stuck here.

 

So, please help on below request:

1. Are you still continuing the test or you have stopped the test with Q2-24 release + patch?

2.You mentioned our latest release is more stable than previous one, do you mean only the latest release? or with our patch? Did you also test without patch?

3.Can you please provide us logs based on Q2-24 release + patch? So that we can confirm:

a) Our patch worked or not? 

b)Identify whether the disconnection and reconnection is because of nmcli's new activation.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

1,875 Views
ArthurC
Contributor III

Hello @Christine_Li,

 

Why is NXP making the auto_fw_reload enabled(default)? 

Is there known expected issue to NXP's product needed to re-init the FW?

 

We didn't do any nmcli command to reconnect during iperf3 test.

Why shoud we do this thing to break our test?

How do NXP make sure NXP's driver & FW is OK?

Why the latest driver is more stable then previous one?

Why issue will not be occured in 5G Hz Wi-Fi?

Why issue is not occured in bcm4356a2 ?

 

Is ths NXP's service quailty?

 

 

 

 

0 Kudos
Reply

1,981 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

Please find the inline response of your feedback.

[Gigabyte]
With probe parameter about drvdbg=0xa0037 and auto_fw_reload=0 both, our system will be hang every time.
[NXP]
As your usecase is with auto_fw_reload enabled(default) so auto_fw_reload=0 is not the your usecase and it is for debugging purpose only. Please let us know if any hang observed with your usecase scenario.

 

[Gigabyte]
We probe without any parameter, the 88W8997 is alive about Bluetooth 3M A2DP Audio play & 2.4 G Wi-Fi working in the same time now, but there are continuously disconnections & reconnections occured in 2.4G Wi-Fi during iperf3 test. 
[NXP]
For the disconnection, please help to provide us related logs.  We suspect  it might be same reason with the below image.

Christine_Li_0-1721788477618.png

 

Can you please confirm if you are facing the same error like above image when disconnection is observed, If yes as already concluded this due to NetworkManager issued Disconnect to Supplicant, reason highlighted that new Activation request from Network manager.

New Activation request will cause disconnect and reconnect and this disconnect is not triggered from NXP side DRV or FW side.

Can you please check if you have issued a new Activation on the same Network using nmcli? Please check it in your nmcli path.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

1,990 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

Thanks for your feedback.

Can you please provide me related logs for further checking the disconnection and reconnection issue?

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

2,005 Views
ArthurC
Contributor III

Hello @Christine_Li,

 

We tested latest release Q2-24 (https://www.nxp.com/webapp/sps/download/license.jsp?colCode=GENPCIE16.92.21.p119.3MM6X16437P21GPL&ap...) with patch applied yesterday.

 

With probe parameter about drvdbg=0xa0037 and auto_fw_reload=0 both, our system will be hang evety time.

 

We probe without any parameter, the 88W8997 is alive about Bluetooth 3M A2DP Audio play & 2.4 G Wi-Fi working in the same time now, but there are continuously disconnections & reconnections occured in 2.4G Wi-Fi during iperf3 test

 

Why is the 88W8997's signal too weak to keep 2.4G connection stable when Bluetooth 3M A2DP audio playing in the same time?

 

Is there any method to improve the Wi-Fi connection stability about 88W8997 2.4G Wi-Fi?

0 Kudos
Reply

2,054 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

Please check your email, Yao has shared the download link to you.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

2,058 Views
ArthurC
Contributor III
0 Kudos
Reply

2,064 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

I saw in your demsg logs show that you are still using Q1-24 release: PCIE8997--16.92.21.p119.2-MM6X16437.p3-(FP92)

Our suggestion is: please switch to latest release Q2-24 (https://www.nxp.com/webapp/sps/download/license.jsp?colCode=GENPCIE16.92.21.p119.3MM6X16437P21GPL&ap...) and apply the patch on top of Q2-24 release. It is recommended to always use the latest-release for stability.

 

Can you please help us to  understand how you start Soft AP on uap0, if nmcli creating two instances(mlan0 and uap0) of wpa_supplicant?

Also, Please help to monitor if you see error “cmd 0x107 error, result=0x4” during test, after this error our handling will work.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply

2,071 Views
ArthurC
Contributor III

Hello @Christine_Li,

 

This community was blocked my permission about post reply last week...

 

The patch applied to the driver but the result is negative, the 88W8997 module is still enter abnormal status without any response...

 

 In this test, there is no driver probe parameter in initial for common test.

 

So there is no fw_dump working.

 

Please face this issue.

 

Thank you.

0 Kudos
Reply

2,129 Views
Christine_Li
NXP TechSupport
NXP TechSupport

Hi, @ArthurC 

We have shared our patch based on latest release for one week.

Do you have any updates? Did you get any chance to start the verification test?

Please let us know your result.

I just worry case will be automatically closed because it is answered back status for several days.

 

Best regards,

Christine.

Tags (1)
0 Kudos
Reply