Head note to the reader, this is a hardware issue (we believe) and is a rare occurrence in a production level setting that we have seen multiple times.
Backstory: We utilize the FXOS8700 configured for I2C in a production device of ours. Our PCBs are manufactured and populated with a contract manufacturer, we have fielded three to four hundred of these devices and have experienced the following failure mode of the FXOS8700 in a handful of instances. We would like to know if NXP or any other end users of the FXOS8700 have experienced this same failure mode and if they have been able to determine the root cause of the issue.
Failure Mode Description: At some point in the devices life (whether it be days, months, or a years of normal operation) the FXOS8700 permanently drives the SDA line of the I2C bus to logic low (it is not fully driven to ground which is shown in scope captures below). The SDA line is stuck in logic low state regardless of what we do to the device. Our error handling code attempts to reset the FXOS8700 using the external reset pin, and we have even tried a full power cycle of the device by removing the power source and re-inserting it.
In order to verify that the FXOS8700 was indeed driving the SDA line low instead of the host processor, we obtained a device displaying this failure mode and cut the SDA trace past the pull-up resistor as shown in the schematic below:
We probed the FXOS8700 side of the pull-up resistor and it was being driven logic low by the FXOS8700 as shown in the scope captures below:
We then hooked a discrete resistor to VCC and probed the other side of the SDA line that is hooked up to the processor. This was done as an effort to prove that it was the FXOS8700 driving the signal low and not the processor. As seen in the below scope capture the line was pulled high by the pull-up resistor.
As stated above, this is not an issue that presents it self right away. If it were a manufacturing defect when populating the PCB, these devices would fail immediately instead of some time into their service life after normal operation. I am unsure what the problem might be, whether it could be mechanical stresses on the PCB, or if there is something that is breaking internally on these FXOS8700 devices after some time. Or something else.
Tomas,
I have returned with another data point. Since my last post in July, we have been running the PCB with the transplanted FXOS8700 in application. The other day the same failure mode was encountered. Again this is the same issue where a hard reset of the device, and attempted clock pulsing rescue will not recover the FXOS from it's failure state. I wrote a small program to attempt to rescue the device and shed a little more light on this issue but I have so far been unsuccessful. I am attaching the source code for this application in a zip file for anyone who comes across this post to look at.
I am somewhat tempted to send you the PCB displaying this problem for you to examine with your engineering team as I am unsure how much more we can do here to test it. The "bad" FXOS8700 was removed from it's original PCB and mounted on a brand new (problem free) PCB. After a few months of application the "bad" FXOS failed again. This leads me to believe that it is not so much the PCB as it is the FXOS it self.
Please see the attached application code and let me know if there is more that I can do to rescue the device from I2C lock up.
Hi Warren,
I have not heard of such a problem so far and I am not really sure what is causing that. I am afraid it is going to be difficult to identify a potential root cause since the issue appears sporadically and on a few devices only.
Based on your descriprion, is my understanding correct that even a hardware reset or a full power cycle does not release the SDA line?
One thing you can also try out is to send nine clock pulses as recommended in the UM10204, page 20:
„If the data line (SDA) is stuck LOW, the master should send nine clock pulses. The device that held the bus LOW should release it sometime within those nine clocks. If not, then use the HW reset or cycle power to clear the bus.“
I have one more question. In your previous thread, you were asking about the SPI mode. Are you considering using the SPI instead of I2C because of this issue?
Best regards,
Tomas
Hi Warren,
It sounds really strange. I have asked also my colleagues from our application team to look into it.
Meanwhile, could you please post here your complete schematic and let us know if you have tried to replace the FXOS8700CQ on a PCB having this issue? Could you please also try to use „the faulty“ part on a PCB without this issue? I want to figure out if it is a part or board problem.
Best regards,
Tomas
Tomas,
Okay we have successfully transplanted the FXOS8700 in question to a new PCB. To keep things simple these are the notations of this post:
Old PCB = The original PCB that had the original FXOS8700 on it, which this post is about.
New PCB = The new PCB that was shown to work fine, and the original FXOS8700 was transplanted to.
We took the old PCB and removed the FXOS8700 in question from it, we repaired the trace we cut and placed a brand new FXOS8700 on the old PCB. We booted it up and everything worked fine!
We took the original FXOS8700 that was removed from the old PCB and placed it on a brand new PCB that was shown to be functional, and.. it worked fine!
So, now we are at a loss as to what was happening, we checked and triple checked all the electrical connections to ensure there was no short to the ground plane, we tried resetting, clocking 9 clocks out, and hard resetting and none of that worked. As soon as we remove the part from the PCB and put it on a new PCB it works fine. And the old PCB works fine with a new part as well..
- Warren
Hi Tom,
Based on your descriprion, is my understanding correct that even a hardware reset or a full power cycle does not release the SDA line?
Yes, a full power cycle of the entire system does not release the SDA line. It should also be noted that the SDA pin is not shorted to ground (we did quite a lot of probing to make sure). It's being held at about 0.1 V.
If the data line (SDA) is stuck LOW, the master should send nine clock pulses. The device that held the bus LOW should release it sometime within those nine clocks. If not, then use the HW reset or cycle power to clear the bus.“
We do try an external reset, and then attempt to read the WHO_AM_I register, however we don't explicitly try to just pump out 9 clock cycles. That is something we can try here, however since a power cycle doesn't release the bus I am unsure if this will help.
I have one more question. In your previous thread, you were asking about the SPI mode. Are you considering using the SPI instead of I2C because of this issue?
The main reason we are looking to switch to SPI is the increased speed of the bus (400 kHz vs 1 MHz), and also less overhead / code. Internally we would like to understand exactly why this particular issue is happening regardless of if we make the switch to SPI in our next generation of boards.
I will try to clock out 9 cycles on the SCL line to see if we can release the bus and get back to you on it.
Thanks!
- Warren