I2C device dead-lock recovery

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

I2C device dead-lock recovery

17,507 Views
mjbcswitzerland
Specialist V

Hi All

 

There are a couple of posts and/or blogs that look to be related to this so I thought I would report some of my own details.

 

While working on a FRDM-KL25Z and continuously reading the accelerometer data via I2C bus I noticed that when powering the board it was always reliable, but when resetting the board (commanded reset or reset button) there was quite a high chance that the I2C bus would be blocked.

 

The reason for this is due to the fact that the accelerometer reading (4 bytes) is taking place using continuous repeated-start mode and the slave device is practically sending an acknowledgement about 15% of the time. If the reset button is pressed during the time that the accelerometer is sending the I2C ACK it will block in that state and continue driving the SDA line. After the reset is is not possible to use the I2C bus since it is continuously in the busy state and further resets of the processor don't help - a power cycle does however since it resets the slave device.

 

In the mentioned case the risk of this state was very high and so a reliable solution was investigated that would automatically detect the state and resolve.

 

The solution is as follows:

- rather than configure the I2C pins when the I2C interface is initialised they are left as inputs with pull-ups enabled

- the first time that the I2C bus is to be used for transmission (transmission takes place as first operation of any read or write) the bus state is checked.

- if it is detected in the busy state, the SCL pin is set to an output and clocks generated until the busy state no longer exists (the clocks remove the slave from its present ACK state - several clocks may be required)

- the pins are then set to their final I2C peripheral usage

- once the pins are set no further checking is required for subsequent I2C use and normal interrupt or DMA driven modes are possible

 

The result was that, although the I2C slave was still holding the bus in about 15% of reset cases, the automatic detection/recovery reliably resolved it with no user code intervention.

 

This is now included as integral part of the uTasker I2C driver code - for reference, the detection/recovery code for the pins as used by the accelerometer on the KL25 freedom board is show below.

 

In case of interest in trying this, I have attached a binary for the FRDM-KL25Z which continuously reads the accelerometer via I2C - no pauses). If the SDA line is measured when the reset button is held down it will be seen that the bus busy state is reprocuced often - subsequent recovery is 100%. The accelerometer values are printed to the UART (via OpenSDA virtual COM) ever 100th reading (about 4x per second).

 

Regards

 

Mark

 

Pin configuration during I2C initialisation is restricted to ensuring they are inputs:

_CONFIG_PORT_INPUT_FAST_HIGH(E, (PORTE_BIT25 | PORTE_BIT24), (PORT_ODE | PORT_PS_UP_ENABLE));

 

On first use the following code checks the bus state and recovers if neede, followed by setting the final peripheral pin functions:

 

while (_READ_PORT_MASK(E, PORTE_BIT25) == 0) {   // if the SDA line is low we clock the SCL line to free it

    _CONFIG_DRIVE_PORT_OUTPUT_VALUE_FAST_HIGH(E, PORTE_BIT24, 0, (PORT_ODE | PORT_PS_UP_ENABLE)); // set output '0'

    fnDelayLoop(10);

    _CONFIG_PORT_INPUT_FAST_HIGH(E, PORTE_BIT24, (PORT_ODE | PORT_PS_UP_ENABLE));

    fnDelayLoop(10);

}

_CONFIG_PERIPHERAL(E, 25, (PE_25_I2C0_SDA | PORT_ODE | PORT_PS_UP_ENABLE)); // I2C0_SDA on PE25 (alt. function 5)

_CONFIG_PERIPHERAL(E, 24, (PE_24_I2C0_SCL | PORT_ODE | PORT_PS_UP_ENABLE)); // I2C0_SCL on PE24 (alt. function 5)

Original Attachment has been moved to: FRDM_KL25Z_ACCELEROMETER_I2C_RECOVERY.zip

Tags (2)
5 Replies

4,353 Views
jeremie_chirat
Contributor II

Hi Mark,

 

Indeed we use NXP SDK for S3241XX with i2c_pal /LPI2C libraries and I could not find the feature.

Fortunately we found the bug in-house early and not the client (even if the real-life scenario for the bug had a low probability of happening, when it happens late in production phase it can be a real pain...).

And as our use-case is automotive so, asking the client to power-cycle is problematic (unplugging the car battery).

 

Again, thanks for sharing your experience.

 

Jérémie

0 Kudos

4,371 Views
jeremie_chirat
Contributor II

Hi,

The post is old but still relevant so I share my experience.

Same bug happened to me on a S32K.

We haven't yet discovered the exact slave device that creates this situation. We are checking that no I2C transfer is active before resetting, but the situation still occurs on some resets.

However the fix presented by @mjbcswitzerland works reliably, even with aggressive tests resetting continuously the board.

Thanks,

Jérémie

0 Kudos

4,360 Views
mjbcswitzerland
Specialist V

Hi Jérémie

The post is old but the problem is inherent in all systems using I2C and slaves that don't have their own reset line (the problem doesn't go away with time

Products developed without dead-lock recovery will ALWAYS fail if tested properly or their developers have decided to simply accept that their customers need to power cycle their equipment when it happens rather than solving it.

Congratulations to you for actually doing the work to test and solve the typical I2C weakness which is inherent when not done properly (typical I2C library drivers often miss this basic feature).

Regards

Mark
[uTasker project developer for Kinetis and i.MX RT]
Contact me by personal message or on the uTasker web site to discuss professional training, solutions to problems or rapid product development requirements

For professionals searching for faster, problem-free Kinetis and i.MX RT 10xx developments the uTasker project holds the key: https://www.utasker.com/kinetis/FRDM-K32L2A4S.html

0 Kudos

3,569 Views
LukeJokkim
Contributor I

Hi @mjbcswitzerland , I am currently facing the I2C Bus Busy issue with the micro S32R294 (connected to PMIC via I2C), i would like to try the approach that you had mentioned. could you please share the original attachment that you had on the first post. The link to the attachment seems to be broken. 

Thanks in Advance,

Luke

Tags (1)
0 Kudos

3,560 Views
mjbcswitzerland
Specialist V

Hi

I don't know what was attached at the time (it was 7 years ago) but I have shared the uTasker I2C (valid for all Kinetis single or double buffered I2C impementations) and LPI2C (valid for Kinetis and i.MX RT LPI2C implementations) drivers attached.

These first configure the I2C pins as GPIO and check the bus state the first time the bus is used, clears the bus if needed and then sets to I2C mode.

Regards

Mark

0 Kudos