topic Re: baremetal I2C bus lockup in Kinetis Microcontrollers

baremetal I2C bus lockup

alex323qp — Mon, 02 Nov 2020 13:27:18 GMT

Hi, I'm working on a bare metal I2C multimaster library for the MKL25Z4 and I'm having issues when trying to enumerate attached devices to the bus.

The process I follow is basically: START --> Write[address] --> STOP. By checking the ACK bit I know if there is a device listening or not.

It works from time to time but on most occasions the bus stays lock with both SDL and SCL lines low:

The picture above shows a device attached "listening" on address 0x03. See how the ACK is issued by the servant device, but the master seems unable to STOP the transmission. This doesn't seem to happen when sending actual data (START -->WRITE[address] --> BYTE[1] --> BYTE[2] --> ... --> STOP).

My Read method:

int read_byte(){
     _i2c_t.i2c->C1 &= ~I2C_C1_TX_MASK;                    // Set RX mode
     _i2c_t.i2c->C1 &= ~I2C_C1_TXAK_MASK;               // Enable ACK
     byte = (_i2c_t.i2c->D & 0xFF);                                   // Read
     wait_for_bit(I2C_S_IICIF_MASK);                              // Wait for the transfer to be over
     _i2c_t.i2c->S |= I2C_S_IICIF_MASK;                        // Clear out the interrupt flag
}

I'm not using interrupts so the servant device is continuously checking the status register in order to detect a transaction:

onTick(){
if(_i2c.i2c->S & (I2C_S_TCF_MASK | I2C_S_IAAS_MASK | I2C_S_BUSY_MASK | I2C_S_IICIF_MASK)){
// Master sending
while(_i2c_t.i2c->S & I2C_S_BUSY_MASK){  // Keep reading as long as the line is active
read_byte();
}
}
}

I have tried multiple things but the lines always end up locked (low). I even tried enabling the Glitch Filter just in case it was a noise issue, but it didn't work. Lines use 2k2 resistors to VCC.

Also, any advice on how to recover from this? I can detect it using the SMB->SHTF1 and SHTF2 registers, but so far the only way to do it is by hard resetting the chip.

Any help will be appreciated.

Regards,

Alex.

Re: baremetal I2C bus lockup

alex323qp — Mon, 02 Nov 2020 13:27:20 GMT

Ok, I don't really know what else to do at this point. I'm about to start praying to all known deities and probably offer my cat (will have to get one first) in sacrifice.

I implemented the following logic (not optimized, just exactly as described. I'm desperate here!) as suggested on the datasheet (p. 718). It runs in every clock tick.

// Clear flag
  _i2c_t.i2c->S |= I2C_S_IICIF_MASK;
  // Check if arbitration is lost
  if(_i2c_t.i2c->S & I2C_S_ARBL_MASK){
    _i2c_t.i2c->S |= I2C_S_ARBL_MASK;                  // Clear arbitration
    if(_i2c_t.i2c->S & I2C_S_IAAS_MASK){
      if(!(_i2c_t.i2c->S & I2C_S_SRW_MASK)){
        _i2c_t.i2c->C1 &= ~I2C_C1_TX_MASK; // Set Receiver mode
        data = (_i2c_t.i2c->D & 0xFF);                // Dummy read
      }else{
        // Master requesting?
        // Not interested right now...
      }
    }
  }else{
    if(_i2c_t.i2c->S & I2C_S_IAAS_MASK){
      if(!(_i2c_t.i2c->S & I2C_S_SRW_MASK)){
        _i2c_t.i2c->C1 &= ~I2C_C1_TX_MASK; // Set Receiver mode
        data = (_i2c_t.i2c->D & 0xFF);                // Dummy read
      }else{
        // Master requesting?
        // Not interested right now...
      }
    }else{
      // Read the actual data!
      data = (_i2c_t.i2c->D & 0xFF);
    }
  }

I'm sending 11 bytes per transaction and it seems to work fine for the first few:

(each "bar" is a transaction of 11 bytes).

Notice how at the end the bus remains locked. A closeup basically looks exactly as my previous post:

It's also worth mentioning that there is a 100ms spacing between transactions, and that if I increase this to 2000ms (2 seconds) it seems to work "fine" indefinitely.

Again, Any help will be appreciated.

Alex.

Re: baremetal I2C bus lockup

bobpaddock — Fri, 18 Dec 2015 16:48:22 GMT

If it works slower that points in the direction of wrong value for the bus pull up resistors.

Someone once did a thesis on how it is impossible to meet all of the I2C electrical specs with *any* value of pull up resistor.

Try lowering the pull up resistors to ~1.5k and see what happens. Linear Tech makes active pull-ups for I2C as well for the extreme need.

I2C is an idea that should have never happened in the world...

Re: baremetal I2C bus lockup

mjbcswitzerland — Fri, 18 Dec 2015 17:33:33 GMT

Multi-master I2C is something that I have always avoided after being involved with a system based on it in the past. The two wires may have been cheap in the HW but the cost of the problems that needed to be solved (devices that glitched, didn't fully meet specs. etc.) burned up the savings a thousand-fold...

One practical method of debugging is to connect (small-ish) serial resistors to each master's SDA/SCL lines, each with different values. Then it is possible to see which master is driving the bus (due to the voltage drop). By using a logic analyser with different logic thresholds set the I2C signal can then also be filtered.

I this case I think that it is nothing as complicated as that but I stick to I2C master and I2C slave roles because they are practical and reliable. When multi-master is required there are usually alternatives solutions that won't cause complications.

Regards

Mark

Re: baremetal I2C bus lockup

alex323qp — Mon, 02 Nov 2020 13:27:22 GMT

Hi Bob, thanks for the advice.

I tested with 1.5k resistors but the result is the same

Regarding Mark's comments, yes a multimaster implementation can be a pain in the back side but I'm afraid I haven't even got that point yet. Here I'm just testing a standard master-slave transaction and those are the only devices in the bus.

I'm not sure if this is a hardware issue, I tested with multiple FRDM-KL25Z boards as well custom ones, the result is the same. I also tried using other libraries (mbed's I2C) and even a SoftwareI2C master version, but the problem persists.

I forgot to mention that I connected the same devices to an i2c EEPROM memory and it works just fine! reading and writing multiple bytes, the problem seems to happen only when connecting 2 MKL25Z4 devices.

Something curious I noticed; if in the original code, I keep reeding in a second loop (line 30) the problem goes away, so I believe this could be a timing issue. The state machine in one of the 2 devices (or both) seems to go out of sync for some reason:

// This works! (replacing line 30)
while(true){
    data = (_i2c_t.i2c->D & 0xFF);
}

Any other ideas?

Thanks,

Alex.

Re: baremetal I2C bus lockup

mjbcswitzerland — Sat, 19 Dec 2015 18:11:52 GMT

Alex

I have master/slave I2C operation in the uTasker project ( http://www.utasker.com/ ) (for KL, K, KE, KEA, KV parts) - I just wouldn't do multimaster (unless someone was offering good incentives for it to be added because it would be horrible to have to support in case of practical issues).

KL25 was no problem - KL27, KL43 etc. (and others with double-buffered implementations) need different handling, although the migration application note states that they are SW compatible (which someone must have dreamt).

I have attached my development notes from KL25 to KL27 master/slave tests in both directions, with and without repeated starts. The expected status register values (and corresponding interrupts that they cause) are shown in the (3) diagrams. Maybe it will hep you find something.

The I2C master/slave operation is also simulated in the uTasker project (for single and double buffered devices) which is useful for stepping code to analyse the I2C workings and comparing to HW cases if there is any unexpected behavior.

Regards

Mark

P.S. It may just be that your slave is not releasing the bus since slaves do 'clock-stretching' - not all I2C slaves in the kinetis family support this but if the KL25 slave doesn't release the bus after being addressed (by clearing the "addressed as slave flag) it will stop any further bus activity (forever) and the master is also blocked.

Re: baremetal I2C bus lockup

bobpaddock — Mon, 21 Dec 2015 15:12:14 GMT

Mark, did you ever look at the FlexIO I2C option on the KL27? Repeated Start is implemented in the application note, AN5133, AN4955. Was wondering if it really worked (having lost faith in any Freescale documentation)?

Sadly my devices is not on FlexIO pins, so I did not even try. Next board rev will change pins to FlexIO pins.

Re: baremetal I2C bus lockup

mjbcswitzerland — Mon, 21 Dec 2015 15:41:22 GMT

Bob

I haven't used FlexIO yet.

Regards

Mark

Re: baremetal I2C bus lockup

Jorge_Gonzalez — Wed, 30 Dec 2015 23:07:37 GMT

Hello Alexander:

Do you still have this issue?

I checked your first code above, and I think it is not a good logic. Checking all status flags at once is not correct, since each flag has a different meaning. Also re-configuring the slave for each read is not always required. Then you used the recommended flow diagram of the Manual and although it is for interrupt mode it can also work by polling IICIF, but not if polling all flags at once. Typically as shown in the diagram you just poll IICIF (or enable interrupts) and then take action based on the flags set.

More specifically about the issue, when the KL25 I2C in slave mode receives a byte, it holds the SCL line low (clock stretching) until the I2C Data Register is read. This must be happening in your case, somehow your flag checks are overlapping and you are missing a read from the register, that is why it works reading the register in a loop.

I hope this helps. Let us know if any progress.

Regards!,
Jorge Gonzalez

Re: baremetal I2C bus lockup

alex323qp — Wed, 06 Jan 2016 21:26:11 GMT

Hi Jorge, thanks for your reply, and apologies to you and mjbcswitzerland for the delay in my response; I was on christmas holidays.

You are correct, the original code was poorly written, the updated version (second post) implemented the flow diagram from the datasheet.

The problem was basically what you indicated; the slave was holding the line low because the program was busy with another subroutine. The interesting part was that even after reading the data from the register (which I was expecting would release the line), the line remained low, it looks like the devices went out of sync or something. The solution was basically making sure the slave device was always available to read the register when a new byte arrived.

Though Mark's response helped me greatly to solve the issue, I'm marking yours as correct since clock stretching was the specific cause of my problem.

Thank you all for your help and happy (belated) new year :-)

Re: baremetal I2C bus lockup

mjbcswitzerland — Wed, 06 Jan 2016 23:00:56 GMT

Hi Alex

If you look at my post from 19th December you may find that I also had given the "correct answer" then.

Quote:

Regards

Mark

Re: baremetal I2C bus lockup

alex323qp — Fri, 08 Jan 2016 05:43:41 GMT

Mark, you're right, I'm really sorry.

After I read your post and checked the attachment, I got absorbed by it and spent the next few hours comparing your results with mine. I guess I completely missed that bit :smileyblush:.

I hope you don't mind leaving Jorge's response as the correct answer since it goes straight to the point? I believe it would help others find the solution more quickly.

Regards,