I²C malfunction/freezing?

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Sun Mar 30 04:32:19 MST 2014
Hello,
there is a strange and very confusing/frustrating problem with the LPC812 I²C bus. I kindly ask for help in avoiding, bypassing or recovering from such problems:

I connect an MS5611 as the only I²C-slave to the LPC812 and I use PIO0_13 as SCL and PIO0_12 as SDA, the distances between the chips are <3cm in my hardware and about 5cm in the LPCXpresso prototype boards, I use external 2k2 resistors.
The I²C-clock is about 400 kHz - earlier it was about 90 kHz as long as I followed the advices in the user manual, table 192: 12 MHz Xtal, 24 MHz clock, LPC_I2C->DIV= 15, now I use LPC_I2C->DIV= 3 and you can see the I²C clock in the sniffer output. But clocking is not the current problem.

The transactions consist of 2-byte command sequences (address and 1 byte command) and of sequences over 2 or 3 bytes to be read (address and 2 or 3 bytes). Several commands and data reads are executed one immediately after the other as a sequence and then a pause of about 10 ms ist inserted. A typical sequence is:
START <address write (EE)> <command read ADC (00)> STOP followed immediately by
START <address read (EF)> <read byte 1> <read byte 2> <read byte 3> NAK & STOP followed immediately by
START <address write (EE)> <command measure temperature> STOP and then about 10 ms pause for the MS5611 to do what being asked for.

After RESET and start this works fine for a while (some times 20 seconds, other times up to 5 minutes or even longer after the clock was changed to 400 kHz) and then the I²C-Bus comes out of order and is stuck.

The attachment contains a logic sniffer display of a wrong going transaction sketched above:
It shows the START <address write> <command read ADC> STOP START <address read> and during this the fault occurs:
The red trace (bottom) going low marks the beginning of my error routine which is called because of a I2C.STAT.MSTSTATE=4, meaning that the <address read> wasn't ACKed by any slave - and this BEFORE the 9th clock for the ACK was sent. The state was saved in the error data structure and read with the debugger. Of course you can see, that one of the clock-low-states during the address was longer than the others, obviously twice as long. We all think that a clock pulse was lost but was counted by the hardware.
The purple trace, input at UART0 (PIO0_4 for both, TxD and RxD), had made me suspect that an interferring interrupt request could be a reason. In most (all but one) observed cases there was something incoming or outgoing to/from UART0/PIO0_4 when the problem arose; only one case was observed with "quiet" UART0, but in this case a Systick-Interrupt could have been occurred but I have no trace of this.
These are the peripheral interrupts which I use so far: SysTick, UART, MRT and I²C.

There are also other, much more complicated erranous sequences which I could not catch fully with my limited logic sniffer: START/STOP sent twice or other incorrect procedure steps. In all cases the the bus freezes.

The problem arises with different hardwares: 2 LPCXpressos and one of my own hardware with a 16-pin device. The chips are of version "M285.102 nB3424C" (I guess it is Rev. 4C, I replaced the old chips in my Rev.-A-LPCXpressos by this new chip).

I tried to write a recovery procedure, which shall be called when a code in STAT.MSTSTATE is not as expected. It mainly logs the event, tries to send an extra stop condition and even stops and resets the I²C peripheral (LPC_SYSCON->PRESETCTRL&= ~(I2C_PRESETCTRL_Bits); LPC_SYSCON->SYSAHBCLKCTRL&= ~ (I2C_SYSAHBCLKCTRL_Bits);), but nothing helps.
There were different such attempts, but the result was allways the same: After such a fault the data line remains static (low or high) and there isn't sent anythong else on the clock line (stays high) until, at least, reset through the debugger or even power off.

The first question is: What could I have done wrong that all this happens?.
The second question is: Is there any hint how to avoid or bypass this problem?
Third question: If the problem cannot be avoided or bypassed - how do I recover from it? My attempts so far failed.
Is timeout the given way to recover from such a fault? How would this to be done? How do I reliably sense the fault and how do I reliably restart the I²C peripheral?

This is the source of the I²C- and logical MS5611- driver:

/** \file I2Csensor.c
\brief Treiberschicht für das kDevice812: I²C-Anbindung und LowLevel für MS5611.
\author Helmut Stettmaier (stm), helmut@stettmaier.de
\version 1.0
\date 15. Februar 2014
\copyright Es wurde Texte und Code-Muster aus dem LPC812-Manual übernommen.
*/

#include "LPC8xx.h"// Registernamen
#include "LPC8xxB.h"// weitere Definitionen (nicht CMSIS) für Bit-Namen
#include "hal.h"
#include "I2Csensor.h"
#include "main.h"

void I2CCtrlResetEngine();
void I2CCtrlSendAddressWrite();
void I2CCtrlSendAddressRead();
void I2CCtrlSendResetCmd();
void I2CCtrlStartTemp();
void I2CCtrlStartPressure();
void I2CCtrlSendCalReadCmd();
void I2CCtrlReadADCCommand();
void I2CCtrlStop();
void I2CCtrlSequStop();
void I2CCtrlResetHardware(void);
void I2CCtrlReadByte1();
void I2CCtrlReadByte2();
void I2CCtrlCalReadByte2();
void I2CCtrlTempReadByte3();
void I2CCtrlPressureReadByte3();

/* Note for LPC Forum readers: I²C is set up in the HAL as follows:
LPC_SYSCON->SYSAHBCLKCTRL |= (
SYSCON_SYSAHBCLKCTRL_Bits | UART_SYSAHBCLKCTRL_Bits |
GPIO_SYSAHBCLKCTRL_Bits | I2C_SYSAHBCLKCTRL_Bits |
SCT_SYSAHBCLKCTRL_Bits); // SCT temporär für den Test-Ohrhöhrer
LPC_SYSCON->PRESETCTRL= (
SYSCON_PRESETCTRL_Bits | UART_PRESETCTRL_Bits |
GPIO_PRESETCTRL_Bits | I2C_PRESETCTRL_Bits |
SCT_PRESETCTRL_Bits); // SCT temporär für den Test-Ohrhöhrer
...
LPC_I2C->DIV= 3; // 14;
LPC_I2C->CFG= 1;
*/

/** Die I²C-Adresse des Sensors (CS=low) */
#define MS5611_I2C_ADR 0xee

/** Die 6 Kalibrierungswerte des MS5611 ohne CRC. */
uint16_t C[7];

/** Namen: C[0] ist C1 u.s.w. */
#define C1 C[1]
#define C2 C[2]
#define C3 C[3]
#define C4 C[4]
#define C5 C[5]
#define C6 C[6]
/** Zähler für die Kalibrierwerte-Schleife im I²C-Automaten. */
#define cnt C[0]

/** Zwischengröße: Differenz zur "Referenztemperatur" in D[2].
Wird zur Kompensation des Temperatur-Fehlers im Druckwert benötigt. */
uint32_t dT;

/** Export: Temperatur-kompensierter Druckwert in Pa. */
uint32_t rawPressure;

/** . */
uint32_t input;

/** Status des I²C-Automaten. */
uint8_t MS5611state;

/** . */
#define MS5611statePanicFlag 0x80

/** Kopie von LPC_I2C->STAT. */
uint8_t cpyI2CSTAT;

/** Steuer-Bit für I²C: Master - Starte Senden. */
#define MSTSTART 0x02
/** Steuer-Bit für I²C: Master - Setze Kommunikation fort. */
#define MSTCONTINUE 0x01
/** Steuer-Bit für I²C: Master - Beende Kommunikation. */
#define MSTSTOP 0x04

/** Besonderer Zustand der I2C-Engine: Setzt die Maschinerie zurück. */
#define I2CCtrlReset0
/** Besonderer Zustand der I2C-Engine: Schleifen-Beginn für die Kal.-Konstanten. */
#define I2CCtrlCalLoop4
/** Besonderer Zustand der I2C-Engine: Startet neu (ohne Rücksetzen). */
#define I2CCtrlRestart10
/** Besonderer Zustand der I2C-Engine: Endlos-Schleife für Temperatur und Druck. */
#define I2CCtrlLoop10
/** Größte erlaubte Zustands-Zahl für Bereichsmessung. */
#define I2CCtrlUppLimit29

void (*I2CCtrlTable[])() = {
I2CCtrlResetEngine,//  0
/* MS5611-Reset-Kommando */
I2CCtrlSendAddressWrite,//  1 Adresse (Write)MSTSTART
I2CCtrlSendResetCmd,//  2 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,//  3 MSTSTOP, INTENCLR
/* 6mal Kalibrier-Konstante einlesen (C[0]..C[5]) */
I2CCtrlSendAddressWrite,//  4 Adresse (Write)MSTSTART
I2CCtrlSendCalReadCmd,//  5 Kommando-ByteMSTCONTINUE
I2CCtrlStop,//  6 MSTSTOP
I2CCtrlSendAddressRead,//  7 Adresse (Read)MSTSTART
I2CCtrlReadByte1,//  8 lies 1. ByteMSTCONTINUE
I2CCtrlCalReadByte2,//  9 lies 2. ByteMSTSTOP, INTENCLR
/* zurück zu 4 (I2CCtrlCalLoop) falls noch Kalibrierwerte zu lesen sind */
/* Temperatur-Messung einleiten */
I2CCtrlSendAddressWrite,// 10 Adresse (Write)MSTSTART
I2CCtrlStartTemp,// 11 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,// 12 MSTSTOP, INTENCLR
/* Temperatur-Wert einlesen und kalibrieren */
I2CCtrlSendAddressWrite,// 13 Adresse (Write)MSTSTART
I2CCtrlReadADCCommand,// 14 Kommando-ByteMSTCONTINUE
I2CCtrlStop,// 15 MSTSTOP
I2CCtrlSendAddressRead,// 16 Adresse (Read)MSTSTART
I2CCtrlReadByte1,// 17 lies 1. ByteMSTCONTINUE
I2CCtrlReadByte2,// 18 lies 2. ByteMSTCONTINUE
I2CCtrlTempReadByte3,// 19 lies 3. ByteMSTSTOP, INTENCLR
/* Druck-Messung einleiten */
I2CCtrlSendAddressWrite,// 20 Adresse (Write)MSTSTART
I2CCtrlStartPressure,// 21 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,// 22 MSTSTOP, INTENCLR
/* Druck-Wert einlesen und kalibrieren */
I2CCtrlSendAddressWrite,// 23 Adresse (Write)MSTSTART
I2CCtrlReadADCCommand,// 24 Kommando-ByteMSTCONTINUE
I2CCtrlStop,// 25 MSTSTOP
I2CCtrlSendAddressRead,// 26 Adresse (Read)MSTSTART
I2CCtrlReadByte1,// 27 lies 1. ByteMSTCONTINUE
I2CCtrlReadByte2,// 28 lies 2. ByteMSTCONTINUE
I2CCtrlPressureReadByte3// 29 lies 3. ByteMSTSTOP, INTENCLR
/* zurück zu 10 (I2CCtrlLoop) */
};

/** Fehlerroutine: Mitten in der Sequenz (bei unbekanntem Zustand des I²C-Gerätes)
abbrechen und die I²C-Engine auf "Neustart" setzen, Eintrag in Fehlerspeicher
machen. */
void I2CCtrlAbortSequence(void) {
debugRedLow;//<<<<<<<<<<<<<<<<<
int i= (xErrRecord++) & 0x03;
ErrRecord.tag= ErrRecordI2C;
ErrRecord.subtag= MS5611state;
ErrRecord.SysTickStamp= (uint16_t)(SysTickCounter) & 0xffff;
ErrRecord.v.I2C.cpyI2CSTAT= cpyI2CSTAT;
ErrRecord.v.I2C.currI2Cdata= input;
setLED_BlinkPattern(LED_Blink_2Blinks);
LPC_I2C->INTENCLR= 1;
LPC_I2C->MSTCTL= MSTSTOP;
debugOrangeLow;//<<<<<<<<<<<<<<<<
cnt= 6; MS5611state= (I2CCtrlRestart-1) | MS5611statePanicFlag;
debugRedHigh;//<<<<<<<<<<<<<<<
}

/** Sende die MS5611-Adresse (als Beginn einer I²C-Kommandosequenz). */
void I2CCtrlSendAddressWrite(void) {
LPC_SYSCON->SYSAHBCLKCTRL|= (I2C_SYSAHBCLKCTRL_Bits);
LPC_SYSCON->PRESETCTRL|= (I2C_PRESETCTRL_Bits);
debugOrangeHigh;//<<<<<<<<<<
LPC_I2C->MSTDAT= MS5611_I2C_ADR;
LPC_I2C->MSTCTL= MSTSTART;
LPC_I2C->INTENSET= 1;
input= 0;
};

/** Beende eine I²C-Kommandosequenz. */
void I2CCtrlSequStop(void) {
LPC_I2C->MSTCTL= MSTSTOP;
LPC_I2C->INTENCLR= 1; // löst nicht mehr aus.
}

/** Reset I²C Hardware. Panik! */
void I2CCtrlResetHardware(void) {
debugOrangeHigh;//<<<<<<<<<<<
LPC_SYSCON->PRESETCTRL&= ~(I2C_PRESETCTRL_Bits);
LPC_SYSCON->SYSAHBCLKCTRL&= ~ (I2C_SYSAHBCLKCTRL_Bits);
MS5611state&= ~MS5611statePanicFlag;
debugOrangeLow;//<<<<<<<<<<
}

/** Setze den I²C-Automaten zurück. */
void I2CCtrlResetEngine(void) {
cnt= 6; MS5611state= 0;
}

/** Sende die MS5611-Adresse zum Lesen. */
void I2CCtrlSendAddressRead(void) {
if (cpyI2CSTAT==1 /* ist I²C idle? */) {
LPC_I2C->MSTDAT= MS5611_I2C_ADR | 1;
LPC_I2C->MSTCTL= MSTSTART;
} else I2CCtrlAbortSequence();
};

/** Sende den Reset-Befehl als Datenbyte. */
void I2CCtrlSendResetCmd(void) {
//if (cpyI2CSTAT==5 /* Transmit_ready? */) {
LPC_I2C->MSTDAT= 0x1e;
LPC_I2C->MSTCTL= MSTCONTINUE;
//} else I2CCtrlAbortSequence();
}

/** Sende das Initialize-Temperature-Measurement-Kommando als Datenbyte. */
void I2CCtrlStartTemp(void) {
LPC_I2C->MSTDAT= 0x58;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende das Initialize-Pressure-Measurement-Kommando als Datenbyte. */
void I2CCtrlStartPressure(void) {
LPC_I2C->MSTDAT= 0x48;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende das CalConst-Lese-Kommando als Datenbyte. cnt ist # der CalConst. */
void I2CCtrlSendCalReadCmd(void) {
LPC_I2C->MSTDAT= 0xa0 | (cnt<<1);
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** . */
void I2CCtrlReadADCCommand(void) {
//if (cpyI2CSTAT==5 /* Transmit_ready? */) {
LPC_I2C->MSTDAT= 0x00;
LPC_I2C->MSTCTL= MSTCONTINUE;
//} else I2CCtrlAbortSequence();
}

/** Sende Stop-Condition, lasse den Interrupt aber eingeschaltet. */
void I2CCtrlStop(void) {
LPC_I2C->MSTCTL= MSTSTOP; // sendet STOP und löst sofort mit IDLE wieder aus
}

/** Lesen: 1. Byte. */
void I2CCtrlReadByte1(void) {
if (cpyI2CSTAT==3 /* Read_ready? */) {
input= LPC_I2C->MSTDAT;
LPC_I2C->MSTCTL= MSTCONTINUE;
} else I2CCtrlAbortSequence();
}

/** Lesen: 2. Byte (neutral). */
void I2CCtrlReadByte2(void) {
if (cpyI2CSTAT==3 /* Read_ready? */) {
input= input<<8 | LPC_I2C->MSTDAT;
LPC_I2C->MSTCTL= MSTCONTINUE;
} else I2CCtrlAbortSequence();
}

/** Kalibrier-Konstante lesen: 2. Byte, zusammensetzen & abspeichern.
cnt ist # der CalConst. */
void I2CCtrlCalReadByte2(void) {
if (cpyI2CSTAT==3 /* Read_ready? */) {
C[cnt]= input<<8 | LPC_I2C->MSTDAT;
/* Es müssen 6 Konstanten eingelesen werden: C[6]..C[1]. Dies wird mit
cnt (von 0 bis 5) gezählt. Wenn cnt also noch <6 ist muss dieser
Lesevorgang wiederholt werden; dazu wird (ugly) einfach MS5611state
zurück gestellt. */
if (--cnt) MS5611state= I2CCtrlCalLoop-1;
/* Sende Stop-Condition und beende die Sequenz. */
LPC_I2C->MSTCTL= MSTSTOP;
LPC_I2C->INTENCLR= 1;
} else I2CCtrlAbortSequence();
}

/** Lies das 3. Byte des Temperatur-Wertes ein, Berechne dT. */
void I2CCtrlTempReadByte3(void) {
uint32_t D2= input<<8 | LPC_I2C->MSTDAT;
dT = D2 - (((int32_t)(C5)) << 8);
LPC_I2C->MSTCTL= MSTSTOP;
}

/** Lies das 3. Byte des Druck-Wertes ein, Berechne dT. */
void I2CCtrlPressureReadByte3(void) {
int64_t s, o;
uint32_t D1= input<<8 | LPC_I2C->MSTDAT;
o =(((int64_t)(C2))<<16)+ (((C4)* (int64_t)dT)>>7);
s =(((int64_t)(C1))<<15)+ (((C3)* (int64_t)dT)>>8);
rawPressure =((((int64_t)D1 *s)>>21) -o)>>15;
LPC_I2C->MSTCTL= MSTSTOP;
mainStatus|= mstatMS5611PrValread;
MS5611state= I2CCtrlLoop-1;
}

/** Start des MS5611-Abfrage-Automaten. */
void resetMS5611(void) {
set_mrt_repeatedmode (2, (110*__SYSCLOCK_100us)); // 11ms
MS5611state= 0;
}

/* Note for readers from LPC-Forum:
I2C_MRTaction is part of the MRT-Interrupt handler which is copied here:

void MRT_IRQHandler(void) {
if (LPC_MRT->Channel[2].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[2].STAT = MRT_STAT_IRQ_FLAG;
I2C_MRTaction();
}
if (LPC_MRT->Channel[3].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[3].STAT = MRT_STAT_IRQ_FLAG;
MSBRxTiming();
}
NVIC->ICPR[0] |= MRT_INTERRUPT_BIT;
return;
}
*/

/** Bearbeitet den für den I²C-Verkehr ausgelösten MRT-Interrupt:
Startet die nächste anstehende I²C-Kommandosequenz. */
inline void I2C_MRTaction(void) {
cpyI2CSTAT= LPC_I2C->STAT;
if (MS5611state>I2CCtrlUppLimit) MS5611state= 0; // panic
(*I2CCtrlTable[MS5611state])();
MS5611state++;
return;
}

/** I²C-Interrupt: Setzt die laufende I²C-Kommandosequenz fort. */
void I2C_IRQHandler(void) {
cpyI2CSTAT= LPC_I2C->STAT;
if (MS5611state>I2CCtrlUppLimit) MS5611state= 0; // panic
if (MS5611state & MS5611statePanicFlag) I2CCtrlResetHardware();
else {
(*I2CCtrlTable[MS5611state])();
MS5611state++;
}
return;
}

/* Note for readers from LPC-Forum:
This is a data record for logging faults; it is used in the error repai function:
typedef struct sErrRecord {
uint8_t tag;
uint8_t subtag;
uint16_t SysTickStamp;
union uErrRecord {
uint32_t unspecified;
struct sI2C {
uint32_t currI2Cdata;
uint8_t cpyI2CSTAT;
} I2C;
} v;
} tErrRecord;
*/

/** Tag für Fehleraufzeichnung, identifiziert I²C-Fehler. */
#define ErrRecordI2C 1
/** Schreibt eine I²C-Fehlermeldung in das Fehler-Log.
\param stwird als SubTag eingetragen.
*/
void wErrRecordI2C(void) {
disableInterrupts;
int i= (xErrRecord++) & 0x03;
ErrRecord.tag= ErrRecordI2C;
ErrRecord.subtag= MS5611state;
ErrRecord.SysTickStamp= (uint16_t)(SysTickCounter) & 0xffff;
ErrRecord.v.I2C.cpyI2CSTAT= cpyI2CSTAT;
ErrRecord.v.I2C.currI2Cdata= input;
enableInterrupts;
setLED_BlinkPattern(LED_Blink_2Blinks);
}

Thank you very much in advance for your patience and help and
kind regards,
Helmut

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Sat Jan 31 09:09:46 MST 2015
Hello,
I think I found out what happened and the puzzle should be solved here, even after a year of testing the device with the "I²C-reset and repair" code.
During electronica I got an advise from an application engineer to change the interrupt priorities - this was obviously not the solution, but is was the right hint for another bug hunt:

The problem seems to be in the MRT_IRQHandler, which is part of the listing in the 1st post (2014-03-30 12:32):
The MRT has 4 channels, 2 of them in use, all of them share the same interrupt and a collision was not solved correctly:
When the MRT channel for the UART fires an interrupt and the MRT channel for the I²C (channel 2) expires during the time span in which the UART-MRT-interrupt (channel 3) is handled, this expiration was lost.
I made 3 changes:

void MRT_IRQHandler(void) {
NVIC->ICPR[0] = MRT_INTERRUPT_BIT;
checkallMRTints:
if (LPC_MRT->Channel[3].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[3].STAT = MRT_STAT_IRQ_FLAG;
MSBRxTiming();
}
if (LPC_MRT->Channel[2].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[2].STAT = MRT_STAT_IRQ_FLAG;
I2C_MRT_Action();
goto checkallMRTints;
}
return;
}

I exchanged the 2 checks, which finally appears not important (but I only didn't change it back).

I inserted a jump: I know, "goto" is "Yuck!", I could also write a "while" construct handling and checking a flag... (another "Yuck!", but more complex)
So when during handling an interrupt for channel 2 channel 3 expires, this will be checked.
I could not believe that collisions of these quite short interrupt handling routines occur so often, but the fact is: This change definitely cleared the problem.

Finally I repositioned the line clearing the interrupt pending flag from end to the beginning of the routine. I did that reading the 1st sentence in 3.4.4 of the user manual.
What is not 100% clear for me:
Would this allone (change the interrupt state to "not pending" at the beginning of the interrupt handler instead at the end) let the risk of loosing the interrupt, which is handled in the 1st check, vanish? Could I save this way the (yes, ugly) goto? I can't test this reliably.

To ask this more generally:
Is there a standard template for interrupt handlers for shared interrupts like this?

Now the device works reliably, a repair/reset for lost ACKs isn't necessary any longer.

Thank you again for all who tried to help me and
kind regards,
Helmut

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Tue Apr 08 07:54:41 MST 2014
Hello,

resetting the I²C peripheral is easy when it is done completely:

LPC_SYSCON->PRESETCTRL &= ~I2C_PRESETCTRL_Bits;
LPC_SYSCON->PRESETCTRL |= I2C_PRESETCTRL_Bits;
LPC_I2C->DIV= 3; // don't forget to define the clock divider again
LPC_I2C->CFG= 1; // don't forget to set Master function again
LPC_I2C->TIMEOUT= 0x002f; // and don't forget to set timeout again when you use it.
// LPC_I2C->INTENSET= (MSTPENDINGEN | EVENTTIMEOUTEN); // and don't forget this...

Now it works and I can recover from the I²C-problems.
Perhaps I later find out where the problem comes from.

Kind regards,
Helmut

[lpcware]

Content originally posted in LPCWare by MarcVonWindscooting on Mon Apr 07 05:57:19 MST 2014
Hi Helmut,

I can't help here with details, because I'm unable to understand complex programs. But let me comment a little bit from a distance:

It seems you program lacks modularity. You intermix sensor-specific details with the I2C communication. I almost never do such. Instead I first write a generic I2C module and test that with a very simple device, then with a more complex device and so on...
Once you can trust your I2C communication you can focus on the (not too simple) sensor. The LPC812 has I2C ROM API functions. You might give them a try.

You make assumptions and don't prove them. You don't think it's a hardware problem. Honestly I don't think, neither. But I would try to prove it. What if your sensor or controller got damaged while assembling?

Asynchronous actions are almost always a bad idea. It's not, that they couldn't work. It's because the human brain (or at least mine) almost always misses one (or two or three) cases where it goes wrong without special precautions. So either be a real genius or keep away from stopping/restarting/state-changing/... peripheral 'A' from within a handler of peripheral 'B'.

Avoid copies/redundancy and use information hiding:
#define C0 C[0]
uint8_t cpyI2CSTAT;

why half of the code symbolic and the other half literal numbers?

if (cpyI2CSTAT==1 /* ist I²C idle? */) {
LPC_I2C->MSTDAT= MS5611_I2C_ADR | 1;
LPC_I2C->MSTCTL= MSTSTART;
...

Such things are a great source of problems of all kind for programmers like me (at least):
if (--cnt) MS5611state= I2CCtrlCalLoop-1;

Without further looking at your code, I ask if you really didn't mean 'if (cnt-1) ..' and if it's OK that cnt is decremented EVERY time and without limit (!). In another place you access C[cnt]. I expect the worst  ;-)

If you accumulate characters, why not use proven buffer/iterator functions you implemented and tested before. So many programmers re-invent everything over and over. You might think: "Ah come on, I can increment a pointer/counter just in place!". Yes, you can, in 99% of all cases. And in 1% of all cases you will be debugging code and searching for the wrong cause...

I believe, you should rewrite the code from scratch. It should be easier to understand with less questionable constructs, less #defines, more enums, more modularity. The current code is not worth debugging.

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Mon Apr 07 04:04:31 MST 2014
Hello Noah,
you're tireless, and all this for me...
Your notes teached me to check UM-Table 194 more thoroughly. Writing MSTCTL when the I²C master isn't idle/pending is avoided now and the strangest parts of the problems, e.g. disorted clock pulses, vanished.
Thank you very much!

But there are 2 "But"s:
[list=1]
  [*]But the cause of the problems remains.
  [*]But when the I²C-master is not pending, this is a fault condition and I must recover from it. To do this it I must write to MSTCTL... I currently have no idea how to recover from such erranous states.
[/list]
To 1:
The problem has simplified in its symptoms: Status information may become wrong when UART0 communication overlaps I²C action.
Of course I checked the drivers - UART0 uses the UART0 interrupt and channel 3 of the MRT (for IDLE detection a.s.o), I²C uses the I2C-Interrupt and MRT channel 2 for the 11ms raster to be taken for accessing the 5611. So they share the MRT interrupt.

I instrumented the drivers such that they give signals for the logic sniffer - interestingly these traces show:
[list]
  [*]conflicts only appear when UART0 traffic and I²C action overlap, as stated, and
  [*]the I²C Interrupt driver often gos into its error recovering path -->before any interrupt for the UART0 is fired.
[/list]
A typical case is shown in Picture Screenshot (105):
White (0) and grey (1) traces are the I²C traffic,
the purple trace (2) is UART0 (an open drain half duplex 38.4kBaud line, only receiving and not sending here).
Blue (3) is the runtime of the MRT interrupt branch for channel 3, it is fired much later (out of the screenshot).
Green (4) is the runtime of the UART0 interrupt, in this handler the MRT channel 3 is set.
Yellow (5) is the MRT interrupt branch for channel 2, it calls I2C_MRTaction() and this calls I2CCtrlStartSequence() when I²C's state is ok.
Orange (6) is the I2C_IRQHandler, and
red (7) goes low when I2C_MRTaction() or I2C_IRQHandler() detect an "illegal" state of I²C.
You see, the signal on UART0 comes before the last byte on I²C is sent, but the I²C problem is detected before the "green" interrupt is handled.
I still do not know what exactly happens before the last byte. I stopped the 5611 sending data by returning NAK. This is done with "LPC_I2C->MSTCTL= MSTSTOP;" as stated in UM Table 194. Is an extra START STOP sent? How can this happen? The next action of my program is waiting on the interrupt and then calling I2C_MRTaction/I2CCtrlStartSequence which sends START and the slave address.
To make that clearer, the I²C sequence is:
START address(W) ACK 0x00 ACK STOP START address(R) data0 ACK data1 ACK data2 NAK (??) START address(W) after 8 clocks NAK?

In the described case the last byte was the slave address introducing the next command, it was said to be NAKed by the slave, what is illegal. Another sample is shown in Screenshot (99), in this case the UART0 interrupt was fired before the red trace went low.

Recovering from these faults seems to work, although it would be nicer not to have to recover.

A very special flavour of this problem is that the bytes are ACKed, but the I²C engine remains not-pending, what is detected 11ms later in I2C_MRTaction() going into its panic branch.
You can see this behaviour in Screenshot (106).

This leads to problem 2: I2C_MRTaction() detects that I²C is not pending and I would have to write to MSTCTL (really?) to recover.

I asked friends, if they ever hat problems with the MS5611, accidentially sometimes not ACKing data - no, it works absolutely reliably. And I have tried all this with 2 different hardwares.

No, I do not want to argue "the hardware...", I think, something is configured wrong.
Here are the sources again:

/** \file I2Csensor.c
\brief Treiberschicht für das kDevice812: I²C-Anbindung und LowLevel für MS5611.
\author Helmut Stettmaier (stm)
\version 1.0
\date ... 2014
*/

#include "LPC8xx.h"// Registernamen
#include "LPC8xxB.h"// weitere Definitionen (nicht CMSIS) für Bit-Namen
#include "hal.h"
#include "I2Csensor.h"
#include "main.h"

void I2CCtrlStartSequence();
void I2CCtrlSendAddressRead();
void I2CCtrlSendResetCmd();
void I2CCtrlStartTemp();
void I2CCtrlStartPressure();
void I2CCtrlSendCalReadCmd();
void I2CCtrlReadADCCommand();
void I2CCtrlStop();
void I2CCtrlSequStop();
void I2CCtrlResetHardware(void);
void I2CCtrlReadByte1();
void I2CCtrlReadByte2();
void I2CCtrlCalReadByte2();
void I2CCtrlTempReadByte3();
void I2CCtrlPressureReadByte3();
void writeI2CErrRecord(unsigned int v);


/* Note for LPC Forum readers: I²C is set up in the HAL as follows:
LPC_SYSCON->SYSAHBCLKCTRL |= (
SYSCON_SYSAHBCLKCTRL_Bits | UART_SYSAHBCLKCTRL_Bits |
GPIO_SYSAHBCLKCTRL_Bits | I2C_SYSAHBCLKCTRL_Bits |
SCT_SYSAHBCLKCTRL_Bits); // SCT temporär für den Test-Ohrhöhrer
LPC_SYSCON->PRESETCTRL= (
SYSCON_PRESETCTRL_Bits | UART_PRESETCTRL_Bits |
GPIO_PRESETCTRL_Bits | I2C_PRESETCTRL_Bits |
SCT_PRESETCTRL_Bits); // SCT temporär für den Test-Ohrhöhrer
...
LPC_I2C->DIV= 3; // 14;
LPC_I2C->CFG= 1;
LPC_I2C->TIMEOUT= 0x002f; // -->48
*/

/** Die I²C-Adresse des Sensors (CS=low) */
#define MS5611_I2C_ADR 0xee

/** Die 6 Kalibrierungswerte des MS5611 ohne CRC. */
static uint16_t C[7];

/** Namen: C[1] ist C1 u.s.w. */
#define C1 C[1]
#define C2 C[2]
#define C3 C[3]
#define C4 C[4]
#define C5 C[5]
#define C6 C[6]
/** Zähler für die Kalibrierwerte-Schleife im I²C-Automaten. */
static uint8_t cnt;

/** Status des I²C-Automaten. */
uint8_t MS5611state;

/** Zwischengröße: Differenz zur "Referenztemperatur" in D[2].
Wird zur Kompensation des Temperatur-Fehlers im Druckwert benötigt. */
uint32_t dT;

/** Export: Temperatur-kompensierter Druckwert in Pa. */
uint32_t rawPressure;

/** . */
static uint32_t input;

/** Start des MS5611-Abfrage-Automaten. */
void resetMS5611(void) {
set_mrt_repeatedmode (2, (110*__SYSCLOCK_100us)); // 11ms
MS5611state= 0;
cnt= 6;
input= 0;
}

/** Wert von LPC_I2C->STAT. */
static uint32_t cpyI2CSTAT;

/** Steuer-Bit für I²C: Master - Starte Senden. */
#define MSTSTART 0x02
/** Steuer-Bit für I²C: Master - Setze Kommunikation fort. */
#define MSTCONTINUE 0x01
/** Steuer-Bit für I²C: Master - Beende Kommunikation. */
#define MSTSTOP 0x04

/** Besonderer Zustand der I2C-Engine: Setzt die Maschinerie zurück. */
#define I2CCtrlReset0
/** Besonderer Zustand der I2C-Engine: Schleifen-Beginn für die Kal.-Konstanten. */
#define I2CCtrlCalLoop3
/** Besonderer Zustand der I2C-Engine: Startet neu (ohne Rücksetzen). */
#define I2CCtrlRestart9
/** Besonderer Zustand der I2C-Engine: Endlos-Schleife für Temperatur und Druck. */
#define I2CCtrlLoop12
/** Größte erlaubte Zustands-Zahl für Bereichsprüfung. */
#define I2CCtrlUppLimit31

//void (*I2CCtrlTable[])() = {
/** . */
typedef void (* I2Cstep)(void);
/** . */
const I2Cstep I2CCtrlTable[]= {
/* MS5611-Reset-Kommando */
I2CCtrlStartSequence,//  0 Adresse (Write)MSTSTART
I2CCtrlSendResetCmd,//  1 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,//  2 MSTSTOP, INTENCLR
/* 6mal Kalibrier-Konstante einlesen (C[1]..C[6]) */
I2CCtrlStartSequence,//  3 Adresse (Write)MSTSTART
I2CCtrlSendCalReadCmd,//  4 Kommando-ByteMSTCONTINUE
I2CCtrlStop,//  5 MSTSTOP
I2CCtrlSendAddressRead,//  6 Adresse (Read)MSTSTART
I2CCtrlReadByte1,//  7 lies 1. ByteMSTCONTINUE
I2CCtrlCalReadByte2,//  8 lies 2. ByteMSTSTOP, INTENCLR
/* zurück zu 4 (I2CCtrlCalLoop) falls noch Kalibrierwerte zu lesen sind */
/* MS5611-Reset-Kommando */
I2CCtrlStartSequence,//  9 Adresse (Write)MSTSTART
I2CCtrlSendResetCmd,// 10 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,// 11 MSTSTOP, INTENCLR
/* Temperatur-Messung einleiten */
I2CCtrlStartSequence,// 12 Adresse (Write)MSTSTART
I2CCtrlStartTemp,// 13 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,// 14 MSTSTOP, INTENCLR
/* Temperatur-Wert einlesen und kalibrieren */
I2CCtrlStartSequence,// 15 Adresse (Write)MSTSTART
I2CCtrlReadADCCommand,// 16 Kommando-ByteMSTCONTINUE
I2CCtrlStop,// 17 MSTSTOP
I2CCtrlSendAddressRead,// 18 Adresse (Read)MSTSTART
I2CCtrlReadByte1,// 19 lies 1. ByteMSTCONTINUE
I2CCtrlReadByte2,// 20 lies 2. ByteMSTCONTINUE
I2CCtrlTempReadByte3,// 21 lies 3. ByteMSTSTOP, INTENCLR
/* Druck-Messung einleiten */
I2CCtrlStartSequence,// 22 Adresse (Write)MSTSTART
I2CCtrlStartPressure,// 23 Kommando-ByteMSTCONTINUE
I2CCtrlSequStop,// 24 MSTSTOP, INTENCLR
/* Druck-Wert einlesen und kalibrieren */
I2CCtrlStartSequence,// 25 Adresse (Write)MSTSTART
I2CCtrlReadADCCommand,// 26 Kommando-ByteMSTCONTINUE
I2CCtrlStop,// 27 MSTSTOP
I2CCtrlSendAddressRead,// 28 Adresse (Read)MSTSTART
I2CCtrlReadByte1,// 29 lies 1. ByteMSTCONTINUE
I2CCtrlReadByte2,// 30 lies 2. ByteMSTCONTINUE
I2CCtrlPressureReadByte3// 31 lies 3. ByteMSTSTOP, INTENCLR
/* zurück zu 10 (I2CCtrlLoop) */
};

/** Sende die MS5611-Adresse (als Beginn einer I²C-Kommandosequenz). */
void I2CCtrlStartSequence(void) {
LPC_I2C->MSTDAT= MS5611_I2C_ADR;
LPC_I2C->MSTCTL= MSTSTART;
debugRedHigh;//<<<<<<<<<<<<<<<<<
LPC_I2C->INTENSET= (MSTPENDINGEN | EVENTTIMEOUTEN);
input= 0;
};

/** Beende eine I²C-Kommandosequenz. */
void I2CCtrlSequStop(void) {
LPC_I2C->INTENCLR= 1;
LPC_I2C->MSTCTL= MSTSTOP;
}

/** Sende die MS5611-Adresse zum Lesen. */
void I2CCtrlSendAddressRead(void) {
LPC_I2C->MSTDAT= MS5611_I2C_ADR | 1;
LPC_I2C->MSTCTL= MSTSTART;
};

/** Sende den Reset-Befehl als Datenbyte. */
void I2CCtrlSendResetCmd(void) {
LPC_I2C->MSTDAT= 0x1e;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende das Initialize-Temperature-Measurement-Kommando als Datenbyte. */
void I2CCtrlStartTemp(void) {
LPC_I2C->MSTDAT= 0x58;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende das Initialize-Pressure-Measurement-Kommando als Datenbyte. */
void I2CCtrlStartPressure(void) {
LPC_I2C->MSTDAT= 0x48;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende das CalConst-Lese-Kommando als Datenbyte. cnt ist # der CalConst. */
void I2CCtrlSendCalReadCmd(void) {
LPC_I2C->MSTDAT= 0xa0 | (cnt<<1);
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** . */
void I2CCtrlReadADCCommand(void) {
LPC_I2C->MSTDAT= 0x00;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Sende Stop-Condition, lasse den Interrupt aber eingeschaltet. */
void I2CCtrlStop(void) {
LPC_I2C->MSTCTL= MSTSTOP;
}

/** Lesen: 1. Byte. */
void I2CCtrlReadByte1(void) {
input= LPC_I2C->MSTDAT;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Lesen: 2. Byte (neutral). */
void I2CCtrlReadByte2(void) {
input= input<<8 | LPC_I2C->MSTDAT;
LPC_I2C->MSTCTL= MSTCONTINUE;
}

/** Kalibrier-Konstante lesen: 2. Byte, zusammensetzen & abspeichern.
cnt ist # der CalConst. */
void I2CCtrlCalReadByte2(void) {
C[cnt]= input<<8 | LPC_I2C->MSTDAT;
if (--cnt) MS5611state= I2CCtrlCalLoop-1;
LPC_I2C->MSTCTL= MSTSTOP;
LPC_I2C->INTENCLR= 1;
}

/** Lies das 3. Byte des Temperatur-Wertes ein, Berechne dT. */
void I2CCtrlTempReadByte3(void) {
uint32_t D2= input<<8 | LPC_I2C->MSTDAT;
dT = D2 - (((int32_t)(C5)) << 8);
LPC_I2C->MSTCTL= MSTSTOP;
}

/** Lies das 3. Byte des Druck-Wertes ein, Berechne den Roh-Druckwert. */
void I2CCtrlPressureReadByte3(void) {
int64_t s, o;
uint32_t D1= input<<8 | LPC_I2C->MSTDAT;
o =(((int64_t)(C2))<<16)+ (((C4)* (int64_t)dT)>>7);
s =(((int64_t)(C1))<<15)+ (((C3)* (int64_t)dT)>>8);
rawPressure =((((int64_t)D1 *s)>>21) -o)>>15;
LPC_I2C->MSTCTL= MSTSTOP;
mainStatus|= mstatMS5611PrValread;
MS5611state= I2CCtrlLoop-1;
}

/* Note for readers from LPC-Forum:
I2C_MRTaction is part of the MRT-Interrupt handler
which is copied here:

void MRT_IRQHandler(void) {
if (LPC_MRT->Channel[2].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[2].STAT = MRT_STAT_IRQ_FLAG;
debugYellowLow; //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
I2C_MRTaction();
debugYellowHigh; //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
}
if (LPC_MRT->Channel[3].STAT & MRT_STAT_IRQ_FLAG) {
LPC_MRT->Channel[3].STAT = MRT_STAT_IRQ_FLAG;
debugBlueLEDon; //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
MSBRxTiming();
debugBlueLEDoff; //<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
}
NVIC->ICPR[0] = MRT_INTERRUPT_BIT;
return;
}
*/

/** Bearbeitet den für den I²C-Verkehr ausgelösten MRT-Interrupt:
Startet die nächste anstehende I²C-Kommandosequenz. */
inline void I2C_MRTaction(void) {
if (MS5611state>I2CCtrlUppLimit) {
debugRedLow;//<<<<<<<<<<<<<<<<<<
writeI2CErrRecord(3);
mainStatus&= ~mstatMS5611PrValread;
MS5611state= I2CCtrlRestart;
}
if (!(LPC_I2C->STAT & MSTPENDING)) {
debugRedLow;//<<<<<<<<<<<<<<<<<<
writeI2CErrRecord(1);
_Panic_; // still write a panic handler <<<<<<<<<
}
I2CCtrlStartSequence(); // ex (*I2CCtrlTable[MS5611state])();
MS5611state++;
return;
}

/** I²C-Interrupt: Setzt die laufende I²C-Kommandosequenz fort. */
void I2C_IRQHandler(void) {
cpyI2CSTAT= LPC_I2C->STAT;
debugOrangeLow; // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
if (MS5611state>I2CCtrlUppLimit)  {
mainStatus&= ~mstatMS5611PrValread;
MS5611state= I2CCtrlRestart;
}
if (!(cpyI2CSTAT & MSTPENDING)) {
writeI2CErrRecord(2);
_Panic_; // still write a panic handler <<<<<<<<<
}
if (cpyI2CSTAT & EVENTTIMEOUT
|| ((cpyI2CSTAT & MSTSTATEMASK)==MSTADDRNAK)
|| ((cpyI2CSTAT & MSTSTATEMASK)==MSTDATANAK)) { // ugly
//debugRedLow;//<<<<<<<<<<<<<<<<<<
writeI2CErrRecord(0);
setLED_BlinkPattern(LED_Blink_2Blinks);
LPC_I2C->STAT |= EVENTTIMEOUT;
LPC_I2C->INTENCLR= (MSTPENDINGEN | EVENTTIMEOUTEN);
LPC_I2C->MSTCTL= MSTSTOP;
mainStatus&= ~mstatMS5611PrValread;
cnt= 6;
MS5611state= I2CCtrlRestart;
} else {
(*I2CCtrlTable[MS5611state])();
MS5611state++;
}
debugOrangeHigh; // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
return;
}

void writeI2CErrRecord(unsigned int v) {
int i= (xErrRecord++) & 0x07;
ErrRecord.tag= ErrRecordI2C+v;
ErrRecord.subtag= MS5611state;
ErrRecord.SysTickStamp= (uint16_t)(SysTickCounter);
ErrRecord.v.I2C.I2CSTAT= cpyI2CSTAT;
ErrRecord.v.I2C.currI2Cdata= input;
}

2 questions (not only for Noah  ) :

[list=1]
  [*]Which is the simplest way to detect that the 812's I²C-engine doesn't want to go pending? Waiting on an interrupt will not work, neither polling. Sadly the I²C timeout feature obviously does -->not detect this. Currently I have my own timeout (11ms MRT), but I don't like using this.
  [*]Which are the steps to be taken to bring I²C back into the pending state? Reset (I had tried this earlier, but without success)? Clock off and reset? Clock off, power off and reset? Something simpler?
[/list]

I kindly suggest to write some explaining words into chapter 16.7.2 User Manual, how "...potentially do something to alleviate the condition..." is meant.

Kind regards,
Helmut

[lpcware]

Content originally posted in LPCWare by noahk on Sun Mar 30 12:29:56 MST 2014
Hi Helmut,

In I2C_MRTaction, you call I2CCtrlTable functions. In these functions, you check cpyI2CStat for the expected status. But if the status doesn't match, you call I2CCtrlAbortSequence. The abort sequence writes to MSTCTL without first making sure that the I2C MSTPENDING is high. That is illegal. You are trying to send a stop while the I2C master logic is busy. I'm not sure what you intend with that function, but you need to make sure that it is pending first.

The same issue is not a problem in your design when called from I2C_IRQHandler b/c the assumption is that I2C_IRQHandler would only be called when MSTPENDING is high (b/c you only use master logic).

Although it would be much better to check MSTPENDING in both I2C_MRTaction and I2C_IRQHandler before calling I2CCtrlTable functions.

Hope this helps,
Noah

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Sun Mar 30 09:25:40 MST 2014
Hello Noah,
thank you for the quick and convincing response.
I exchanged the 2 lines, but it didn't help. I'll check it thoroughly tomorrow.
Kind regards,
Helmut

[lpcware]

Content originally posted in LPCWare by noahk on Sun Mar 30 07:02:37 MST 2014
Hi Helmut,

In I2CCtrlSequStop, please write to INTENCLR before writing MSTSTOP.  Otherwise you have a race condition on whether you will receive a new MSTIDLE interrupt before you can clear the interrupt enable.

Let me know if that helps,
Noah

[lpcware]

Content originally posted in LPCWare by Helmut Stettmaier on Sun Mar 30 05:18:08 MST 2014
Edit:
The "typical sequence" is truncated because I used &lt; and &gt;... Here it is once more:
START [address write (EE)] [command read ADC (00)] STOP followed immediately by
START [address read (EF)] [read byte 1] [read byte 2] [read byte 3] NAK & STOP followed immediately by
START [address write (EE)] [command measure temperature] STOP and then about 10 ms pause for the MS5611 to do what being asked for.

Tanks & kind regards
Helmut