I have an LPC824 as the sole master of an I2C bus with two slaves, using the I2C ROM API. It can successfully write to either slave device, but only one of them, a DAC, can be read reliably. This is implemented on a custom board design of which I have programmed 8 units - only 4 of these can read the sensor device. The units that that fail return an error code of 0x60004 - arbitration lost. After the read is issued to the sensor, it ACKs and the 824 will stop clocking. A logic analyzer reveals that up to that point both the functional and non-functional boards have identical clock and data patterns. Functional boards continue to clock normally, of course. As mentioned above, reading from the other I2C device works normally. Upon encountering the error, forcing bus clocks by switching the pins to GPIO and toggling SCL will cause the slave to output the expected data.
Of particular interest is that by merely connecting an unterminated test lead to SDA will allow normal operation, so I'm inclined to suspect an impedance problem that causes the I2C hardware to become confused (the bus is very short, 2 cm or less). Varying pull ups (I've tried 1.2K, 2.2K and 4.7Ks) and bit rate has no effect.
Attached are screenshots of bus activity on working and non-working boards. I'm running out of rocks to turn over, ideas?