topic Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips in LPC Microcontrollers

LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Wed, 13 Jan 2021 23:40:59 GMT

I've never seen something like this. Ever.

This software has been in production for some time, but we've recently had a fallout of a considerable percentage. I've spent the last day trying to figure it out, and I've been able to prove that if I turn off optimization (going to -O0 instead of -Os), it runs. Otherwise, it near immediately hard faults.

I've got a batch of identical production run devices, and the one that fail all have similar coding:

LPC11E67J
23 13
ZSD20
121A

I'm running FreeRTOS 10.0.1

I have one task that does some serious DSP on audio, so I can't live with the RTOS running slow. In fact, to get it to run, I had to disable that task. If I enable it, it hits a hard fault.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Wed, 13 Jan 2021 22:50:29 GMT

I upgraded to FreeRTOS v10.4.3 - it runs with full optimization. But, I have to disable the I2S (via SPI & DMA) to prevent a hard fault.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Wed, 13 Jan 2021 23:40:34 GMT

A bit more about that - I can turn on the SPI, so long as I don't actually turn on the DMA that drives it. I have to comment out the last two lines of the init function.

Even if my DMA ISR just dumps the data in the trash, it hard faults - but only on some MCUs. On most boards this is working rock solid.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

diego_charles — Fri, 15 Jan 2021 00:41:31 GMT

Hi @chrispflieger

That's an interesting problem.

If I understand you properly , the issue seems to be parts with the part marking that you gave.

Are you able to provide some demo code were the issue is replicated?

In you FreeRTOS tasks, could you provide more details on how the SPI and DMA peripherals accessed and configured ? Do you have some kind of synchronization technique for this?

Thank you in advance.

All the best,

Diego.

Edith: re-phrasing

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Fri, 15 Jan 2021 18:51:03 GMT

I've attached the I2S driver. On a bad board, I need to disable the last line, which then keeps it from running.

I still can't figure out why an older version of FreeRTOS wouldn't run, but a newer one would. I ran it with minimal tasks, and with no devices, or off chip interfaces... I even tried to run code from another project (we use that chip and code base in a lot of production devices), it hard faulted there as well.

Even with I2S turned off, I still think there's occasional hard faults - maybe every ten or twenty minutes. I need to investigate that more.

Debugging hard faults on a CM0 is a real pain, but I can provide whatever data you want.

Note, I do see some problems on ZSD20 331A parts, but never on any made in 2019 or 2018.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

diego_charles — Mon, 18 Jan 2021 19:35:46 GMT

Hi @chrispflieger

Thank you for your code and additional information.

Regarding hardware , Are you able to perform the following swap test and let me know your results?

1 Solder a chip from 2020 into a 2019 or 2018 board.

2 Solder a chip from 2018 or 2019 into a 2020 board .

Many thanks

Diego.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Mon, 18 Jan 2021 22:09:59 GMT

I failed on the first swap - I'll try another one. My solder rework skills aren't the best.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

diego_charles — Tue, 19 Jan 2021 06:04:07 GMT

Hi @chrispflieger

Thank you for your update and help for doing this procedure.

Diego

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Fri, 29 Jan 2021 19:41:52 GMT

I had a few more failures in the last production run, but my CM is telling me they don't have anymore parts with these lot numbers.

So, at this point, I don't have the time to chase this anymore.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

diego_charles — Fri, 05 Feb 2021 04:42:52 GMT

Hi @chrispflieger

I just wanted to provide follow up.

Many thanks you for update and your efforts to follow my previous recommendation.

You may also check with your distributor if you are getting more failures with more lots .However I am not sure how your distributor may work. As far as I know some distributors use to request swap tests before checking for any possible quality incident. I am not implying that your chips have a quality issue, but just mentioning this to narrow down the issue.

Is there something else where I can help you with?

I am sorry for the delay.

Diego.

Re: LPC11e6x + FreeRTOS + optimizer = hard fault on _some_ chips

chrispflieger — Fri, 05 Feb 2021 16:08:18 GMT

We had another production run with chips made in Q4 of 2020, very low fallout on that run.

There's another run today, so I'll be monitoring it.

Right now, I'm considering this a one-off event. Which hopefully it is because I've got four more products in the pipeline based on this chip.