CRC calculation of read buffer / flash incorrect with data caching enabled

[thegreendroid]

Hi All,

I am trying to debug a problem in our product based on Kinetis K70 (revision 3) and MQX 4.1.

This is what I am doing -

1. Write N bytes to a memory location in external memory (DDR) or program flash (upper half)

2. Flush and invalidate the data cache line of just written memory using _DCACHE_FLUSH_MBYTES( addr, N ) and _DCACHE_INVALIDATE_MBYTES( addr, N )

2. Calculate CRC of just written memory (using the hardware CRC peripheral or a SW CRC implementation)

3. Compare calculated CRC with expected CRC, it is incorrect

If I disable data caching, the CRC is correct every time. This is obviously a cache coherency issue. Are there any silicon bugs in the Kinetis family that I need to workaround? Disabling caching is not an option at all as the performance hit is unacceptable.

Thanks for your help.

Message was edited by: Shreyas Balakrishna Corrected typo

[RadekS]

You are right that problem is obviously based on some issue with cache (it didn’t write whole data block, or you read wrong data – probably directly from cache instead of flash).

Unfortunately I was not able found _DFLASH_FLUSH_MBYTES( addr, N ) or _DFLASH_INVALIDATE_MBYTES( addr, N ) in MQX 4.1 code.

I suppose that these functions are your own or part of some MQX release modification.

Did you use flashX driver for write into flash or you use your own flash driver?

There are few modifications FlashX driver between MQX4.1 and MQX4.1.1. These modification adds additionally pipeline flush for ensures that all previous instructions are completed before executing new instructions in flash.

Could you please try implement flash_ftfe.c from MQX4.1.1 (in attachment)?

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[thegreendroid]

Hi RadekS,

Thank you for the reply.

I made a typo in my original post, I meant _DCACHE_FLUSH_MBYTES and _DCACHE_INVALIDATE_MBYTES. These are standard MQX macros (unmodified) defined in kinetis.h.

We used the standard FlashX driver from MQX 4.1. I'll give the new FlashX driver from 4.1.1 a try and post my results.

Thanks again.

- Shreyas.

[RadekS]

Thank you for your clarifying.

Could you please additionally check whether you first erased whole area (all sectors) where you write your data?

Idea:

Just for sure, could you please specify these values?

file_ptr->FLAGS & FLASHX_SECTOR_CACHE_ENABLED

file_ptr->FLAGS & FLASHX_FLASH_BUFFER_ENABLED

Did you call IO_IOCTL_FLUSH_OUTPUT or FLASH_IOCTL_FLUSH_BUFFER after you wrote to flash?

What I mean: When you want write into flash and this write will not occupied whole sector, flashX driver will hold these data in RAM until you add data to the rest of sector or you call IO_IOCTL_FLUSH_OUTPUT or FLASH_IOCTL_FLUSH_BUFFER. This technique is here for minimizing of write/erase cycle number.

If you will read flash by flashX driver it will return correct data which you wrote by flashX driver however it doesn’t mean that these data are already written into flash (it will read partially from RAM buffer).

If you are using simple data copy&paste (what I suppose), you will probably read 0xFF values inside last sector.

Please call IO_IOCTL_FLUSH_OUTPUT or FLASH_IOCTL_FLUSH_BUFFER for ensure that all data are written into the flash.

[thegreendroid]

Radek, after writing to flash we close the flashx driver by calling fclose. I had a look at the code and this should flush the buffer to flash. Also, we are not currently erasing the area prior to writing our data. Is this step required?

The same issue occurs if we read an entire file using MFS (and hence the eSDHC driver) into external DDR memory and then calculate the CRC on the buffer. It doesn't seem to matter if I use a SW CRC implementation or the HW one.

Were there any changes in the eSDHC driver from MQX 4.1 to 4.1.1? The problem is most likely in the flashx and the eSDHC driver.

[RadekS]

_io_flashx_close will flush buffers in case when FLASHX_FLASH_BUFFER_ENABLED=1.

Could you please confirm that BUFFERING is enabled?

Could you please confirm that SECTOR_CACHE is enabled?

flashX driver should check whether target sector is erased or not. I would like to just exclude potential issue in this area. Anyway, communicative writes are not allowed. Flash area must be erased prior writing. Could you please just try adding erasing command prior you write to flash for exclude of this potential issue?

I would like to apologize, but I am little bit confused. eSDHC module works as interface for SD/MMC/… cards. DDR memories are typically connected trough DDR1/2/LP SDRAM Memory Controller (DDRMC).

I suppose that you mean SD card instead of DDR RAM memory.

There are no changes in eSDHC driver between MQX4.1 and MQX4.1.1

If there are the same problems with flash and SDCARD I suppose that problem will be more likely in IO interface/CPU/compiler or in your code.

Could you please post here your code where you write into flash, read from flash and execute CRC checksum computation – that we can reproduce it on our side?

[thegreendroid]

Radek, we were able to fix the issue by looking at the esdhc driver implementation in uClinux. Their implementation disables cache during DMA operations.

Click on the below links to look at the source code -

linux-emcraft/esdhc.c at 7f201b2d12bb3747d575879197bc04c8f6e288bf · EmcraftSystems/linux-emcraft · G...

https://github.com/EmcraftSystems/linux-emcraft/blob/7f201b2d12bb3747d575879197bc04c8f6e288bf/arch/a...

We are doing the same in the esdhc driver in MQX 4.1 now and everything works as it should.

Should we be doing the same during flashx write operations?

[RadekS]

Thank you for your notification. Could you please send me your modifications of MQX esdhc driver that we can compare it with our MQX code and try debugging where is main difference?

I will report it into our bug database…

[sohamd]

I know its some old problem... is there any solution in MQX 4.1??

[thegreendroid]

Thanks for the reply again. I apologise for the late reply, we've been a tad busy.

I will try to reproduce the problem with a really simple bit of code like you suggested and post it here once I am successful in doing so.