how can I prevent corruption in sd card writes, MPC5748G

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

how can I prevent corruption in sd card writes, MPC5748G

5,854 Views
liam_battershel
Contributor I

Hi All,

Ive been working on writing to a text file on an SD card using an MPC5748G with processor expert generated code(SDHC,FATFS) this works ok but i am trying to increase the write speed. I have found that the speed of the transfers is proportional to the amount of data I write to the SD card however I can only go up to around 1050 characters in a call to f_write or I end up with corruption of the file by which I mean I see null characters (\0) written to the text file.

can anyone help me to either improve the write speeds with transfers < 1000Bytes?  or could someone help me to increase the amount of data I can transfer at a time without it getting corrupted?

Ive added my data from the transfers from 1 to 1100 characters to the sd card done 20 times each

Many thanks

Liam 

0 Kudos
Reply
9 Replies

5,613 Views
cezarionescu
NXP Employee
NXP Employee

Hi Liam.

We are testing SDHC stack for read and write performance performance using both raw and file system mode.

The results we have for file system mode using a 32 GB SD card :

read speed = 10773 Kbps, for reading 14Kbytes of data (i.e. 2 files of 7 Kbytes each)

write speed = 4086 Kbps, for writing 14Kbytes of data (i.e. 2 files of 7 Kbytes each)

in raw mode the numbers are:

read speed = 13227 Kbps

write speed = 5330 Kbps

so, the file system overhead is small.

We do test also write of text files, but we don't measure the write speed for that operation and we're using a small (several bytes) text file to write.

There is no configurable user setting which influences the max write speed. The file system is calculating the chunk to be written based on file system type and size to be written.

the generic API call for writing is f_write(&fd, fileBuffer, bytesToWrite, &bytesWritten);

So technically, when you pass the full file size the write should be done at maximum speed

Alternatively you should consider eMMC which is much faster than SD.

0 Kudos
Reply

5,613 Views
liam_battershel
Contributor I

Hi Cezar,

Thank you for the reply and the information on the speeds.

OK so because I can't improve the write speed of small amounts of data I have been trying to find out what leads to the corruption of data I am seeing...

I have created a small function based on the test functions provided to write a certain number of bytes from a buffer to the SD card 

int SDCardBuffer1_count = 0;

/* Data buffer for writing */
ALIGNED(4096U)
uint8_t SDCardBuffer1[DATA_BUFFER_SIZE] __attribute__((section(FEATURE_CUSTOM_DATA_SECTION)));

FRESULT WriteBuff_1()
{
/* File object */
FIL fd;
/* File stat object */
FILINFO fst;
/* API result code */
FRESULT status;

static UINT bytesWritten;
UINT bytesToWrite;


status = f_open(&fd, path, (BYTE)(FA_WRITE | FA_OPEN_APPEND));
if(FR_OK == status)
{

bytesToWrite = SDCardBuffer1_count;
bytesWritten = 0U;
//randomizeBuffer((uint8_t *)&g_dataWrite[0], bytesToWrite);

status = f_write(&fd,SDCardBuffer1 , bytesToWrite, &bytesWritten);

if(FR_OK == status)
{
status = f_sync(&fd);
}
if(FR_OK == status)
{
status = f_stat(path, &fst);
}
if(FR_OK == status)
{
printFileInfo(path, &fst);
status = f_close(&fd);
}
else
{
(void)f_close(&fd);
}
}
return status;
}

This function seems to work ok however I am seeing corruption of the data on the sd card with NULL characters added into the text files.

SDHCCorrupteddata.PNG

 I used the debugger to see if there was any null data in my buffer but there isn't so i really don't know where it comes from. i have run some tests and have found the conditions that lead to the corruption... it seems as though if I only write small (< ~1000 characters)  amounts of data to the SD file it works fine, if I only write large(> ~1000 chars) amounts of data its also ok...but if I do a mixture of writes some more than aprox 1000 characters and some below aprox 1000 characters then i will see corruption of the data (the boundary isn't 1024 though I checked that). the code below shows this.(the databuffers are prepopulated)

so this would work...  

Mount();

for(i = 100;i < 1000;i = i + 10){
SDCardBuffer1_count = i;
timebefore = RTC_DRV_GetTicks(0);
(void)WriteBuff_1();
timenow = RTC_DRV_GetTicks(0);

}

UNMount();

and this would work...

Mount();

for(i = 2000;i < 40000;i = i + 100){
SDCardBuffer1_count = i;
timebefore = RTC_DRV_GetTicks(0);
(void)WriteBuff_1(); 
timenow = RTC_DRV_GetTicks(0);

}

UNMount();

but this would lead to corruption...

Mount();

for(i = 500;i < 1500;i = i + 10){
SDCardBuffer1_count = i;
timebefore = RTC_DRV_GetTicks(0);
(void)WriteBuff_1(); 
timenow = RTC_DRV_GetTicks(0);

}

UNMount();

Edit: I have just noticed when I was tidying up my code to put on here that there seems to be a time factor(how long waited between calls to WriteBuff_1()) that effects if there will or will not be corruption to the file.

If anyone could help me in figuring this out id be very grateful.

Many thanks

Liam

0 Kudos
Reply

5,613 Views
cezarionescu
NXP Employee
NXP Employee

Hi Liam.

We retest using the described failing use cases (i.e. use a mix of small and big packages in raw mode). We'll come bask as soon (i.e. rather sooner than later) as we have some results of this test

Have you tried also to use the file system (alternate big and small files) and check whether the same issues happens?

0 Kudos
Reply

5,612 Views
liam_battershel
Contributor I

Hi Cezar,

Any luck checking the middleware yet? it would be good to know if i have made a mistake or if its something i need to get an updated sdk to fix?

Many thanks

Liam

0 Kudos
Reply

5,610 Views
Gus_Guzzler
NXP Employee
NXP Employee

Hi Liam,

The solution to this issue is to change these 2 lines of code in S32_SDK_S32PA_RTM_3.0.0\middleware\sdhc\fatfs\usdhc_fatfs_impl.c file:

line 306 :

                    status = SD_ReadBlocks(card, g_bufferedData, sector, 1U);

to

                    status = SD_ReadBlocks(card, g_bufferedData, sector + j, 1U);

and line 397:

                    status = SD_WriteBlocks(card, g_bufferedData, sector, 1);

to 

                    status = SD_WriteBlocks(card, g_bufferedData, sector + j, 1);

To improve writing/reading speed use buffers address location that are aligned by 4 bytes, and sizes of the buffer multiple of 512 bytes. If the size of the buffer is to big for your application use the size of the buffer at least multiple by 4 bytes. If these conditions are not meet additional copy will occur in to internal buffer that is aligned by 4 bytes.

Best regards,

Nicu.

0 Kudos
Reply

5,610 Views
cezarionescu
NXP Employee
NXP Employee

Hi Liam.

We were able to reproduce via upgraded tests the issue that you reported on your side. We're preparing some fixes and will reply when we can share what those fixes are. I hope that will be pretty soon.

0 Kudos
Reply

5,613 Views
cezarionescu
NXP Employee
NXP Employee

Hi Liam.

We've opened an internal ticket for adding new tests as per your comments. We did not yet add those tests. I'll increase the priority of the task.

0 Kudos
Reply

5,613 Views
liam_battershel
Contributor I

Hi Cezar,

Have you managed to test the failing use cases yet as discussed above? I'd love to hear about your results if you have?

Many thanks

Liam Battershell

0 Kudos
Reply

5,613 Views
liam_battershel
Contributor I

Hi Cezar,

Thanks again for the support. The only thing i've tried is using the FATFS provided in S32DS and i've used the methods from the "fatfs_middleware:fatfs" component in the same way as the example test functions from the example project "sdhc_fatfs_mpc5748g" from SDK PA RTM v3.0.0.

is there another way i can write a text file to the sd card? if so ill try it.

Many thanks

Liam 

0 Kudos
Reply