TWRK64 MQX 4_2 MFS SD-CARD demo Volume and LFN long file name collision

francescocontin · ‎06-18-2015

Hi,

i'm testing the MFS in MQX 4.2 for a data-logger application.

Starting from the sd-card demo a simple function writes multiple files in multiple folders whith indexed names: every 200ms a row is added to the current file in the current directory and so on for 100 rows, 4000 files in 4000 folders.

Problems:

1) it's not possible to create a root sub-folder with the volume name.

2) using LFN long names "a:/log/FOLDER_##########/FILE_##########.csv" after 340 files the add file to cluster function returns a write error (the short name collide with an existing file).

Note: using a short name format "a:/log/####/log_####.csv" the test passed.

this is the log:

Shell (build: Jun 12 2015)

shell>

shell> Installing MFS over partition...

SD card installed to a:

shell> dir

TEST 0 05-14-2015 12:32:12 V TEST

LOG 0 01-01-1980 00:00:00 D LOG

shell> mkdir test

Error creating directory test --> error: TEST is the volume name

shell> write test

Error, unable to open file test. --> error: TEST is the volume name

shell>

shell> cd log

shell> dir

. 0 01-01-1980 00:00:00 D .

.. 0 01-01-1980 00:00:00 D ..

FOLDER~1 0 01-01-1980 00:00:00 D FOLDER_0000000000

shell> cd FOLDER_0000000000

shell> dir

. 0 01-01-1980 00:00:00 D .

.. 0 01-01-1980 00:00:00 D ..

FILE_0~1.CSV 111217 01-01-1980 00:00:10 A FILE_0000000000.csv

FILE_0~2.CSV 111217 01-01-1980 00:00:20 A FILE_0000000001.csv

FILE_0~3.CSV 55617 01-01-1980 00:00:24 A FILE_0000000002.csv

FILE_0~4.CSV 111217 01-01-1980 00:00:10 A FILE_0000000003.csv

FILE_0~5.CSV 111217 01-01-1980 00:00:18 A FILE_0000000004.csv

FILE_0~6.CSV 111217 01-01-1980 00:00:28 A FILE_0000000005.csv

FILE_0~7.CSV 111217 01-01-1980 00:00:36 A FILE_0000000006.csv

FILE_0~8.CSV 111217 01-01-1980 00:00:44 A FILE_0000000007.csv

FILE_0~9.CSV 111217 01-01-1980 00:00:54 A FILE_0000000008.csv

FILE_~10.CSV 111217 01-01-1980 00:01:02 A FILE_0000000009.csv

FILE_~11.CSV 111217 01-01-1980 00:01:10 A FILE_0000000010.csv

... (other files)

FILE~336.CSV 111217 01-01-1980 01:15:06 A FILE_0000000335.csv

FILE~337.CSV 111217 01-01-1980 01:15:26 A FILE_0000000336.csv

FILE~338.CSV 111217 01-01-1980 01:15:48 A FILE_0000000337.csv

FILE~339.CSV 111217 01-01-1980 01:16:10 A FILE_0000000338.csv

FILE~340.CSV 0 01-01-1980 01:28:02 A FILE_0000000339.csv

FILE_000.000 808464432 01-16-2004 06:01:32 DA FILE_000.000

shell> write FILE_0000000340.csv

Error, unable to open file FILE_0000000340.csv.

the LFN to SFN function retuns an error or a collision in file names.

a wrong FILE_000.000 is created

shell> write FILE_0000000341.csv

shell> write FILE_NEW

shell> dir

.... (all the old files)

FILE~342.CSV 0 01-01-1980 01:39:18 A FILE_0000000341.csv

FILE_NEW 0 01-01-1980 00:12:48 A FILE_NEW

changing names or skipping a number in the sequence allows to create new files.

it's better to move to KSDK 1.2 and fatfs?

Regard

Francesco Contin

RadekS · ‎06-24-2015

Hi Francesco,

I tried reproducing it on my side several times, however without success.

I formatted my SD card, created the directory structure as in your case and start write loop to files.

I used Shell_write() function as template and modified it for writing in the loop.

I was able write 500 files (with size 111217) without any problem.

In attachment you can find modified sh_write.c and demo.c file.

I am afraid that problem will be probably on your card (e.g damaged FAT table). You are right, FILE_000.000 parameters are strange and I suppose that it has something to do with this issue.

Did you have chance to test it with different SD card?

I hope it helps you.

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

lfschrickte · ‎09-01-2015

Hi,

I've posted this issue also: MFS on MQX 4.2 problem when multiple files with common name root in same directory which I think may be a "duplicate". In the topic I've created a simple command to use in the Shell example which demonstrates the problem easily - I think it may help reproducing/debugging the problem!

Thank you

Luiz

RadekS · ‎06-25-2015

Hi Francesco,

I would like to apologize.

MFS in MQX4.2 contains bug that cause wrong display data by dir command.

In fact these files are correct, however “dir” command do not display it correctly.

So, you already have on your sdcard file:

FILE~341.CSV 0 01-01-1980 01:28:02 A FILE_0000000340.csv

Therefore you cannot write them again by write command. However information about this file was simply messed and you see this line:

FILE_000.000 808464432 01-16-2004 06:01:32 DA FILE_000.000

If you look at these files in PC, all data should be OK.

There is probably some issue with long names at edge of sector sizes…

I already reported it and it should be fixed in next MQX release.

I am sorry for that I didn’t realize that issue in “dir” command on my side is also root cause of behavior on your side (my dir output was messed up by little bit different way because dir display content of instead of file name).

I hope it helps you.

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

francescocontin · ‎06-25-2015

Hi Radeks

to complete informations:

1) my sd-card is a Trascend SDHC 4Gb formatted in Windows 7 FAT32 with default cluster size / allocation unit (i32k)

2) in the loop of your example Shell_write_loop function if you add something like this:

if (!error)

{

fd = fopen(abs_path, "w");

}

if (fd && !error)

{

write(fd, abs_path, strlen(abs_path)); // <<<< write header

fclose(fd);

fd = fopen(abs_path, "a"); // <<<<< append rows

if (fd)

{

int32_t i,j, idx = 0;

for(j = 0; j < 99; j++)

{

sprintf(buf, "\r\n%010d;", idx++);

write(fd,buf, strlen(buf));

for(i = 0; i < 99; i++)

{

sprintf(buf, "%010d;", i);

write(fd,buf, strlen(buf));

}

sprintf(buf, "%010d", i);

write(fd,buf, strlen(buf));

}

fclose(fd);

}

the append file access will fail at different times with different lenght of LFN.

I think that the same BUG is in MFS for KSDK 1.2

in your opinion, do you think

- there will be a patch for MQX 4.2 in short time (compatibly with the complexity of the bug)?

- it's better to move to KSDK 1.2 and wait for a patch or the next programmed release?

- formatting the sd-card with different allocation unit could be a work-around?

have a great day

F.C.

RadekS · ‎06-29-2015

Hi Francesco,

I suppose that this bug is most probably also in KSDK 1.2 (I didn’t test it), MFS in KSDK should be almost the same (difference should be only in fio/nio interface).

According current time plan, KSDK 1.3 should be released during end of September 2015. MQX 4.2.0.1 don’t have special time plan yet, but it should be also released during next few months.

Patch/workaround for this issue I will place here, when I will get it from designers.

I suppose that formatting the sd-card with different allocation unit will just lead to display bug sooner or later – however I didn’t test it. Therefore I am afraid that this couldn’t be named as workaround.

I hope it helps you.

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

RadekS · ‎07-03-2015

Hi Francesco,

In attachment you can find fixed MFS files as workaround until official patch/new version will be released.

It fixed problem with dir call (description: walk through a directory chain might access extra sector due to a +-1 bug, in particular this caused difficulties for long filenames spanning across cluster boundary).

It also fix problem with volume label. With this fix, new directory with the same name as volume label could be now created.

Volume label is no longer displayed in standard dir call. For display volume label, please use command dir * v or dir * vm in root directory.

I hope it helps you.

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

neilsmith · ‎08-12-2015

Hi RadekS,

I think the above fix where the "sector_count--" is moved also needs to be done in the following functions:

MFS_lfn_chain_store

MFS_free_dir_entry

MFS_check_dir_empty

Please confirm

Regards

Neil

RadekS · ‎08-17-2015

Hi Neil,

you are most probably right. Thank you for notification.

I discussed it with designer and it seems that these functions should also contain the same bug fix.

Fix will be implemented into next release.

Just note: MFS_lfn_chain_store function is not used function yet (it is prepared for next use). So, implementation of bug fix into this function will have no effect on MFS behavior.

I hope it helps you.

Have a great day,
RadekS

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------