USB Bootloader: How to make FAT16?

[lpcware]

Content originally posted in LPCWare by DEGoodmanWilson on Fri Aug 26 06:28:12 MST 2011
I've gotten CodeRed's USB bootloader demo ported to my LPCXpresso1769, and am generally quite pleased with it. However, it appears that, as with the NXP app note it is based on, it implements a FAT12 filesystem. I've never dabbled in filesystems before, but how difficult would it be to update the code to implement a FAT16 filesystem?

[lpcware]

Content originally posted in LPCWare by Rob65 on Sun Aug 28 01:19:17 MST 2011

---

Quote: DEGoodmanWilson
I'll try to borrow a second Mac and get the debugger going in the next few days to see if Windows and OS X are attempting to write the file in a different way.

---

That would be great.
I was already thinking of creating a special bootloader that prints out information on sectors read/written via a UART port. that might already give enough information to proceed forward.

A few things would be worth knowing:

[LIST]
[*]Are sectors in a cluster all written in one sequential flow without interruptions.
[*]Is the FAT updated before or after writing the cluster
[*]When is the directory entry created/updated
[/LIST]
If the sequence in which sectors are being written is the problem we need to rely on a valid FAT structure in order to determine the offset in the flash of the given sector.

Rob

[lpcware]

Content originally posted in LPCWare by DEGoodmanWilson on Sun Aug 28 00:59:38 MST 2011

---

Quote: Rob65


The funny thing is that the FAT type (12 or 16 bits) is just determined by the size of the medium. If the disk contains less than 2^12 - 6 (or 4, I'm not sure) clusters it is a FAT12 type, if it is more it's FAT16. So a floppy disk is always FAT12.
If Mac is able to read/write standard FAT floppy disks then it should be able to read small flash drives as well.

---

I thought that was what I read too, but I wasn't entirely certain.

---

Quote:

I am curious if there are other Apple users who can confirm this problem. I will open a new thread for this (to keep the Apple issue and the FAT16 conversion separate).

---

Probably a good idea, but see below...

---

Quote:

One thing though ... I found that the bootloader does not use the FAT. It just assumes that Windows (or OS-X or Linux or ...) writes the file in a sequential way.
The first 4 sectors contain the bootsector, FAT and root directory and from that point on, the sectors are sequentially mapped onto the Flash. So sector 4 maps on offset 0x0000 in the flash, sector 5 on offset 0x0200 etc ...
It could well be that OS-X writes in a way that is not  compatible with this.

---

Interesting find. I wonder if you haven't hit the nail on the head here. If this is right, then the use of FAT12 is a red herring (since I was able to read files just fine with the Finder, and I could use 'cp' to copy files onto the flash); the issue is that OS X's Finder isn't writing the file in a way that the bootloader is comfortable with. I'll try to borrow a second Mac and get the debugger going in the next few days to see if Windows and OS X are attempting to write the file in a different way. If they are, then I suspect you were originally right in thinking that there is no issue with using FAT12 after all, and that the real issue is in this assumption.

[lpcware]

Content originally posted in LPCWare by Rob65 on Sun Aug 28 00:53:32 MST 2011

---

Quote: DEGoodmanWilson
S'ok. I woke up on the wrong side of the bed this morning myself :D

---

Good thing you were not cicked out of the wrong side :)

I just checked out some pointers regarding the FAT system:

[LIST]
[*]FAT on Wikipedia
Good reference, almost everything you want to know is in here.
[*]ECMA-107
A more formal written document, I think a nice doc. to read
[/LIST]
And I also checked the sources of the rdb1768cmsis2_usb_bootloader sources.

---

Quote: DEGoodmanWilson

I can't find anything definitive that says OS X has problems with FAT12;

---

The funny thing is that the FAT type (12 or 16 bits) is just determined by the size of the medium. If the disk contains less than 2^12 - 6 (or 4, I'm not sure) clusters it is a FAT12 type, if it is more it's FAT16. So a floppy disk is always FAT12.
If Mac is able to read/write standard FAT floppy disks then it should be able to read small flash drives as well.

I am curious if there are other Apple users who can confirm this problem. I will open a new thread for this (to keep the Apple issue and the FAT16 conversion separate).

The automatic FAT12/16 identification on the number of clusters means that we need to do a few tricks to make this happen. First the FAT needs to be enlarged, since that is too big to fit into RAM (and we only need a very small portion of the 'disk' to be usable), we can just return FFF7 (defective cluster) for all other sectors in the FAT.

One thing though ... I found that the bootloader does not use the FAT. It just assumes that Windows (or OS-X or Linux or ...) writes the file in a sequential way.
The first 4 sectors contain the bootsector, FAT and root directory and from that point on, the sectors are sequentially mapped onto the Flash. So sector 4 maps on offset 0x0000 in the flash, sector 5 on offset 0x0200 etc ...
It could well be that OS-X writes in a way that is not  compatible with this.

Regards,[INDENT]Rob
[/INDENT]

[lpcware]

Content originally posted in LPCWare by DEGoodmanWilson on Sat Aug 27 05:22:41 MST 2011
So, I tried copying the existing firmware out of the FAT12 FS in OS X with interesting results. OS X complains that it cannot read or write the file, but nevertheless produces a perfect copy of the flash image (when compared against the image copied in from Windows). So that says something, namely that OS X can read from the bootloader just fine.

So, I copied the same image back onto the LPCXpresso in OS X. I think that the bootloader is in fact executing the user code in this case (as I have turned checksums off for the time being), because it does not load up the USB mass storage driver (i.e., it doesn't show up as a drive on my Mac) unless I short the trigger to ground. So far so good. But what I copy off the "drive" at this point bears no resemblance to the code I uploaded. Thus, although OS X is happy to read the "drive", it cannot write to it correctly. The mangled firmware image is 255 bytes long (where the original is 1140 bytes).

Indeed, copying a very different (and much smaller) firmware image and copying it back out all on OS X yields the same results: an image 255 bytes long that is very nearly identical with the other damaged image.

**UPDATE**
One last observation: Copying the image from the command line WORKS. Finder is trying to do something that the bootloader does not like, where a simple 'cp' gets the job done. Curious!

I don't know how useful this all is, frankly, but here it is anyway. I'm happy to supply the images I wrote and read back out if it's any help.

[lpcware]

Content originally posted in LPCWare by DEGoodmanWilson on Sat Aug 27 04:28:16 MST 2011

---

Quote: Rob65
Ehm, sorry ...
the question was just meant as a question - not as a smack in the face :eek:

---

S'ok. I woke up on the wrong side of the bed this morning myself :D

---

Quote:

I had no idea that OS X has problems with FAT12.
I though that 12 bits would be supported as well.
I should really add a Mac to my collection to prevent me from making those stupid remarks ;)

---

I can't find anything definitive that says OS X has problems with FAT12; just a lot of frustrated users or older cameras. Indeed, my MacBook does not complain in the least: It shows the "drive" with "firmware.bin" in the folder; it lets me delete "firmware.bin", and permits me to copy over a new binary image without complain. But then the bootloader appears to get stuck. Trying again from Windows works perfectly. About to see if I can copy the image out without errors.

---

Quote:

Good to hear this.
This triggers us to check if, and how, this can be fixed.
I still need to look at CodeRed's bootloader (I like to remove the 64 kB constrains and add CRP as well) and I don't think that upgrading to FAT16 is that much of a hassle.

I don't have too much time the next few weeks but I could spend a few hours in the evening to read up on FAT and the read the application note for extra hints. It's an easy job as soon as I've figured out the differences.

Rob

---

Thank you, this is a very generous offer.

Having only one computer handy, I can't debug the bootloader and test it with OS X at the same time (as I installed Windows in its own Bootcamp partition, I can't use a PC emulator to accomplish this either, grr.)

That said, I'm happy to help out where I can. I don't really understand the differences between the two filesystems from the host's perspective. I presume the difference lies in  that FAT12 requires writes to the file-allocation-table to be 12bit words, and FAT16 as 16-bit words? If you find a good resource on the subject online, please share it here so I can read it too, and offer what assistance I can render.

[lpcware]

Content originally posted in LPCWare by Rob65 on Sat Aug 27 03:30:50 MST 2011

---

Quote: DEGoodmanWilson
Geez. Thanks for the warm welcome.

---

Ehm, sorry ...
the question was just meant as a question - not as a smack in the face :eek:

I had no idea that OS X has problems with FAT12.
I though that 12 bits would be supported as well.
I should really add a Mac to my collection to prevent me from making those stupid remarks ;)

Good to hear this.
This triggers us to check if, and how, this can be fixed.
I still need to look at CodeRed's bootloader (I like to remove the 64 kB constrains and add CRP as well) and I don't think that upgrading to FAT16 is that much of a hassle.

I don't have too much time the next few weeks but I could spend a few hours in the evening to read up on FAT and the read the application note for extra hints. It's an easy job as soon as I've figured out the differences.

Rob

[lpcware]

Content originally posted in LPCWare by DEGoodmanWilson on Fri Aug 26 21:14:34 MST 2011
Geez. Thanks for the warm welcome.

The problem with FAT12, as I understand it, is that it is not properly supported by OS X. Indeed, I cannot upload new firmware images from OS X without causing problems (code is never run). This is a problem, as many of my customers use Macs, and they will want access to firmware updates as well.

Of course, perhaps I am mistaken in my assessment of FAT12? As I said before, I am not very familiar with filesystems.

[lpcware]

Content originally posted in LPCWare by Rob65 on Fri Aug 26 13:16:33 MST 2011
Why would you want to do this?
There is no need to use FAT16 and FAT12 works perfectly well as it is right now.

Rob