Hi all,
We have been battling issues with USB memory stick support for some time with the i.MX28 Windows CE BSP. We have implemented a number of 'fixes' ranging from Microsoft updates, Errata Work-arounds and general Windows CE fixes from other BSPs or user experiences found online and in these forums.
We have an issue where after several insertions of a memory stick the USB port appears to lock up and on the next device attachment the 'AttachDevice' process fails at DEVICE_CONFIG_STATUS_SCHEDULING_GET_DEVICE_DESCRIPTOR_TEST. This is actually the first point in the attach process that an attempt is made to communicate with the device and uses ENPOINT 0 on a control pipe created during earlier stages of the AttachProcess.
Once the port has failed it no longer responds to any other USB device and fails at this same point every time. The pipe has the halt flag set and is non-functional. The only way to recover seems to be to power cycle the unit.
We have found forum posts that recommend increasing the delay from ResetAndEnablePort (the previous step) and sending any data. By default in the BSP this is 10ms but some devices may not be ready to communicate at this point and still coming out of reset. We have tried increasing to 100ms and the results initially seemed promising but we could eventually get it to fail again after repeated insertions.
We have used a USB logger to see what happens on the bus through several insertions to failure. For each successful attach and detach we see no bus errors and can decode the SCSI packets. The last successful transfer on removal was the last SCSI Test Unit Ready packet which is sent to poll for disk removal.
When we insert the disk and there is a failure we just see babbled data out of the USB port. There isn't a single valid packet but random data bytes being received by the analyser following device reset. The analyser cannot tell if the memory stick is babbling or the i.MX28 processor as it is just receiving random bytes constantly.
I have attached the last successful transfer below (note that NAKs and SOF are hidden). The memory stick was unplugged in the middle of this screenshot which is why the analyser detected suspend (no bus activity). The Reset following the suspend is when the stick was re-inserted. You can see that following this we just have babbled data. The bytes are random values.
On a successful remove / attach we see that after the suspend there are two resets and we have the initial 'get descriptor test' followed by the address set and the rest of the successful enumeration.
I have been adding debug messages at various points to try and figure out what could be going on. I am not sure why the USB port has the random data on it. Perhaps there is a corrupt qHead/qTD that is just running through memory sending out random data bytes but the data is not in any of the proper transfer types (SETUP/IN/OUT).
The transfer queue should be torn down on removal and cleared out and then recreated on the first transfer on attach.
Has anyone seen anything similar or could offer some debugging advice?
Thank you in advance, Mark
As a quick test I also tried with SBUSCFG at 6 and SDIS turned off (streaming mode enabled) which is the default for the BSP but this has the same generated packets and same error. I had changed SBUSCFG to 0 and enabled SDIS to 1 after reading other threads on iMX28 USB errors but this made no difference.
Just another thought. As the controller doesn't seem to be able to generate the SOF packets correctly then surely the error condition can't be because of corrupted transfer descriptors as the controller creates the SOF itself.
I searched through a valid transfer and found the same data as the CRCs for valid SOFs so I am certain this is just generating Sync and CRC fields:
If you look below I have lined them up. On the left is a valid sequence of SOFs with rolling frame number. On the right is the data seen by the analyser in the error state when just the CRC seems to be output. You can see the 'Data' field in red on the right follows the CRC field sequence for valid SOF on the left.
In fact the 0x0F 'CRC' is misinterpreted by the analyser as a PID (last four bits complement the first) as does the 0x1E later.
So I am pretty confident now that in this state the controller is generating SYNC and CRC but not the data in between. This goes for SOF and other packets too.
I hope this additional info is useful to help with this issue.
Kind regards, Mark
Sorry for the update delay. I have been working on other projects while trying to get to the bottom of this USB issue!
I have been performing lots of different diagnostics tests, checking the asychronous qHead list, the qTDs of each qHead. I have also been scanning through various USB protocol analyser logs and we also bought a new oscilloscope with built-in USB decode which has helped.
As an update I can confirm that I can still reproduce the issue where the USB port seems to lock up after several insertions/removals of a USB device.
Once in this state attaching any USB device the port no longer generates valid packets. I have been able to see this on a USB protocol analyser but also seen it on the oscilloscope. The packets start with valid SOF but the data content is invalid. In fact I believe the SOF and CRC are generated but the data (including PID) is not valid. As the SOF packets contain a rolling frame number the CRC would be rolling too.
An example invalid packet captured on the scope - these start being generated the second the USB device is connected and port enabled:
You can see from the above that the packet starts with a sync, then a random byte (shown as PID error). I believe the random byte to be the CRC.
I noticed a pattern in my USB trace logs when the port is in the stuck state. Within the random data (which I believe to be attempted SOF with missing PID and frame number) I can see attempts at communication when the attach process attempts to send the initial setup packet for the first control transfer.
The reason why I think this is this transfer is that the pattern I see is a reset condition followed by three attempts at sending the same data. This is repeated three times (the attach process tries three times) and eventually the port stops sending anything as the attach process disables the port on error.
This is the expected pattern of data transfers during the USB device attach process. It tries to send an initial control transfer which will try three times (USB CERR counter). If this fails the attach process resets the port and tries again. It tries comms three times and then disables the port.
An example of this is shown above. We see the pattern of 08 packet then three packets ending in D7 29. This process is attempted three times.
Interestingly when the port is functional this same initial control transfer looks as follows:
If you look at the expected transfer the Setup Packet should have CRC 8 and the following Data0 packet CRC D729. When the port is in its error state I see a data 8 (sync followed by 0x08). Then a data packet ending in D729.
So from this comparison I strongly believe that in the error state the port is trying to send the packet but just the CRC seems to be correct.
Now analysing the data packet differences there seems to be some correlation between the data in the valid packet and when in the error state but this is not clear.
Error State: 60 00 80 00 00 10 00 D7 29
Working State: 80 06 00 01 00 00 08 00 with CRC D7 29.
From my analysis is definitely seems that the port can get into a state where the Sync and CRC appears to be generated but the data in between is not valid.
This shows in the fact the SOF are missing the PID and frame number and the attempted control transfers are incorrect too. But the CRCs still seem to be valid.
I am assuming that the controller generates the sync pattern itself as this is always the same but perhaps the PID and rest of the data being fetched is corrupted. Interestingly though (I can't believe this is just chance) the CRC seems to be valid too.
I have looked at the transfer being set up and this seems to be correct. This is probably why the CRC is correct. I am now puzzled as to what could be causing the resultant corrupted transfers.
From the slight correlation between the data in the error state and working state I am wondering if this is something else to do with how the data is transferred on the internal busses. For reference I have SDIS set and SBUSFCG at 0. I will retry with these set to my original settings of SDIS off and SBUSCFG at 6.
This may need help from Freescale as well as Adeneo.
Kind regards,
Mark
I have been looking in more detail at the USB protocol analyser output. On a failed attach there is random data on the USB bus shown in my protocol analyser. But when I look at the timing and bit levels I can see that they are produced every 1ms (USB frame duration at full speed) and each attempt at a transfer starts with a valid SYNC packet of 0x01.
The PID that follows the SYNC is the random aspect and as the protocol analyser does not detect a valid PID it just shows 'data'.
I have included an example below and every packet starts with a valid SOF but then a random PID:
So to me the controller is consciously trying to send something but that something has an invalid PID which helps strengthen the fact that it might not be real data.
Occasionally (by chance) the PID field happens to have the second four bits as the complement of the first four and match a valid PID so my analyser shows an attempt at decoding it, but is is not a real packet. The random data just happened to have something that matched a valid PID. The rest of the packet is missing because it wasn't intentional. Anyway in this case it happened to look like the start of a PING?!
So I now have to understand what puts the controller in this weird state and how to prevent it or have a recovery mechanism on device re-insertion to save having to power cycle.
Initially it could be one of the Erratum such as suggested by Yuri (although according to the iMX6 version of this erratum ERR006308 the host controller only sends SOF packets in this state but I am not seeing valid SOF.)
The only other erratum is Errata 2858 USB controller may access a wrong address for the dTD (endpoint transfer descriptor) and then hangs. I have tried using other SBURSTCFG modes to work around this with no luck but perhaps there is something in postponing freeing the last TD in case memory could be re-used while the controller is still pointing to it.
Hopefully someone can help me understand this or if there are any fixes applied to other BSPs that I may need. Windows CE support is not as readily available as Linux though!
Mark
