USB Host Performance on i.MX6

[harald_krapfenb]

Hello community,

During the evaluation of the i.MX6 Quad for a product, we ran into USB performance problems.

We are using Freescale's kernel from BSP version 4.0.0.

We have connected a USB thumb drive to the OTG port and tested read and write speeds.

The device is USB 2.0 High-Speed compliant, so the port is operating at 480Mbps max.

The real bandwidth we measured was:

writing of 30MB file takes 10sec  -> 3MB/s

reading of 30MB file takes 2.5sec -> 12MB/s

Important for our project is the reading speed. The test was a simply large-file copy.

Why is the performance so bad? Do you know of any optimizations than can be applied to the Linux driver?

We have conducted the tests on our custom i.MX6Q design as well as on the SabreSDP with i.MX6Q on it. The performance was identical.

What do you suggest?

Thanks!

Harald

[baodongchen]

I'am using 3.10.53 android kernel, using otg for data transfer, currently the data transfer speed is about 12M bytes / s.

but we need more faster speed, any suggestions?

[xabiermarquiegu]

I'm also interested in news about improved USB performance in Linux.

I need as good performance as possible using usbnet. I'm currently testing in Linux 4.1.28, and CPU load is too high.

[dabrain]

Hello,

I would be interested for any update on this subject. Today i'm using yocto dora's branch on imx6. My kernel is a 3.0.35 and  with hdparms -t .dev/sdb with a USB 2.0 dongle, the trasnfer rate is limited to 20MB/s. On a PC it is more like 25MB/s.

Is there anyway to improve this performance ?

[dabrain]

Is there any update on this issue ? Is there any project where the speed has been increased?

[richard_stulens]

Hi Harald,

I'd suggest to check first  if the thumb drive or the destination drive is not limiting the throughput, .

I use the dd command for these tests and use the null device as destination to eliminate any delays on the destination side.

Here's what I ran on the Vybrid board that I have on my desk:.

# time dd if=/mnt/p1/Vybrid_Dual_Core_Operation of=/dev/null bs=512

97857+1 records in

97857+1 records out

real    0m 2.37s

user    0m 0.00s

sys     0m 0.00s

Vybrd is less performant  than i.MX6 and still I get a better read performance (21 MB/s)

50102818 bytes/ 2.37 sec = 21140429 bytes/sc (21 MB/sec).

So, I think you first  need to run some tests to find out where the bottleneck is.

Make sure you unmount / mount the device before each test, otherwise the read will be from the disk cache instead of the actual device.

Best regards,

Richard

[harald_krapfenb]

Hi Richard,

thanks for your answer.

The USB thumb drive was not the bottleneck, on a PC we can read with 25MB/s from it.

However, we are using now a SATA docking station, which achieves 60MB/s reading performance on a PC.

We changed the test to using 'dd', and reading to /dev/null.

The i.MX6 only achieves 18MB/s reading performance with the same docking station.

How can this be increased? Or is this the maximum throughput the i.MX6 is able to handle due to its internal bus structure and the like?

Best regards,

Harald

[richard_stulens]

I'm a bit confused now about the configuration that you use to test.

In your original setup yo had a USB thumb drive connected to the OTG port and copied a file from the USB thumb drive.

Now you mention a SATA docking station. Is that docking station connected to the SATA interface of the i.MX6?

If so, where is the USB driver that reads from the Thumb drive?

I'm assuming that the SATA dock has a USN controller and driver software built-in and that it encapsulates the SCSI commands on the SATA to USB MSC encapsulates commands.

In that case the i.MX6 USB driver is not involved.

Anyway, it is probably best to capture a USB protocol log so we can see what happens on the USB bus, That may tell us more about where to look, but if the SATA docks works the way I assume, than the issue is not in the USB driver.

Regards,

Richard

[harald_krapfenb]

Hi Richard,

The SATA docking station is for the i.MX6 like a thumb drive, and it's connected to USB OTG of i.MX6.

By "USB driver" I meant the USB low-level driver for the USB Hardware Controller of the i.MX6. You can't tell me that it's not involved ;-)

Do you know how I can capture such a USB protocol log?

Thanks,

Harald

[richard_stulens]

Ok, now I understand. It is not the same thumb drive that you are working with, but a SATA HDD drive with USB to SATA converter.

18 MB/s is indeed very slow. I did performace tests on i.MX31 several years ago and achieved 30 - 35 MB/s. and i.MX31 is much slower than i.MX6.

I guess there must be some issue in the Linux driver that limits the performance. I'm niot a Linux engineer though, so i'll have to look for someone from our Linux support  to pick up this thread.

For the USB protocol og, you need a USB protocol analyzer. It sits between the host and the device and captures everything that happens on the USB bus.

Best regards,

Richard

[harald_krapfenb]

That sounds good with respect to fixability...

I'm not sure if we have a USB protocol analyzer here, but you should be able to gather these numbers by yourself, using the SabreSDP reference board.

Awaiting your Linux expert :-)

-Harald

[richard_stulens]

Harald,

This is a note on what transfer rate to expect on High-speed USB .

The 60 MB transfer rate mentioned earlier must have been over a SATA connection to the PC because it is impossible to have a transfer rate of 60 MB over HS USB, The absolute (mathematical) maximum for bulk of HS USB is 53 MB/s and that is when packets are sent back-to-back.

With Mass Storage Class overhead (sending commands, receiving status, driver decision time etc.) the maximum will be closer to ~40 MB/s average.

Best regards,

Richard

[harald_krapfenb]

Hi,

Actually it was the same SATA docking station connected via _USB_ to a Linux PC.

I suspect that our calculation of the transfer rate contains some mistake, but the relative difference between the i.MX6 and the PC should be correct, since it was the same test conducted.

I will check that with my colleague, but please stick to that issue.

Thanks,

Harald

[richard_stulens]

Harald,

The file may have been in the disk cache on the Linux PC from a previous read. You will get much better results when you read the file a second time.

When measuring, please unmount the drive (or disconnect) to flush the disk cache. Then measure the first read of the file.

Also, make sure you are not using USB3.0 on the PC as that will obviously be much faster.

Best regards,

Richard

[harald_krapfenb]

Hi Richard,

I have measured reading bandwidth with the SabreSD i.MX6Q board. The test was now as you suggested.

Bandwidth for reading: 25MiB.

I have measured both 4.0.0 and 4.1.0 BSP releases, the latter is marginally faster.

Since the calculated bandwidth required for our application is 18 MiB, that should be sufficient - BUT:

Our device is a camera, which uses Bulk transfers, and the application uses libusb to get USB data.

Maybe that introduces some overhead.

If we approach 13MiB/s transfer rate from the camera, we are stuck - it won't go any higher.

I don't know where this limit comes from, but naively, I would think that USB on the i.MX6Q is simply too slow.

Compared to a Linux PC, we don't see this issue. However, we can simulate it there, by adding a proper delay into the loop.

What can you do? How can you help us?

Thank you!

Harald

[richard_stulens]

Hi Harald,

Somehow I did not notice this message when you sent it.

Now, there is not really anything I can do to help improve it other than sharing with you that the hardware should be capable of running at almost theoretical maximum.

The controller uses linked lists of transfer descriptors stored in memory to execute transfers. I've done a hardware-only test, long time ago on i.MX35, with a very small stand-alone program that simply kept feeding the USB controller. I could transfer over 40 MB/sec. There is no reason to assume that i.MX6Q cannot do the same.

There is of course overhead from protocols, like handshakes in BULK transport, class protocol overhead like commands and status transfer interleaving the data transfer, and then there is the OS overhead, which can be significant in OS's that have to operate on all kinds of different hardware.

One thing that can improve performance without much optimizing work is to increase the transfer size of the commands. For example, requesting 1000 sectors form a HDD in 1 command is much more efficient than 10 requests of 100 blocks.

Then there is also the competition for resources like memory access bus bandwidth etc. To achieve high USB bandwidth, USB needs to get a little over 60 MB memory & bus bandwidth on average. Peripherals like LCD or hardware accelerators take bus bandwidth away from USB. The USB controller must also get access to the memory before it's FIFO runs empty as a FIFO underrun will result in an aborted transfer which must be repeated. The USB controller can be tuned to load more data in the FIFO before starting the transmission, reducing the risk for underrun.

I've definitely seen better performance than 25 MB/sec, but that was not with Linux.

Best regards,

Richard

[harald_krapfenb · Accepted solution]

Seems that there is not much that can be done about this restriction.

I had the chance to test an early 3.10 kernel on SabreSD, and it was not faster either.

-Harald