topic Re: Maximal sustained transfer rate in FlexSPI of RT685? in i.MX RT Crossover MCUs

Maximal sustained transfer rate in FlexSPI of RT685?

Vyssotski — Thu, 19 May 2022 00:14:55 GMT

I am interested in maximal sustained transfer rate of FlexSPI module of RT685 processor. Please share your experience.

My personal tests lead to results below expectations. Maximal instantaneous transfer rate of FlexSPI of RT685 is achieved at 200 MHz clock DDR and constitutes 400MBps for 8-bit bus. This transfer rate is visible at oscilloscope attached to the bus of RT685 evaluation kit, but only for short 512-byte packets. I used slightly modified code of examples "psram_polling_transfer" and "psram_dma_transfer".

FlexSPI of RT685 can be accessed through AHB bus and/or IP bus. AHB bus is faster, but DMA is not supported there. IP is slower, but it has DMA support.

When reading through AHB with flexSPI at 200MHz DDR I achieved maximally 115.84 MBps sustained data rate. There was no any processor activity in the background and default MCU core clock was 250 MHz. The transfer through AHB is done by memcpy() command by writing 1kB blocks to flexSPI memory space. It is known that Cortex-M4 that is very close to Cortex-M33 needs two clocks to transfer 32-bit word from memory to memory by memcpy(). Thus, for 250MHz expected transfer rate memory to memory is 500 MBps. I was able to get only 1/4 of this for unknown reason. This speed may be sufficient, but it is achieved with practically 100% processor core utilization. This is not good because processor resources are needed for other tasks. To test dependence of transfer rate from flexSPI clock I measured transfer rate with 62.5 MHz DDR. I achieved 108.25 MBps sustained data rate that is only slightly smaller than sustained data rate with 200 MHz clock. I attached the oscilloscope screen shot of this test. Any ideas how to get higher transfer rate and/or smaller processor utilization with flexSPI at AHB? I probably can use Tensilica HiFi4 core for memcpy() with flexSPI at AHB, but Tensilica is energy hungry, and it is a pity to run it only for memcpy() function.

Test with DMA (through IP bus) leaded to maximal sustained WRITE data rate 62.6 MBps and sustained READ data rate 34.712 MBps only. The higher data rate is achieved only for the first 128 bytes of the packet. To test this I configured flex SPI for clock 41.666 MHz DDR. As you see from the attached oscilloscope picture showing WRITE command, elevated to 83.333 MBps data rate is achieved only for the first 128 bytes containing values from 0 to 127. Then it drops to the data rate of ~62.5 MBps.

This is disappointing for the DMA of the current processor, especially for READ speed 34.712 MBps picture of which I don’t show. The reason of disappointment is shat DMAC of PSoC 6 processor from Cypress clocked at 100 MHz achieves only slightly smaller sustained DMA read speed 32.576 MBps (8 bit Octal SPI, 50 MHz SDR clock, block size 14190 bytes). Its Cortex-M4 core was also clocked by 100 MHz. It seems to be that NXP RT685 clocked by 250 MHz has the same Octal SPI DMA transfer rate as Cypress PSoC 6 at 100 MHz. I expected more from 2.5x higher clock rate. I also know that DMA clock can be different from the processor core clock. Does RT685 has 100 MHz DMA clock with 250 MHz core clock?

To be honest, I did not try to optimize "psram_dma_transfer" example. If you achieved higher DMA READ rate of flexSPI, please let me know.

Re: Maximal sustained transfer rate in FlexSPI of RT685?

jingpan — Tue, 31 May 2022 10:28:29 GMT

Hi @Vyssotski ,

The IP TX FIFO size is 128*64bit and the IP RX FIFO size 64*64 bit. So I think the first 128 byte is faster is just because the FIFO size and watermark. Please refer to 33.4.9.1 in user manual. Can you please try to adjust the watermark size to see if there is improvement?

Regards,

Jing

Re: Maximal sustained transfer rate in FlexSPI of RT685?

Vyssotski — Fri, 03 Jun 2022 13:42:31 GMT

Dear Jing, thank you for your comment.

I was able to increase DMA reading/writing speed by increasing the length of the packet from default in example 1K to maximal supported by DMA controller 4K. Changing the packet length increased sustained data read rate from mentioned 34.35 MBps to 50.20 MBps in 31.25MHz DDR mode (peak transfer rate 62.5 MBps). Soldered on evaluation board psram does not support 4K packets, and memory output above 1K contains rubbish. This is not important for our speed test.

I noted that during DMA writing clock stretching is observed after 128 bytes in the modes with the clock exceeding 31.25 MHz DDR (i.e. 41.66 MHz DDR). DMA reading behaves differently. Clock stretching is observed after reception of approximately 2625 bytes during 26.25 us with the clock 50MHz DDR (100 MBps peak data rate). You noted that IP RX fifo size is 512 bytes. Thus, it unlikely that observed after 2625 bytes clock stretching can be linked with watermark level of this buffer. However, it can be linked with some data cashing. As far as I understand, DMA reading procedure does not use watermarks and generated by them interrupts at all.

Also, the main problem is not a clock stretching observed after some amount of data, but fixed duration of packet + time gap after it before possibility of reading/writing next packet. For 4KB packets reception the packet time almost does not depend on bus frequency:
200 MHz DDR, 4K -> 80.8 uS -> 50.639 MBps
50 MHz DDR, 4K -> 81 uS -> 50.568 MBps
31.25 MHz DDR, 4K -> 81.4 uS -> 50.319 MBps.

I probably will stay with 31.25 MHz DDR DMA mode 4K block with sustained data rate 50.20 MBps until new ideas will appear. One can try to use watermarks in 512-byte buffer, but it is questionable whether it will be more effective than 4K DMA reading that should handle buffer filling with introduction of necessary delays automatically.

Re: Maximal sustained transfer rate in FlexSPI of RT685?

Vyssotski — Sun, 05 Jun 2022 00:09:38 GMT

Dear Jing, I want to add some comments to my previous letter.

I have tested DMA data transfer of flex SPI from FPGA on my custom board. MCU core (and probably DMA controller) was clocked by 297 MHz, 1.1V core voltage. Flex SPI clock was 99 MHz, SDR. The system was configured for transfer of packets 3696 bytes that slightly less maximal DMA transfer size 4K. There was still some clock stretching at the end of the packet, but very small. Clock is shown as channel 3 of attached oscilloscope screen shot. Clock stretching is seen as drops to zero. Without stretching 99 MHz signal does not reach zero level because of high capacitive load on this line. This line has a copy of the clock signal that goes from MCU to FPGA. Real data line has no such parasitic capacitive load. If we shall consider only DMA transfer time, achieved estimated sustained data rate will be 96.88 MBps. If we shall take into account interrupt request processing time (that probably can be optimized), we shall get 85.06 MBps sustained transfer rate.

Comparing with PSoC 6 MCU| from Cypress that I have tested before, it seems to be that both systems have DMA controllers with similar performance per clock. Maximal clock of PSoC 6 DMA fabricated in 40 nm process is 100 MHz, even if its Cortex M4 core can run up to 150 MHz. And in RT600 MCU, judging by performance, it seems to be that DMA engine can run with the same clock as MCU Cortex M33 core, up to 300 MHz.

Re: Maximal sustained transfer rate in FlexSPI of RT685?

jingpan — Mon, 06 Jun 2022 07:43:26 GMT

Hi @Vyssotski ,

"the main problem is not a clock stretching observed after some amount of data, but fixed duration of packet + time gap after it before possibility of reading/writing next packet."

What the time percentage of each stage, normal data transfer, stretching and time gap? If transfer time is short, does it have a long time gap?

Regards,

Jing

Re: Maximal sustained transfer rate in FlexSPI of RT685?

Vyssotski — Mon, 06 Jun 2022 22:30:59 GMT

Dear Jing, 100

Thank you for your comment.

Mentioned problem of gaps between packets was essentially down-scaled by switching from 1K packets to 4K. The gap is caused by configuration of DMA transfer done just before the transfer. This is about 5.3 us at 297 MHz MCU clock. The packet of 3696 bytes is transmitted during 38.15 us. Thus, by eliminating 5.3 us delay it is possible to accelerate the data transmission by 5.3/(5.3+38.15)*100 = 12.20% only. Such acceleration is not essential and does not worse efforts. Achieved 80-90 MBps flexSPI transfer rate is sufficient for the current application.

Currently, optimizing of SDHC transfer rate is more actual. I have tested FAT FS library writing speed to uSD card. I achieved sustained data rate about 30.20 MBps with 297 MHz core clock. FAT FS library uses ADMA2 for data transfer. In the past testing ADMA2 transfer rate in SDHC of Cypress PSoC 6 cl;ocked by 100 MHz I achieved similar transfer rate about 30 MBps. Assuming linear scaling, I would expect something about 90 MBps from RT600 SDHC controller clocked by 300 MHz. However, achieved FAT FS transfer rate is about 1/3 from the expected. Thus, the question is, in which extent it is possible to approach desired 80-90 MHz transfer rate by switching from FAT FS to pure ADMA2 transfer?

More about uSD writing speed is in this thread:
https://community.nxp.com/t5/i-MX-RT/uSD-write-speed-in-RT600/m-p/1469543#M19953

Alexei