Parallel QuadSPI

anthony_huereca · ‎06-27-2013

We've been helping one of our alpha customers with using QuadSPI in parallel mode on Vybrid, so I wanted to create a thread about some helpful tips when trying to do the same. There will be a parallel mode booting example in the next release of the sample code.

Vybrid has two QuadSPI modules: QuadSPI0 and QuadSPI1. QuadSPI0 then has two QuadSPI ports associated with it: QuadSPI0_A and QuadSPI0_B. This lets you use the QuadSPI flashes connected to those ports in parallel. QuadSPI1 only has one port, and thus parallel mode cannot be used with it. So in summary, there are two modules on Vybrid, capable of interfacing with up to 3 flashes in all, but only QuadSPI0 can do parallel mode.

Parallel mode is enabled by setting QuadSPI_BFGENCR[PAR_EN]=1. When set, the data at offset 0x200 of each QuadSPI would be read at offset 0x400 when combined. This is very important to keep in mind when programming data into each flash.

Each flash in parallel mode holds a nibble (4 bits) of data from each byte. The way that data is split is a somewhat complicated, and so it's very important to properly split the data that ends up being programmed in each of the two flashes. The sample code contains a function split() which does this a word (2 bytes) at a time. The code is attached. It takes a source memory address, and outputs each half into the two destination memory addresses.

After programming the flashes, put the QuadSPI into parallel mode, and make sure the data you read back from the memory window matches what you think you should have. This will double check your programming and splitting algorithms.

Parallel Booting:

The QuadSPI boot header must be programmed into only *one* flash connected to QuadSPI0_A. The BootROM reads this configuration data, and only then puts the QuadSPI in parallel mode. So all other boot data and code after that must be programmed as if being read in parallel mode. What this ends up meaning is that you need three programming passes:

Program the QuadSPI boot header into QuadSPI0_A flash at offset 0x0
Program the split data starting at offset 0x200 (per the sample code) into QuadSPI0_A
Program the split data starting at offset 0x200 (per the sample code) into QuadSPI0_B.

The sample code will show this being done.

There are only a few minor changes in the QuadSPI boot header necessary.

Set the "Parallel Port Enable" field
Make sure the "Flash B1 Size" field is properly sized
Make sure the "Port B Enable" field is set

There should be no other code changes necessary if you already have single QuadSPI flash boot working.

Debugging Tips:

Make sure the data read back in parallel mode when you program the flashes match the binary you're trying to boot (except for the boot header section, which will look incorrect when read in parallel mode since it was only programmed into the single QuadSPI0_A flash).
Make sure the code works in single flash mode still
Make sure your IO Mux config isn't turning off the QuadSPI0_B pins.
As with any boot debugging, try turning on an LED as one of the very first instructions. If you reach there, showing you did boot successfully, then keep moving it back through your code until you find the spot it crashes on.

Original Attachment has been moved to: split.c.zip

kef2 · ‎08-26-2014

Hi Nancy,

For file attachments I click Use advanced editor link on the top right corner of reply box, and in advanced editor you should find attach link on bottom right corner.

Those 3 (or more) modified files should be enough to rebuild your hello2 app.

Regarding IAR and CMSIS on Vybrid card, perhaps you didn't reflash OpenSDA firmware. Don't know if the best thread, search for more if it doesn't help:

TWR-VF65GS10 / give me OpenSDA MSD and CMSIS-DAP Firmware

Edward

nancyb · ‎08-26-2014

Hi Edward,

I totally missed the attach link at lower right. I have attached the relevant files.

I will check out the thread link you provided.

Thank you,

Nancy

kef2 · ‎08-27-2014

Hi Nancy,

*.icf linker file format seems being not supported by DS-5. Instead of trying your files, I tried changing hello2 load address to QSPI and booting it using my own QSPI bootloader. Yes, MQX doesn't work and hangs in init_hw.c file clock_init() routine. With my top QSPI boot speed setting (99MHz) code dies at following line:

// select fxosc as source of FAST_CLK_SEL

// select FAST_CLK_SEL as source of SYS_CLK_SEL

CCM_CCSR = 0x00000020;

At 99MHz it makes QSPI not readable. It is not very clear from Vybrid RM which QSPI boot speed setting doesn't use PFD clocks, which this line disables, you may try different speed settings, may be some other speed will work. But it would be better to fix it more reliably. You need to either move MQX clock init routine to RAM or don't use XIP boot.

Edward

nancyb · ‎08-27-2014

Hi Edward,

Is your issue different from mine, or is the failure to correctly execute an instruction another manifestation of QSPI read issues? I will try a different clock configuration to see if I can fine one that yields valid XIP capability.

If Vybrid has trouble with parallel mode QuadSPI XIP then that issue will adversely ripple back through our system design and validation.

Your assistance has been crucial to helping me implement QuadSPI parallel XIP.

Nancy

kef2 · ‎08-27-2014

Hi Nancy,

Problem is not in Vybrid's QSPI implementation, problem is MQX's clock setup routine, which you shouldn't execute from QSPI. I faced the same problem some time ago with my bare metal no OS design. I faced both, problems with QSPI reinitalization while executing in place from QSPI, and also (QSPI) clocks reinitialization while executing from QSPI. No wonder here. CPU shouldn't try to break memory, from which it is reading instructions... Reinitialization is not safe, and this was mentioned few days ago. Clocks initialization is also not safe, unless you are setting up registers, which don't affect QSPI clocks. No single clock glitch is allowed. It is the best to move QSPI setup and clocks initialization routines to RAM and execute from there. You could also try preloading clocks setup and QSPI ini routines to instruction cache, but this is much more difficult.

Edward

nancyb · ‎08-27-2014

Hi Edward,

Since my code never gets past the first instruction in PSP boot.s I wonder if the problem I am experiencing is different from yours.

I am also curious about how I can successfully execute my MQX parallel mode XIP image if I attach to Vybrid already running an image in SRAM, perform a reset, and observe the MQX image successfully execute. In that case the MQX clock setup routine did not scrozzle QSPI execution.

Nancy

kef2 · ‎08-27-2014

Nancy,

Answer regarding MQX boot: yes, I was able to run MQX from parallel-QSPI, XIP. See attached modified init_hw.c . I was able to make DS-5 compiling it and linking with instead of symbols from BSP. It works with top speed 99MHz.

Regarding problems on first instruction in PSP. This Cortex-A5 CPU on Vybrid has very smart cache (as for my taste), so I have some doubts regarding what you observe and how do you interpret it. Once it runs away, a lot of weird things may happen. CPU may jump to exception handler and recover from it, perhaps jump to boot address after few runaway iterations, but with not power-on state of peripheral registers and CP15 settings... HW reset button, then debug from button in DS-5 are first things I always do after seeing some X-files. This helps a lot.

Forever loop is a nice approach, and perhaps you are right in your conclusions. But MRC instruction works and whole MQX hello2 with modified init_hw.c is working for me. If it still doesn't work, next step would be checking boot config and then hardware problems.

Edward

nancyb · ‎08-27-2014

Hi Edward,

init_hw.c change did not help. When I attach to the target running in QuadSPI, pause execution, and step outside the forever loop the debugger dies. I have attached a screen shot of the error. At this point the only recourse is to abort the debug session. I am also continuing research into using CMSIS-DAP.

Nancy

I

kef2 · ‎08-28-2014

Hi Nancy,

OK. Then we need to go further and check boot config. Since nothing wrong was noticed before, I tried to play with differences of your setup and mine. I'm not familiar with split technique, so I applied your quad_conf settings to my code. And ooops, finally and I got it not working :-).

Your quad_conf claims you have four memory chips, so both CSx chipselects should be used. I tried enabling CS1 for channel A and it worked! I can't explain this without looking at boot ROM source codes. Then I disabled CS1 back and set A2 and B2 memory sizes to 0. It worked again. Works at 99MHz and 60MHz.

Regarding init_hw.c Please check if your code works at both 60 and 99MHz boot speeds. If not, make sure linker uses your supplied file and doesn't link with one from original BSP library.

(I'm fed up with this a bit :-) )

Regards

Edward

nancyb · ‎08-28-2014

Hi Edward,

Based on your results I performed a series of experiments; changing QuadSPI configuration parameters, setting Vybrid boot configuration to QuadSPI, and power cycling the target. For reference, this is the technique I use to update and verify execution after updating QSPI flash. The bare metal project "hello_world" always boots without issue. My MQX project "hello2" fails boot. Both projects use identical QuadSPI configuration parameters and Image Vector Tables.

First I set A2/B2 size zero; no joy. Then I set Enable CS1 on Port A; no joy. Finally I set Enable CS1 on Port B; no joy. Then I rebuilt and flashed "hello_world" using these values and it executed in parallel mode without exception.

I flashed "hello_world" using all of the above settings with serial clock frequency value (offset 44) set to 1, 2 and 3; all worked.

Setting the QuadSPI configuration parameters has always been problematic for me because when the boot fails I have no feedback to tell me exactly what went wrong. I have had good results copying setup values that other folks have created, such as the LUT to enable quad mode.

Again, thank you for your assistance. I can relate to being "fed up with this a bit". I've been at this tooth & tong since early July. The good news is that I have become quite familiar with rapidly executing my QuadSPI experiments; the less than good news is I still do not have a bootable MQX image.

Nancy

Here are my current QuadSPI configuration parameters settings that work with bare metal image and fail with MQX image:

kef2 · ‎08-28-2014

Nancy,

please disregard what I said about not working setup with A2 and B2 sizes set to 0x01000000 and CS1A and CS1B disabled. Perhaps I failed to flash that image properly. I tried again and it works well despite of A2/B2 flash size and CS1 enable settings. MQX still is running well in parallel QSPI mode with modified init_hw.c and with your quadspi_conf settings, also tested with 3 speed settings SCLK=1,2,3. No problems.

Could you please confirm again that MQX XIP works for you in non-parallel mode? If it worked, then I'm clueless.

Edward

nancyb · ‎08-29-2014

Hi Edward,

My MQX image hello2 works fine n single (non-parallel) mode. The only difference between the parallel and single mode image is clearing parallel enable (8 bit value offset 0x38) and quadspi_load flashing QuadSPI in single mode.

I use the same QuadSPI configuration parameters and Image Vector Table for the MQX image hello2 and the bare metal project hello_world.

It is a mystery to me why the only configuration that fails to boot XIP is the MQX image in parallel mode; MQX image in single mode and bare metal project in either single or parallel mode work without exception.

I am also without clue to explain this.

Nancy

nancyb · ‎09-02-2014

Hi Edward,

Things took a turn on Thursday. I made a small change to quadspi_load to read the single/parallel mode from the QuadSPI configuration parameters loaded with the image. Prior to this the selection of single/parallel was selected with a #define. Therefore changing between single and parallel mode required two changes and now there is only one. I tested quadspi_load for both single and parallel mode and both booted MQX hello2 without exception. While running hello2 from QuadSPI I attached to the target, paused execution, reset the debugger and observed the image boot exactly as expected with QuadSPI0_BFGENCR.PAR_EN set to 1.

I have no explanation for why parallel mode works. I am not please to not know why my image works now. I am tentatively declaring victory as I explore this further and possibly understand why it now works.

I observed that parallel mode QuadSPI would not boot unless I applied your updates to init_hw.c.

Finally, some good news. Thank you for your help.

Nancy

kef2 · ‎09-02-2014

Hi Nancy,

Sounds like you didn't try rebuild all and IAR failed on dependency check. Perhaps not. I'm glad my init_hw.c mods were useful.

Regards

Edward

nancyb · ‎09-03-2014

Hi Edward,

Your modified init_hw.c was the last puzzle piece getting QuadSPI parallel mode operational.

Do you have insight into whether your modification will be part of some future MQX release, or is it an interim workaround to fix this specific issue?

I will also raise this question with our factory rep Hari Arimilli.

Regards,

Nancy

kef2 · ‎09-04-2014

Hi Nancy,

Nothing will be done without special efforts. Regarding MQX support for QSPI XIP, the right way to report bugs and/or ask for improvements is submitting service request here:

https://www.freescale.com/webapp/servicerequest.create_SR.framework

My mood at the moment is not compatible with questions, which service people are likely to ask me :-). Try explaining them your issue, shouldn't be hard having good English skills.

Regards

nancyb · ‎09-05-2014

Hi Edward,

I will follow up with this per your suggestion.

English skills are only half the battle explaining the QuadSPI situation. Paring down the technical details so that a non-QuadSPI person can understand the issue is the hill to die on.

Again, thank you for your help. Your QuadSPI configuration parameters and modified init_hw.c were key parts to success. Anthony's split.c and updates to quadspi_load.c round out the effort.

Regards,

Nancy

nancyb · ‎08-27-2014

Hi Edward,

I will try out your init_hw.c.

I agree that my interpretation of what I am seeing with the mrc instruction may be flawed. The debug tools I am using do not handle Vybrid problems and seem to work only for "good day" scenarios. From the CMSIS-DAP thread you pointed to yesterday it appears that Rev G Vybrid boards have the latest support code. I cannot connect with IAR so there may still be problems at that level. I could sure use some debug tools that knew how to interact with Vybrid when things go off the rails.

Nancy

nancyb · ‎07-25-2014

Is there a viable example project implementing parallel QuadSPI? I checked VybridSampleCode download and the last update was last year. I am currently using the quadspi_load project to flash QuadSPI on a Vybrid Tower board.

I am very interested in demonstrating parallel QuadSPI for our product and would like to leverage from a working example.

Regards,

Nancy Burkholder

anthony_huereca · ‎07-25-2014

Hi Nancy,

See the attached quadspi_load.c file which can overwrite the current one in the SC. The split() function is in the file attached to the original post. That should work then for you.

To use parallel mode, change #define PARALLEL_MODE to 1 in quadspi_load.c

You will also need to modify \src\boot\quadspi\quadspi_boot.c to enable the parallel boot option:

1, //Parallel Port Enable (0=disabled, 1=enabled)

nancyb · ‎07-29-2014

Hi Anthony,

I updated my quadspi_load project to use your quadspi_load.c and split.c. I added the following #defines to quadspi_load.c

#define PARALLEL_MODE 1

#define FLASH1_BASE_ADR 0x20000000

#define FLASH2_BASE_ADR 0x21000000

In source file quadspi.c I modified quadspi_erase( void ) to quadspi_erase( unsigned long address ) in order to work with your code in main(). I did not modify any other quadspi code in Vybrid Sample Code folder.

I modified quadspi_conf in the hello_world example code:

...

1, /* Parallel Mode Disable */

...

The image would not boot when I configured the jumpers to boot from QSPI. I could debug the image executing in QSPI through IAR EW IDE and j-link. I stepped through the image and verified it was executing from QSPI.

I am continuing to investigate.

Regards,

Nancy

Here is the hello_world quadspi_conf:

const SFLASH_CONFIGURATION_PARAM quadspi_conf = {

0, /* DQS LoopBack */

0, /* Reserved 1*/

0, /* Reserved 2*/

0, /* Reserved 3*/

0, /* Reserved 4*/

0, /* cs_hold_time */

0, /* cs_setup_time */

0x400000, /* A1 flash size */

0, /* A2 flash size */

0, /* B1 flash size */

0, /* B2 flash size */

0, /* SCLK Freq */

0, /* Reserved 5*/

1, /* Single Mode Flash */

0, /* Port - Only A1 */

0, /* DDR Mode Disable */

0, /* DQS Disable */

1, /* Parallel Mode Disable */

0, /* Port A CS1 */

0, /* Port B CS1 */

0, /* FS Phase */

0, /* FS Delay */

0, /* DDR Sampling */

/* LUT Programming */

0x0403,

0x0818,

0x1c08,

0x2400,

};