Hello Zhang Yamming,
Cold you please clarify more information about your algorithm test?
Also, please describe what type of memory configuration you are running. By "out of chip memory", are you using Hyperflash, QSPI, etc.? Is this where you are loading the code onto?
When you refer to the "kernel", what exactly do you mean?
What methods are you taking to disable D-Cache? If you haven't already, please try to referring to the Application note found at the following link: https://www.nxp.com/docs/en/application-note/AN12042.pdf
Thank you for choosing NXP.
Let us know if you have any more questions.
Manuel, Jacob