Hello Kunsen,
When secure boot is completed successfully, the SWD will followed the OTP fields (SWD_EN0 and SWD EN1). Below flow chart in UM is correct.

I suggest that you uses a flash programmer to program a correct secure image into external flash.
And please be careful when config OTP, there is an application note, you can have a look:
https://www.nxp.com/docs/en/nxp/application-notes/AN12352.pdf