Thanks for the reply, Jiri.
I think the example you gave in your answer will work only in the case of debugging 1 image per core, where we can use the launch group to get each core to run a separate debug session with source level debugging.
My issue was more similar to the previous similar question, where multiple images for the same core. I guess like the answer in the post pointed out, you can only either adding the bootloader elf into the application project and only debug application or the other way around, but not both at the same time.
The original goal of this question was to see if there is any way to debug both images in a single debug session with all source/symbols loaded.