When you enter the interrupt the M4 pushes all its registers onto the stack, the M4 then enters "HANDLER" mode. it starts running your ISR. in handler mode all interrupts of a lower or equal priority are blocked.
Only a few commands will let you exit "HANDLER" mode. In your code you are doing a branch long. By doing a branch long the M4 thinks you are still in "HANDLER" mode and continues to block lower level interrupts.
The only commands that will let you leave "HANDLER" mode and have the interrupts enabled are listed in section 2.3.7 of the manual.
From section 2.3.7
Exception return
Exception return occurs when the processor is in Handler mode and executes one of the
following instructions to load the EXC_RETURN value into the PC:
• an LDM or POP instruction that loads the PC
• an LDR instruction with PC as the destination
• a BX instruction using any register.
To get interrupts back on and the data that was pushed onto the stack, off the stack, you must run one of the commands listed in the copied section. Basically you must push the EXC_RETURN code into the program counter.