Hello,
As a general rule, the code within an ISR is best kept as short as possible to avoid interrupt priority problems, and delayed servicing of other interrupts. If possible, avoid placing lengthy operations such as 32 bit multiply and divide operations within the ISR. It is usually better for the ISR to communicate with the "main" code using flags and global variables, and do the lengthy processing within the latter.
Having said that, a function should be re-entrant provided only local variables are used. I assume the calculations you refer to simply do multiply and divide using long values.
You will need to define what you mean by "system crash". The usual cause of unexpected MCU resets is the COP timer. This may need to be reset at the start of your lengthy functions, to avoid the possibility of timeout occurring part way through the function.
If the results of your calculations are being corrupted, particularly if global variables are unexpectedly changing, a likely cause could be that you are exceeding the allocated stack size - you may need to increase this size. You do not mention the MCU type your are using. If a "low end" type, it might not have sufficient RAM resources for what you are trying to achieve.
Regards,
Mac