So I am developing SPI communication (using spidev device) with NXP i.MX8M and I have encountered that the SPI stops in long transmissions (> 4096 bytes). I have also discovered that this is a uP limitation and a lot of people is having the same problem.
What I'm noticing that works differently from the questions people are asking is the following: typically, people have this problem and the gap they experience is around 5 us. The problem is that in my case, sometimes the gap is 5 us, but in other cases, it can extend to 500 us. All of this seems to happen randomly.

In the image, it can be observed that there are 8 short pauses, until the 9th one occupies a long period of time, and so on.
Any ideas why this might be happening?