Problems start with a cache parity error. Most likely, this has to do with overclocking, noisy, unstable or inappropriate voltage power, external conditions like high level of radiation or overheating. The only way the software can directly create this condition is described in E5500RM, Section 5.4.5. I do not think Linux has any code for cache error injection, suggestions below are actually sanity checks:
Check if it violate the errata.
Make sure no customizations have been done to u-Boot and/or Linux to switch on/off cache error protection without cache invalidation.
Compare the problematic kernel configuration against the SDK default to make sure the build flags are correct for the CPU and no unsupported code is included in the build.
Use SDK provided build tools to ensure the kernel is not miscompiled.
Check general system operation conditions, make sure there is no overheating, overclocking, ESD, radiation.
Check the power rails for stability and noise. The fact that the problem aggravates with more cores enabled together with the observation that the error shoots when a core leaves the normal idle routine, point to power as the most likely root cause.