This looks strange. Make sure that RCW bits 184, 232 and 234 are set properly, see details in section 3.1.5 of the P2041EC. Note, RCW value 0x1260000 is default on the P2041RDB, everything works properly. Here is u-boot log for sure:
U-Boot 2013.01-04009-g7bcd7f4 (Mar 19 2013 - 18:46:53)
CPU0: P2041E, Version: 1.1, (0x82180111)
Core: E500MC, Version: 2.2, (0x80230022)
Clock Configuration:
CPU0:1500 MHz, CPU1:1500 MHz, CPU2:1500 MHz, CPU3:1500 MHz,
CCB:750 MHz,
DDR:666.667 MHz (1333.333 MT/s data rate) (Asynchronous), LBC:93.750 MHz
FMAN1: 583.333 MHz
QMAN: 375 MHz
PME: 375 MHz
L1: D-cache 32 kB enabled
I-cache 32 kB enabled
Board: P2041RDB, CPLD version: 4.0 vBank: 1
Reset Configuration Word (RCW):
00000000: 12600000 00000000 241c0000 00000000
00000010: 648ea0c1 c3c02000 de800000 40000000
00000020: 00000000 00000000 00000000 d0030f07
00000030: 00000000 00000000 00000000 00000000
...