OK, I found my problem. The switch between the single-ended clock and the differential clock is controlled by the "IFC_WE0_B" pin. Since this pin is part of the IFC bus which I am not planning on using, I just tied it high in my schematic. That is a mistake! As that would force me into single ended clock mode only, unless it can be overwritten after the RCW is loaded.
I strongly recommend App Note AN5252 as it has much more information about the clock modes than any of the datasheets. Refer to section 5.30.