I am running into this same issue (using a LPC1315). I'm new to this so please let me know if I'm missing something obvious.
As Nick says, the default GPIO pins are in effect driving a 1 after reset due to the weak pull-up, and that is a problem if the default is supposed to be an Input. So I have 2 questions:
1)Why is defaulting to Input but with a pull-up a good CPU design? I see why defaulting to Input is good. Not being snarky, I'd really like to know.
2)The pins that my firmware eventually configures as outputs are in fact appearing as hi to my UDN2987 during reset. What is the standard way to get around this? Is a pull-down the right way? Is it the only way?
An external pull-down resistor of 10k (or stronger) is indeed the appropriate way to override the internal pull-up in those cases where you must force a logic low level.
We should leave the "why" question to the philosophers.