It took me a while, plus a few contradicting and confusing answers from ARM support until I managed to make this work. Therefore, I do want to share with you the easiest method in order to save you some precious time (tested with MQX 4.0):
1. Make sure to install keil's TAD plugin during the MQX installation (it can also be installed separately from MQX\tools\keil_extensions\uVision4).
2. Make the following changes in the MQX configuration file "user_config.h":
#define BSPCFG_ENABLE_IODEBUG 1
#define BSP_DEFAULT_IO_CHANNEL "iodebug:"
3. Rebuild BSP debug library.
4. Configure Debugger as detailed in your MQX release documentation, which can be found in the following folder: MQX\doc\tools\uv4\MQX-uVision4-Getting-Started.pdf
That's it, you're good to go - now you can see printf's using your uVision4 IDE console!