I was barking up the wrong tree! It was a USB issue, not power.
The project is a pocket-size device that will log the time each time a button is pressed. At the end of the day it will bit connected via USB to my PC to upload the results.
I had the USBKeyboard declaration at the top of the code, so the USB initialisation occurred long before it was needed.
I moved it into the function that does the transfer with a 2 second delay after it and the project works as intended.