Hello X YY,
You may want to use separate power supplies to meet with the limiting values for the PCA9685. Please note table 13 and table 14 from datasheet where states the operating power supply voltage (VDD) range of 2.3V to 5.5V and supply current (IDD) is 6mA to 10mA; with a 5.5V tolerant inputs (VI/O).
I hope this information can be helpful.
Have a great day,
Paulina