Hello Richard,
You seem to have overlooked the fact that the 'GW64 device, when using one of the FLL engaged modes, has a maximum DCO output frequency of 20MHz, and therefore a maximum bus frequency of only 10MHz. Using a standard 32.768kHz crystal will produce a bus frequency of either 8.388608MHz or 9.961472MHz, depending on the DMX32 control bit setting.
Neither of these frequencies produce acceptable results for a data rate of 115200 baud.
To provide exact baud rates, you would need an ICS reference frequency of 36kHz, to give a bus frequency of 9.216MHz. You could use a low frequency crystal of 36.0kHz, or one of the following high frequency crystals - 1.152, 2.304, 4.608 or 9.216MHz. To my knowledge, none of these frequencies are "standard".
If you do not require a crystal controlled bus frequency for any other purpose, an alternative could be to use FEI mode, with the internal reference trimmed to 36.0 kHz. This would be of sufficient accuracy and stability for SCI use.
Regards,
Mac