Hey @bobpaddock & @jingpan
I relocated the fly wires from one set of UART pins to another and things have been running fine and the problem seems to have gone away. This involved quite a bit of moving other functionality around but it's a prototype board so nothing is really set in stone.
The fly wire length went from 170cm to 100cm and they no longer run over the space taken by a boost supply on the other side of the board (which I guess was the problem, but I thought I was okay with a large ground plane in between).
I tried terminating the lines with 100k resistors to GND (also a fly wire), but that didn't fix the problem - changing the pins used for the function so that the wires could be picked up on another part of the board seemed to do the trick.
Lesson learned: keep unterminated wires away from a switch mode power supply on the other side of the board.