I will assume that you select FBE mode when running the crystal (not FEE mode), and FEI when using the internal clock. The operation of the FLL will give jitter dependent on the relationship between the clock frequency (internal or external) and the bus frequency. The datasheet specifies a maximum 0.2 percent average value over a 2 ms interval. The average jitter for shorter intervals would be correspondingly higher.
For your test code, the averaging period is only a few microseconds, so may well account for the jitter you are observing.
For applications where jitter is critical, the FLL cannot be used. Another approach is to use a higher frequency external crystal (perhaps 4 MHz, or thereabouts) in FEE mode, to generate the bus clock (complete with jitter). For the timing critical events, TPM interrupts are used, with the external clock selected as the TPM clock source. The interrupt events would not be subject to the FLL jitter.
For SCI operation, the bit period should normallly be sufficiently long that jitter will not represent a problem. If my calculation is correct, I would estimate a worst case of about 7 percent jitter at 19200 baud.