Autobaud is lying to you. You're misleading it by sending it the wrong characters, so it is misleading you in return.
Autobaud works by measuring the width of the Start Bit. Which is followed by data bits 1 to 8 (or 7 or whatever).
Autobaud was first widely implemented on Modems, and for it to work, the modem commands always start with the characters "AT". There's a reason for that. "A" is 0x41 or 0b01000001, or as a string of bits on the serial port, "S10000010s" where the first "S" is the Start bit with the value of "0" and the "s" is the Stop bit. So the serial port has the bit stream "....1111111010000010111111..." That first zero is the start bit and the width of that is what the Autobaud measures. The "T" is 0x54 or 0b01010100, and in combination with the "A" the modem can work out the number of bits and the parity as well. It also works with "at".
Autobaud also works on Carriage Return as that's 0x0D or 0b00001010, but it can only tell the baud rate.
If you're sending 0x4A (a "J"???), then that's 0b01001010 which will be "...1111110010100101111..." and I'd expect "Autobaud" to measure that as 4800 baud as it would see the first two zeros as a single bit.
So forget Autobaud. Look at the actual waveform and decode it yourself.
Are you sure your system clock is 80MHz? Have you measured that separately? Can you enable CLKOUT and measure its frequency?
Tom