Here is the answer for those interested:
The usefulness of auto dmx-polarity is dependant on application.
A lot of our devices get used in architectural lighting systems and are packaged in different forms, the simplest being a small cylinder 13mm dia. x 30mm long with bare leads, there can be 100's of these used in a single installation - and often get incorrectly wired - they are RDM enabled.
The UART in your processor needs to be able to invert the Rx input (and Tx output if using RDM), if not an external xor gate will be needed.
A three-stage filter is used on the dmx packets signature. Framing errors, overrun errors, slot count and start-code are added to the filter on the following dmx packet's break. Using a simple algorithm to compare the filter entries it can be determined if any changes in signature are caused by noise or data errors (noise is a random event and can happen anywhere in the packet). Packets with data errors are counted and when a preset value is reached it can be assumed that the dmx-polarity is wrong - whereby the the UART Rx and Tx inverters are enabled, sometimes you can get a good dmx-packet with incorrect dmx-polarity so this needs to be taken into account.
It can sometimes take 10 or more packets to determine if the dmx-polarity is reversed. Our devices only update their outputs when the filter determines a good dmx packet has been received, which means it is delayed but this has little issue with most lighting systems and also synchronises them.
RDM packets get processed immediately and are not added to the filter, if there are errors after a correct RDM start-code this will be picked up during RDM processing.
This may seem a pretty crude solution but it works very well, keep things simple.
Ken...