Hello @gzleng,
try to modify the Start Delay parameter. Set it to 1/4 of the excitation signal period, or 1/2 of the excitation signal period. This parameter sets the initial phase shift of the excitation signals. A proper settings enables to speed up the RDC start-up sequence by quick settle of excitation signal phase-shift adjustment. Wrong settings might cause what you see - that it sometimes settles to 0deg and sometimes to 180deg.

1. how is the SWG as the excitation is synchronized with the SDADC ? i cannot seem to find the triggering mechanism.
SDADC runs continuously. There is no way to trigger/adjust the sampling points. SDADC samples arrives to eTPU as buffers of 16+16 samples. We need that one period of a modulated signal fits perfectly into that buffer of 16+16 samples.
It is achieved the way, that the excitation signal is phase-shifted (delayed). The phase-shift is controlled by a slow PI controller (actually just I-controller). The error input to the controller is the zero-crossing error of sin/cos signals in the input buffers. When settled, the excitation signal delay is tuned, sin/cos input signals are aligned in the buffers, their samples number 0 and 16 are all zeros.
2. how is the etpu synchronized with the SDADC?
16 samples ready in SDADC output buffer -> DMA triggered -> DMA moves those samples to eTPU DATA RAM & links another DMA channel -> that DMA channel transfers a constant value from RAM to eTPU channel HSR -> a write to HSR evokes eTPU event -> eTPU processes the SDADC samples and updates resolver angle and speed.
This happens every 50usec @ 10kHz excitation. There are also faster configurations.
(simplified - DSPSS omitted, talking about just one SDADC signal instead of both sin & cos)
This way the processing of resolver angle is completely independent of the CPU. Not a single interrupt to the CPU.
3. i cannot find where in the eb tresos to set the SDADC to differential mode.. is it hidden or by default?
EB tresos: Adc->AdcHwUnit->AdcChannel

This example uses differential signal between AN0 and AN1 inputs as the input to ADC.