For (D2D) HD measurement using PTA6 ADC1_S18, you should also include the voltage divider factor into previously mentioned equation.
Such voltage divider factor is configured with CFG[HDHSDIV] register (see reference manual):

The voltage divider factor values are stated datasheet:

So equation 2 should be updated to:
ADCresult = ((2^N) (Vin / divider_factor)) / VREF
I hope this information is helpful.