Just figured this out, might be helpful to someone else. The following assumes that you mux'd the pins correctly, so that the quadrature encoder is hooked up to the FTM's phase A and B inputs.
I am setting the counter to 30000 so that the motor can move in either direction without an overflow.
ftm_config_t flex_config;
FTM_GetDefaultConfig(&flex_config);
FTM_Init(FTM2, &flex_config);
FTM2->MOD = 60000U;
FTM2->CNTIN = 30000U;
FTM2->CNT = 1U;
FTM2->CNTIN = 0U;
ftm_phase_params_t ftm_phase_par;
ftm_phase_par.enablePhaseFilter = true;
ftm_phase_par.phaseFilterVal = 8;
ftm_phase_par.phasePolarity = kFTM_QuadPhaseNormal;
FTM_SetupQuadDecode(FTM2, &ftm_phase_par, &ftm_phase_par, kFTM_QuadPhaseEncode);
Note that if you want to set the counter value as above, you must first stop the quad decoding. E.g. as follows:
FTM2->QDCTRL &= (~1U);
FTM2->CNTIN = 30000U;
FTM2->CNT = 1U;
FTM2->CNTIN = 0U;
FTM2->QDCTRL |= 1U;
Feedback welcome, of course!