problem with float conversion in CW3.1

amyliao — Thu, 30 Mar 2006 23:58:18 GMT

Hi,

I have a function which does some multiplication and division all in float. Then it casts the result from float to unsigned long.

The result in float is 225000000, which is correct. For some reason, when it gets casted into unsigned long, it becomes 224999984. Is there any way this could be solved?

Below is the code:

float calibration = 2000;
unsigned int time_rate = 1;
unsigned int dist_rate = 2000;
const float TIM0_INC = 1.6e-5;

unsigned long get_xover_pw (void)
{
float pw;

pw = 3600.0 / calibration / (float)time_rate * (float)dist_rate / TIM0_INC;

return (unsigned long)pw;
}

void main(void) {
unsigned long y;

y = get_xover_pw ();
}

Re: problem with float conversion in CW3.1

bigmac — Fri, 31 Mar 2006 02:58:11 GMT

Hello,

If you need the answer to be accurate to more than 6 digits, I guess you will need to use double rather than float - assuming you have enough resources available to do so.

Provided the range of values required for each parameter is not too wide, you may do better using integer maths, with careful sequencing of intermediate results to ensure adequate precision without overflow.

Regards,
Mac

S12 / MagniV Microcontrollersのトピックproblem with float conversion in CW3.1

problem with float conversion in CW3.1

Re: problem with float conversion in CW3.1