We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › My Error or Bug
Page Index Toggle Pages: 1
My Error or Bug? (Read 597 times)
My Error or Bug?
Aug 13th, 2009, 4:20pm
 
I'm getting strange rounding errors. Is this my mistake, a conceptual problem or should i file a bug-report?
Here is the Code:

void setup() {
 size(600, 30);
}

void draw() {
 println(mouseX +
 " ? "+ 255*round(100.0*mouseX/width)/100.0  +
 " ≠ " + 2.55*round(100.0*mouseX/width));
}


When run in Processing 1.0.6 this returns:
17 ? 7.65 ≠ 7.6499996

This rounding error is not from the round command. I tested this, it returns correct integer values. The rounding error results from the 2.55 multiplication. Or does it result from the string-conversion in the println?

Thanks in advance for any hint.
Re: My Error or Bug?
Reply #1 - Aug 14th, 2009, 2:18am
 
Not an error as such, it's just an inherent problem (edit)when working with and(/edit) rounding floating point numbers.
Page Index Toggle Pages: 1