Processing 1.0 - Processing Discourse

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

Index › Programming Questions & Help › Syntax Questions › unexpected values

‹ Previous Topic | Next Topic ›

Pages: 1

unexpected values (Read 1023 times)

clankill3r

unexpected values
May 4^th, 2009, 5:14am

hi, I have a code that's very simple but the output is so strange to me, why 0.2000002 and not just 0.2 for example?

0.4
0.3
0.20000002
0.10000002
0.20000002
0.3
0.4
0.5
0.6
0.70000005
0.8000001
0.9000001
1.0000001

Code:

float thresValue = 0.5;

void draw(){ //keeps program running
}

void keyPressed(){
  if (keyCode == LEFT){
    thresValue -= 0.1;
    println(thresValue);  
  }
  else if (keyCode == RIGHT){
    thresValue += 0.1;
    println(thresValue); 
  }
}

NoahBuddy

Re: unexpected values
Reply #1 - May 4^th, 2009, 8:17am

It's actually a side affect of how decimal values are stored.

Since the only option on the computer is binary, it stores 1/2 as 0.1,
1/4 as 0.01, 1/8 as 0.001, etc.

For a float (precision of 32 bits) the smallest increment is 1/pow(2,32)
0.00000000023283064365386962890625

What you are seeing is the nearest estimate to 0.2 that the computer can come up with.

blindfish

Re: unexpected values
Reply #2 - May 4^th, 2009, 8:28am

Interesting. I can't give you a definitive answer - though my guess is it's something to do with the precision of floats (in fact it looks like someone has confirmed this) - but I can offer a workaround: work with whole numbers and divide the output by ten:

Code:

float thresValue = 5;

void draw(){ //keeps program running
}

void keyPressed(){
  if (keyCode == LEFT){
    thresValue -= 1;
    println(thresValue/10);  
  }
  else if (keyCode == RIGHT){
    thresValue += 1;
    println(thresValue/10); 
  }
}

Where is the what if the what is in why?

clankill3r Full Member Offline Posts: 210 Netherlands	Re: unexpected values Reply #3 - May 4^th, 2009, 10:39am ok never new that so that's crappity smacked.

NoahBuddy Ex Member	Re: unexpected values Reply #4 - May 4^th, 2009, 10:57am Unfortunately, its a hazard of a digital device. Any programming language, or the hardware running it, will have the same oddity. Remember that the next time you do your taxes on a computer...

clankill3r Full Member Offline Posts: 210 Netherlands	Re: unexpected values Reply #5 - May 5^th, 2009, 6:36am Quote: Remember that the next time you do your taxes on a computer...Tongue yeah there must be a way to profit of it

PhiLho YaBB Moderator Offline Posts: 4190 Near Paris (France)	Re: unexpected values Reply #6 - May 5^th, 2009, 12:10pm Some years ago, an accountant tweaked a program to give all fractions of cent to his bank account. On a bank manipulating lot of money with funky rates, it cumulated to some millions of dollars. Of course, today the trick is well known and no longer possible.

NoahBuddy Ex Member	Re: unexpected values Reply #7 - May 5^th, 2009, 4:01pm If anyone ever gets bored, eh... I mean wants to avoid this, there is a way to have exact numbers in base 10. http://en.wikipedia.org/wiki/Binary-coded_decimal I worked with it a little. You would think something we were taught in elementary school would be easier to implement. Though it is relatively simple using hardware.

Pages: 1

‹ Previous Topic | Next Topic ›