how to fix this simple addition 9.6-0.2

jckimo

here is the code

float x = 9.6;

x -= 0.2;

println(x); // the answer shown 9.400001

koogs

how would your write 1/3 as a decimal number?

that's the same problem computers have (but they use binary)

GoToLoop

// Forum.Processing.org/two/discussion/26195/
// how-to-fix-this-simple-addition-9-6-0-2#Item_2

// GoToLoop (2018/Jan/31)

int x = 96;

void setup() {
  frameRate(1);
}

void draw() {
  background((color) random(#000000));
  print(nf(.1 * (x -= 2), 0, 1) + " - ");
}

jeremydouglass

Approaches:

1. filter your output, e.g. by rounding or dropping precision
2. use higher precision, e.g. double, which may work to an extent in some cases (although it has the same problem)
3. use a special data type, like java.math.BigDecimal
   • https://docs.oracle.com/javase/7/docs/api/java/math/BigDecimal.html
   • https://stackoverflow.com/questions/5200081/more-precise-numeric-datatype-than-double
4. use integers, multiplied to the appropriate level of precision. This gives precise answers, but there can be headaches converting for storage or output.

This last approach is what @GoToLoop is doing. So, if you wanted to use values with two digits of decimal precision, then:

float x = 9.6;
x -= 0.2;

...would become:

int x = 960;
x -= 20; // 940, aka 9.4