how to maintain precision?

Programming Questions

jeemang

in Programming Questions • 3 years ago

Howdy;

I'm trying to implement a program that draws objects according to latitude and longitudes read from a kml file. Because the area over which I'm drawing is relatively small (ie one city), a great degree of precision in the coordinates is required. Here's a sample of the coordinates I'm dealing with:

<coordinates>
-113.442808503237,53.6048438750671,0
-113.442833924505,53.6022101984654,0
-113.442935216077,53.5996562420879,0
-113.43884754156,53.5996087723271,0
-113.43515332971,53.5995059467105,0

I'm having trouble preserving more than about 4-5 decimal points of precision (which is seems to be the approximate limit of the float type). These coordinates are initially read from the kml file as Strings, and my plan was to convert them from Strings to doubles, multiply them by a large number (say 1000000000) and then convert them to floats, a la:

double d = Double.parseDouble(h[l]) * 1000000000; // h[l] is a single coordinate in String form
float f = (float) d;

but this doesn't do the trick; if I display the results using nf() and compare them to the original String, the precision again tails off at about 4-5 decimal points (that is, the String will look like -113.421129528347 and the float will look like

-113421131776.000).

Any insight as to why my plan isn't working or suggestions of better plans would be very much appreciated.

Many thanks,

Josh

Replies(2)

kooogy

Re: how to maintain precision?

3 years ago

you might be out of luck - there are some numbers that you can't represent in floating point (0.1 for instance)

that said...

String number = "-113.421129528347";
double doub = Double.parseDouble(number);
println("doub:" + doub);

can't you just use doubles?

kooogy

Re: Re: how to maintain precision?

3 years ago

long doubl = (long)(1000000000 * Double.parseDouble(number));
println("doubl:" + doubl);

this also helps, storing the multiplied version in a long rather than a float - i think floats are normalised so .12345 and 123.45 are stored in same way, only with a different power (in base two, though, not base 10 like the example). and if it has trouble storing the small version then it'll have the same trouble storing the big version.

also have a look at java's BigDecimal which is arbitrary precision (at the cost of performance)