Processing 1.0 - Processing Discourse

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

Index › Programming Questions & Help › Syntax Questions › Math problems

‹ Previous Topic | Next Topic ›

Pages: 1

Math problems (Read 791 times)

petros

Math problems
Feb 28^th, 2007, 10:24am

Hi all,

I've noticed some math calculations going wrong in some of my sketches, especially when working with the "width" and "height" properties. To test, I wrote this simple bit:

Code:


size(640,480);
int mywidth = 640;
int myheight = 480;
float aspect = width/height; //SHOULD GIVE 1.333333
float myaspect = mywidth/myheight; //SHOULD ALSO GIVE 1.333333
println(aspect); // INCORRECTLY GIVES 1.0
println(myaspect); // INCORRECTLY GIVES 1.0

It seems that my results are being rounded off. Looked around for related bugs but no luck. - am I missing something?

I'm on running P0124 on XP/SP2...

Thanks!
Petros

petros

fjen

Re: Math problems
Reply #1 - Feb 28^th, 2007, 11:01am

yes, these roundoffs happen if you do the math with integer (int) values. in your calculations the math is done with integer values, therefore Processing assumes the result should be int too, hence the roundoff. to give Processing a hint that you're actually want a float result do:

size(640,480);
int mywidth = 640;
int myheight = 480;
float aspect = width/float(height);
float myaspect = mywidth/float(myheight);
println(aspect);
println(myaspect);

F

petros

Re: Math problems
Reply #2 - Feb 28^th, 2007, 11:41am

also tried declaring mywidth and myheight as float - this works as well:
-----------
size(640,480);
float mywidth = 640;
float myheight = 480;
float aspect = width/height; //SHOULD GIVE 1.333333
float myaspect = mywidth/myheight; //SHOULD ALSO GIVE 1.333333
println(aspect); // INCORRECTLY GIVES 1.0
println(myaspect); // CORRECTLY PRINTS 1.33333...

Thanks for your help!
Petros

petros

mflux

Re: Math problems
Reply #3 - Feb 28^th, 2007, 2:17pm

Yep.. just one of those things you gotta watch out with ints and rounding.

Personally I prefer the syntax:

int a = 10;
float b = (float) a / 20;

Which casts a temporarily into a float FIRST, then divides by twenty. That should yield a float back out.

note that this won't do the same thing

int a = 10;
float b = (float) (a / 20);

Because you're dividing two ints first, and then turning it into a float.

Portfolio

petros YaBB Newbies Offline Posts: 6	Re: Math problems Reply #4 - Feb 28^th, 2007, 2:54pm OK, i got it. I thought it was enough to declare "b" (in your example) as "float" and that would be it...can't be to picky with those data types thanks again! Petros
	petros

fry YaBB Moderator Offline Posts: 2623 mass, usa	Re: Math problems Reply #5 - Mar 12^th, 2007, 12:41pm also covered in the troubleshooting section: http://processing.org/reference/troubleshooting/#integers

Pages: 1

‹ Previous Topic | Next Topic ›