Processing 1.0 - Processing Discourse - calculations slowing down

icantgetno

calculations slowing down
Jan 29^th, 2010, 6:26am

hi,

I have a program that simply consists of eight huge 3d massives of integers and constantly recalculates them with some random additions (a genetic algorythm). There is also a 3d visualisation of one of the massives but it is turned off by default, only with some counters output to the screen.

The problem is that after some time the counters slow down (after 5 hours of processing the speed is about 30-40% of the original speed) and the only solution I've found so far is to save the massives to a file, reboot the pc, and continue with saved files.

is there a way to deal with this without stopping the program?

NoahBuddy Ex Member	Re: calculations slowing down Reply #1 - Jan 29^th, 2010, 11:49pm It's a difficult question to answer, especially without the code Does your program create objects, new arrays, etc. throughout the run time?

icantgetno

Re: calculations slowing down
Reply #2 - Jan 30^th, 2010, 4:19am

after your reply I made all the variables from draw() initialize as global, just in case :)

yes, some of the functions create new variables, such as "i" for "for(i=0;i<5;i++)" etc. I thought that since they are local function variables, they disappear with the end of the function.

I'm attaching the draw() part, visualisation part and one of the functions with those "i"s.

icantgetno

Re: calculations slowing down
Reply #3 - Jan 30^th, 2010, 4:23am

Code:

void draw() {
     
	
	
	for(int k=0;k<NumberOfSpecimen;k++)
	     UsefulnessArray[k]=func(population3d[k], msx,msy,msz);     //filling an array of target function values
    
	strr=strongest(UsefulnessArray);					 //selecting the maximum value out of the array
 
	strongestnumber=strr[NumberOfSpecimen-1];			    
	fvalue=func(population3d[strongestnumber], msx,msy,msz);	 //target function value of the best specimen in a population

	background(#FFFFFF);
	 
	if(viewchange!=3)
		  DrawSpecimen(population3d[strongestnumber]);		//all the stuff with pushmatrix and popmatrix and visualisation
	stroke(#000000);
	fill(#000000);
	 
	 
	hour_passed=int((time-begintime)/3600);
	minute_passed=int((time-begintime-3600*hour_passed)/60);
	second_passed=time-begintime-60*minute_passed-3600*hour_passed;
	time = hour()*3600+minute()*60+second();				 
	if(time-t_end>19)
	  
	 {
	   t_start=t_end;
	   t_end=time;
	   f_start=f_end;
	   f_end=fvalue;
	  
	   
	 }
	text ((f_end-f_start)*3+"/min", 500,600);			 //target function value per minutes output
	text(fvalue, 50,600);						   //target function value output
	text(generations+" generations", 50,650);			 //generations counter output
     
	text (hour_passed+" "+minute_passed+" "+second_passed,500,650);	//time passed output
	begingen++;
	 
	  //viewchange variable is controlled by mouse clicks
	if(viewchange==1)						     
	  {
	   fov=int(dist(width/2,height/2,mouseX,mouseY))-300;
	   text("zoom",100,50);
	  }
	if (viewchange==2)
	  {
	   transparency=int(dist(width/2,height/2,mouseX,mouseY));
	   text("see through "+ColorTransparencySettings[1],100,50);
	  }
	 
     
     
     if(StopProcess==0)
     {
	for(int k=0;k<NumberOfSpecimen/2;k=k+2)			 //Main part of the algoryrhm. genetic variation. all the calculations that are heavy on memory
										    //Functions child1(),child2(),mutation(),average() include some local variables and arrays.
										    //Is it possible that these variables are not cleared from memory when the function ends?
	{
	
	
	population3d[strr[k]]=child1(population3d[strr[NumberOfSpecimen/2+k]],population3d[strr[NumberOfSpecimen/2+1+k]]);
	population3d[strr[k]]=mutation(population3d[strr[k]]);
	population3d[strr[k+1]]=child2(population3d[strr[NumberOfSpecimen/2+k]],population3d[strr[NumberOfSpecimen/2+1+k]]);
	population3d[strr[k+1]]=average(mutation(population3d[strr[k+1]])); 
     //population3d[strr[k+1]]=mutation(population3d[strr[k+1]]);
    
	}
  generations++;
  
     }



}

icantgetno

Re: calculations slowing down
Reply #4 - Jan 30^th, 2010, 4:24am

void DrawSpecimen(int[][][] specimen)
{
pushMatrix();

translate(width/2, height/2,fov);

rotateX(radians(340));
rotateY(radians(450));

rotateY(radians(anglex+width/2+190));
rotateX(radians(angley+height/2));

translate(-msx*cubesize/2,msz*cubesize/2,-msy*cubesize/2);

stroke(#000000);

noFill();
translate(-(bx1-bx2)/2,-(by1-by2)/2,-(bz1-bz2)/2);
translate(bx1,by1,bz1);

box(bx2-bx1,by2-by1,bz2-bz1);

translate((bx1-bx2)/2,(by1-by2)/2,(bz1-bz2)/2);
translate(-bx1,-by1,-bz1);
for (int i=0;i<msx;i++)
{
for (int j=0;j<msy;j++)
{
for (int rr=0;rr<msz;rr++)
{

pushMatrix();
translate(i*cubesize,-rr*cubesize,j*cubesize);
cellcolor=detectcolor(specimen[i][j][rr]);
//noStroke();

if(specimen[i][j][rr]!=0)
if((VisibleLevels[rr]==0)&&(ColorTransparencySettings[specimen[i][j][rr]]!=1))
{

fill(cellcolor,ColorTransparencySettings[specimen[i][j][rr]]);
box(cubesize);

}
popMatrix();
}
}
}

popMatrix();
}

icantgetno YaBB Newbies Offline Posts: 7	Re: calculations slowing down Reply #5 - Jan 30^th, 2010, 4:25am int[][][] mutation(int[][][] specimen) { int m,n,oo,l; for (l=0;l<2;l++) { m=int(random(msx)); n=int(random(msy)); oo=int(random(msz)); if(checkpoint3d(m,n,oo)==1) specimen[m][n][oo]=1+int(random(ColorsNumber)); } return specimen; }

icantgetno YaBB Newbies Offline Posts: 7	Re: calculations slowing down Reply #6 - Jan 30^th, 2010, 4:30am I'm sorry it looks pretty messy, I'm slowly getting to redo it OOPwise i can give you the whole code if necessary. this is how the whole thing looks like http://www.ljplus.ru/img4/d/e/deadmanshorse/printscreen.jpg

NoahBuddy

Re: calculations slowing down
Reply #7 - Jan 31^st, 2010, 2:41am

You're right about the local variables returning resources.

Perhaps the garbage collection is taking a very long time?
Nothing stands out in the code you posted.

My first thought was that you were creating new arrays, but I see that you are returning the same as the passed array (which only passes the reference).

Though I don't expect it to make a difference, it's not necessary to return the array (use void) as you are changing the original specimen[][][].

I wouldn't mind taking a look at the whole program, but I leave that up to you.

Quark

Re: calculations slowing down
Reply #8 - Jan 31^st, 2010, 3:40am

I have looked at the image you posted and I assume that the number of little coloured boxes increases with each iteration.

If that is the case then it is also probably the problem as the program runs there are more and more boxes to display and this is slowing down the program.

You might modify your program so it draws every 100 generations or so.

Also you might modify the triple loop in DrawSpecimen() to

Code:

for (int i=0;i<msx;i++)
{
  for (int j=0;j<msy;j++)
  {
    for (int rr=0;rr<msz;rr++)
    {
      if(specimen[i][j][rr]!=0)
        if((VisibleLevels[rr]==0)&&(ColorTransparencySettings[specimen[i][j][rr]]!=1))
        {
          pushMatrix();
          translate(i*cubesize,-rr*cubesize,j*cubesize);
          cellcolor=detectcolor(specimen[i][j][rr]);
          //noStroke();

          fill(cellcolor,ColorTransparencySettings[specimen[i][j][rr]]);
          box(cubesize);
          popMatrix();

        }
    }
  }
}

As there is no point pushing, translating and popping matrices if there is nothing to draw.

Hope this helps

subpixel

Re: calculations slowing down
Reply #9 - Jan 31^st, 2010, 7:03am

"massives" = "arrays"?

5 hours is a long time for a sketch to be running - it is possible that there exist memory leaks and similar problems in the main Processing codebase, which would mean even if your code didn't have any, you can't get away from the existing problems.

If the cubes are mostly empty/undrawn at the start, then it is to be expected that rendering ten or twenty thousand more cubes later in the run is going to take additional time!.. but you also say you save the "massives", reboot and reload - by that do you mean you save the state of the array, and when you reload it after reboot the speed is improved?

Try running it without rendering at all (that is, don't call box()) and see if that slows down and/or uses up your memory over time.

-spxl

Processing: subpixels.com, OpenProcessing.org | VJ portfolio: SPXL.TV

icantgetno

Re: calculations slowing down
Reply #10 - Feb 1^st, 2010, 2:51am

thanks guys,

yes, "massives" = "arrays", i'm sorry

the problem is that the generations counter slows down even if there is no visualisation at all. I only turn the visualisation on sometimes to have a look.
but yesterday it turned out that the same sketch doesn't slow down on my friend's mac, so I guess there is simply a problem with my winxp

subpixel Senior Member Offline Posts: 261	Re: calculations slowing down Reply #11 - Feb 2^nd, 2010, 1:45pm Well, perhaps not with Windows (generally), though perhaps with the Java Runtime Environment implemented for Windows. Consider upgrading your Java installation (if it isn't already the latest release). -spxl
	Processing: subpixels.com, OpenProcessing.org \| VJ portfolio: SPXL.TV

mapache

Re: calculations slowing down
Reply #12 - Feb 8^th, 2010, 9:29pm

I use AutoCAD (on winXP) at work which has memory leaks - when it slows down (reaching some obscene amount of memory usage) I minimize the window and then bring it up again and it sort of "clears" the memory and speeds up again (task manager shows it using a lot less RAM). You could try minimizing and raising your app periodically, not sure if it will work on this though....