Performance and Memory Issues

Answers

• gohai

  June 2017

  A couple of remarks:

  • Don't use the GL Full KMS option, leave it at Legacy.
  • The "GPU memory" you set is wasted when you use anything but the Legacy graphics driver, as "Full KMS" uses its own separate, pool of memory.

  What resolution is your screen? How do you use your webcam?
• Bhargav

  June 2017

  @gohai

  My screen resolution is 1920x1080 and I am using a normal USB webcam which is Logitech C310 720p webcam. Webcam is used to show live video and click pictures.

  I'll set back the memory to "Legacy : orignal non-GL desktop driver" thanks for pointing out. So this makes the raspberrypi use only 64MB of GPU memory as shown in raspi-config. In Processing preferences I have keep maximum memory of 512MB.
• gohai

  June 2017

  Which renderer are you using in Processing? The default one or P2D/P3D?

  How does the performance/instability improve when you remove the webcam part of the sketch? (Are you using the Video library or the GL Video library btw?)
• Bhargav

  June 2017 edited June 2017

  @gohai

  I have not specified any renderer.

  I will test the program without video and share results in sometime.

  for using webcam I am using processing.video library.

  For showing live camera feed I use : cam.read(); //to read the camera image(cam, x, y);

  For Images I have declared them with Pimage and displaying with image(nimage, x, y) Images are dynamically loaded from memory.

  Code is little long but in short what happens in is : It loads 30images set into memory and showing 4 images at a time from 30. With (first) user's input (via GPIO event) the 4 images can be replaced by any other image from 30 images in memory. Also (second) user's input can changes the image set so it loads another new 30 images will come into memory (time taken for this is okay) replacing previous set. And now another set of images are in displayed and user one can change the 4 images from new loaded set of 30. Another (third) type of user can change the screen turn into a live display or click images. While it is live display it just shows display of camera feed and while it is clicking pictures screen goes black. and only photos are clicked in background.

  and the images are displayed with following way :

  //Display Images
  pushMatrix();
  translate(width/2, height/2);
  rotate(180*TWO_PI/360); 
  scale(1,-1);
  image(imgA,- imgA.width/2, height/2 - imgA.height);
  typeInGuj(texts, -cwidth/2, height/2 - cheight);
  popMatrix();
  
  
  //top
  pushMatrix();
  translate(width/2, height/2);
  rotate(90*TWO_PI/360);
  scale(1,-1);
  rotate(PI);
  image(imgB, -imgB.width/2, height/2-imgB.height);   
  typeInGuj(texts, -cwidth/2, height/2-cheight);
  popMatrix(); 
  
  //Left   
  pushMatrix();
  translate(width/2, height/2);
  rotate(-90*TWO_PI/360);
  scale(1,-1);
  rotate(PI);
  image(imgC, -imgC.width/2, height/2-imgC.height);  
  typeInGuj(texts,-cwidth/2, height/2-cheight);
  popMatrix();
  
  //bottom
  pushMatrix();
  translate(width/2, height/2);
  rotate(-180*TWO_PI/360);
  scale(1,-1);
  rotate(PI);
  image(imgD, -imgD.width/2, height/2-imgD.height);
  typeInGuj(texts, -cwidth/2, height/2-cheight);
  popMatrix(); 
  

  }
• Bhargav

  June 2017 edited June 2017

  @gohai

  i removed all webcam code related to webcam and just started running the program and parallel watching the task manager's bar of performance and CPU usage. Basically now the code has only image loading and displaying activity.

  The CPU usagae seems fine, but found something is wrong with memory, this program somehow for each image being loaded on screen the memory is constantly increasing even if i am overrighting on same PImage variable! Now i realized why the Raspberrypi was getting hang.

  There is one Pimage array which is fetching 30images from sdcard to RAM while program is running. Now actually there are 4 image variables which are pointing to four images from the 30images.

  with users's input to change the 4 images to point on other images from set of 30, each of this time the memory is increasing almost 2MB. and

  with user's input when new set of 30 images are loaded memory increases to almost 30mb.

  I dont know how it is happening because I am loading images on same variables. It should over right the memory right?
• koogs

  June 2017

  The was a long standing bug where memory wasn't cleared when reusing pimages. I think the solution was to call dispose on the image (Google this). I also think this may've been fixed, should hope so.

  If it's a small set of images load them all in setup.

  I've also seen someone using jcache to store a pool of images but don't know whether that's suitable for a pi.
• gohai

  June 2017

  Another thing to keep in mind: Java is a garbage-collected language, and even on an OS-level pages won't necessarily be reclaimed before there is memory pressure. So looking at the output of top, and deducing from some number growing over time that there is a memory leak is not necessarily a valid observation.

  There are specific JVM-level tools available that let you specifically monitor the lifetime of objects and the like, which would be much more apt for what you're trying to do.

  Regarding performance: 1080p with the default renderer means pushing and copying quite a lot of pixels (e.g. from the USB camera to the PImage, from the PImage to the display window, from the display window to the screen). I am not too surprised that this is somewhat taxing the system to some degree.
• Bhargav

  June 2017 edited June 2017

  @Koogs set of images are dynamic. And can't be loaded together. I'll checked for dipose on the pimage but looks like it is for GLVideo. I'll try shifting my video library and see if it works with video part of code. But my problem of memory is while loading images, the memory is constantly increasing even if I am overwriting on same variable for images.

  @gohai I am a beginner level coder so using JVM evel tools and handling objects might be tough for me. Can you share more details and any supportive links about tutorials or anything?
• Bhargav

  June 2017 edited June 2017

  @gohai @koogs

  I was reading few blog posts on forums and I found the following set of instructions in my code which somehow reduced the issue.

  I had set of instructions which were rewriting the images into same variable which eventually was expanding the use of memory. So before I rewrite the images i.e when I update PImage variables with new images I run following commands :

  g.removeCache(imgA);
  g.removeCache(imgB);
  g.removeCache(imgC);
  System.gc();
  

  and the memory issue seem to resolve a little bit. Earlier the memory was expanding for each image being loaded i.e. any activity on screen. So the total amount of time I can run my code is from the time I start processing app on raspeberrypi to until when the memory is full! it will start taking memory in RAM increasing almost 1 to 2 Mb in every couple of seconds! And then in like 5-15 minutes raspberryPi will overload and hang, and will require manual restart.

  Now it is only expanding 1-2 MBs while on each image set being updated. Now somehow My pi will run for almost few hours then minutes. But still it will overload the memory and hang. (Tested for 2 hours and worked well. Yet to be tested for long duration. I guess some code optimization will increase the time. I'll test and share the results.)

  But I found on few blogs that calling System.gc() is a bad practice. I dont know why? Because it is working here. Would like to know if you have to say anything on this. @gohai @koogs
• GoToLoop

  June 2017 edited June 2017 Answer ✓

  https://forum.Processing.org/two/discussions/tagged?Tag=removecache(),system.gc()
• Bhargav

  June 2017 edited June 2017

  @GoToLoop

  Using g.removeCache(img) and System.gc(); is working for me here. I tested multiple times. And as shared by you in this thread. even if it is just a suggestion to java for cleaning but it is helping it. I want to know like as in the same thread and many other posts all are not suggesting to realy on system.gc() even if it works. And as of now i am using it at the end of the draw() function

  As you have backed the system.gc() I would like to know whether you always use System.gc() ? or have you used any alternative methods which might be surely reliable and confirmed by fellow programmers?
• GoToLoop

  June 2017 edited June 2017 Answer ✓

  • About how reliable gc() is I'd say it's pretty good.
  • However we should always have in mind that according to Java's specifications, the JVM's GC can simply ignore a gc() call when it decides that's not the right time to do it!
  • From the few experiments I did w/ gc(), it is very good to delete all of the dangling keys from a WeakHashMap container: http://docs.Oracle.com/javase/8/docs/api/java/util/WeakHashMap.html
  • But then, calling its method clear() would achieve the same 100% guaranteed; although it would take away the non-dangling 1s along as well: http://docs.Oracle.com/javase/8/docs/api/java/util/WeakHashMap.html#clear--
  • A more realistic use for gc() is when we delete a big batch of objects in order to create another new big batch next.
  • On that case, gc() serves as way to guarantee we got memory available right way before the big operation.
  • A 3rd 1 is what you're trying to achieve. That is, aggressively attempting to release memory b/c you're creating new objects faster than JVM's GC can get rid of them.
  • Although it's a losing battle, for eventually memory is gonna run out. Although much later... 8-|
• Bhargav

  June 2017

  @GoToLoop thanks for the description! I'll try to use clear also. And as you mentioned in the last point, i too realized that as the program runs further after a point memory is eventually gonna run out for sure, somehow able to extend some more time with gc() and removeCache().

  This makes me conclude, that I wont be using raspberrypi and processing for a system with such requirement of memory to be cleaned, actually it is not requiring too much memory but due to memory clearance issues it is affecting the whole system setup. Just because of this I am changing my platform to OFx, in future I wont use processing on pi for a setup which has to run on its on. I might only use for testing small things in my presence.

  @Gohai @Koogs Thanks for your support. I found processing easy to code, as I am not a proper programmer so I didn't try managing memory through JVM level tools or core java or linux on pi. In beginning I didn't realize that such issue may come over time. Further versions of processing should can automatically organize this and make processing more reliable to be used on Pi. Thanks :)