Performance and Memory Issues

Bhargav

I am using latest version of processing on raspberry pi 3 latest updated OS. TO give you overview of my project includes using dynamically loading of some images for display and which are dynamically fetched while the program is running. And it uses GPIO pins and captures pictures from a webcam. My program starts with auto run. Code and functionality wise everything is working well to my project but I think its not using the full resources available on Pi and working slow. If it can use more resources of Pi it might run faster but i dont know how to work on it

In the scenario the startup time is takes about 45seconds to 60seconds(a minute) until it completely load the program and starts running. As the program is still loading i cant even show loading symbol on the screen as the pi is loading processing and he program it self.

Also the task manager shows that the memory is used between 400MB to 600MB out of 923MB(total) while the processing code alone is running. Also at the same time the CPU usage is also less the 55% shown in task manager's live display.

And If i leave my program keep running and dont do anything after sometime the raspberry Pi system hangs and I have to forcefully swtich off the power.

Need to how can i make processing on Raspberry Pi make use of more RAM memory and CPU available to it.

I also went into raspi-config and from advance setting to GL Drivers ann enabled the GL Full KMS option. Then I added into 256MB for GPU memory. But it doesnt help much. Can't see any effective results while the code is running.

gohai

A couple of remarks:

• Don't use the GL Full KMS option, leave it at Legacy.
• The "GPU memory" you set is wasted when you use anything but the Legacy graphics driver, as "Full KMS" uses its own separate, pool of memory.

What resolution is your screen? How do you use your webcam?

Bhargav

@gohai

My screen resolution is 1920x1080 and I am using a normal USB webcam which is Logitech C310 720p webcam. Webcam is used to show live video and click pictures.

I'll set back the memory to "Legacy : orignal non-GL desktop driver" thanks for pointing out. So this makes the raspberrypi use only 64MB of GPU memory as shown in raspi-config. In Processing preferences I have keep maximum memory of 512MB.

gohai

Which renderer are you using in Processing? The default one or P2D/P3D?

How does the performance/instability improve when you remove the webcam part of the sketch? (Are you using the Video library or the GL Video library btw?)

Bhargav

@gohai

I have not specified any renderer.

I will test the program without video and share results in sometime.

for using webcam I am using processing.video library.

For showing live camera feed I use : cam.read(); //to read the camera image(cam, x, y);

For Images I have declared them with Pimage and displaying with image(nimage, x, y) Images are dynamically loaded from memory.

Code is little long but in short what happens in is : It loads 30images set into memory and showing 4 images at a time from 30. With (first) user's input (via GPIO event) the 4 images can be replaced by any other image from 30 images in memory. Also (second) user's input can changes the image set so it loads another new 30 images will come into memory (time taken for this is okay) replacing previous set. And now another set of images are in displayed and user one can change the 4 images from new loaded set of 30. Another (third) type of user can change the screen turn into a live display or click images. While it is live display it just shows display of camera feed and while it is clicking pictures screen goes black. and only photos are clicked in background.

and the images are displayed with following way :

//Display Images
pushMatrix();
translate(width/2, height/2);
rotate(180*TWO_PI/360); 
scale(1,-1);
image(imgA,- imgA.width/2, height/2 - imgA.height);
typeInGuj(texts, -cwidth/2, height/2 - cheight);
popMatrix();


//top
pushMatrix();
translate(width/2, height/2);
rotate(90*TWO_PI/360);
scale(1,-1);
rotate(PI);
image(imgB, -imgB.width/2, height/2-imgB.height);   
typeInGuj(texts, -cwidth/2, height/2-cheight);
popMatrix(); 

//Left   
pushMatrix();
translate(width/2, height/2);
rotate(-90*TWO_PI/360);
scale(1,-1);
rotate(PI);
image(imgC, -imgC.width/2, height/2-imgC.height);  
typeInGuj(texts,-cwidth/2, height/2-cheight);
popMatrix();

//bottom
pushMatrix();
translate(width/2, height/2);
rotate(-180*TWO_PI/360);
scale(1,-1);
rotate(PI);
image(imgD, -imgD.width/2, height/2-imgD.height);
typeInGuj(texts, -cwidth/2, height/2-cheight);
popMatrix(); 

}

Bhargav

@gohai

i removed all webcam code related to webcam and just started running the program and parallel watching the task manager's bar of performance and CPU usage. Basically now the code has only image loading and displaying activity.

The CPU usagae seems fine, but found something is wrong with memory, this program somehow for each image being loaded on screen the memory is constantly increasing even if i am overrighting on same PImage variable! Now i realized why the Raspberrypi was getting hang.

There is one Pimage array which is fetching 30images from sdcard to RAM while program is running. Now actually there are 4 image variables which are pointing to four images from the 30images.

with users's input to change the 4 images to point on other images from set of 30, each of this time the memory is increasing almost 2MB. and

with user's input when new set of 30 images are loaded memory increases to almost 30mb.

I dont know how it is happening because I am loading images on same variables. It should over right the memory right?

koogs

The was a long standing bug where memory wasn't cleared when reusing pimages. I think the solution was to call dispose on the image (Google this). I also think this may've been fixed, should hope so.

If it's a small set of images load them all in setup.

I've also seen someone using jcache to store a pool of images but don't know whether that's suitable for a pi.

gohai

Another thing to keep in mind: Java is a garbage-collected language, and even on an OS-level pages won't necessarily be reclaimed before there is memory pressure. So looking at the output of top, and deducing from some number growing over time that there is a memory leak is not necessarily a valid observation.

There are specific JVM-level tools available that let you specifically monitor the lifetime of objects and the like, which would be much more apt for what you're trying to do.

Regarding performance: 1080p with the default renderer means pushing and copying quite a lot of pixels (e.g. from the USB camera to the PImage, from the PImage to the display window, from the display window to the screen). I am not too surprised that this is somewhat taxing the system to some degree.

Bhargav

@Koogs set of images are dynamic. And can't be loaded together. I'll checked for dipose on the pimage but looks like it is for GLVideo. I'll try shifting my video library and see if it works with video part of code. But my problem of memory is while loading images, the memory is constantly increasing even if I am overwriting on same variable for images.

@gohai I am a beginner level coder so using JVM evel tools and handling objects might be tough for me. Can you share more details and any supportive links about tutorials or anything?

Bhargav

@gohai @koogs

I was reading few blog posts on forums and I found the following set of instructions in my code which somehow reduced the issue.

I had set of instructions which were rewriting the images into same variable which eventually was expanding the use of memory. So before I rewrite the images i.e when I update PImage variables with new images I run following commands :

g.removeCache(imgA);
g.removeCache(imgB);
g.removeCache(imgC);
System.gc();

and the memory issue seem to resolve a little bit. Earlier the memory was expanding for each image being loaded i.e. any activity on screen. So the total amount of time I can run my code is from the time I start processing app on raspeberrypi to until when the memory is full! it will start taking memory in RAM increasing almost 1 to 2 Mb in every couple of seconds! And then in like 5-15 minutes raspberryPi will overload and hang, and will require manual restart.

Now it is only expanding 1-2 MBs while on each image set being updated. Now somehow My pi will run for almost few hours then minutes. But still it will overload the memory and hang. (Tested for 2 hours and worked well. Yet to be tested for long duration. I guess some code optimization will increase the time. I'll test and share the results.)

But I found on few blogs that calling System.gc() is a bad practice. I dont know why? Because it is working here. Would like to know if you have to say anything on this. @gohai @koogs

GoToLoop

https://forum.Processing.org/two/discussions/tagged?Tag=removecache(),system.gc()

Bhargav

@GoToLoop

Using g.removeCache(img) and System.gc(); is working for me here. I tested multiple times. And as shared by you in this thread. even if it is just a suggestion to java for cleaning but it is helping it. I want to know like as in the same thread and many other posts all are not suggesting to realy on system.gc() even if it works. And as of now i am using it at the end of the draw() function

As you have backed the system.gc() I would like to know whether you always use System.gc() ? or have you used any alternative methods which might be surely reliable and confirmed by fellow programmers?

GoToLoop

• About how reliable gc() is I'd say it's pretty good.
• However we should always have in mind that according to Java's specifications, the JVM's GC can simply ignore a gc() call when it decides that's not the right time to do it!
• From the few experiments I did w/ gc(), it is very good to delete all of the dangling keys from a WeakHashMap container: http://docs.Oracle.com/javase/8/docs/api/java/util/WeakHashMap.html
• But then, calling its method clear() would achieve the same 100% guaranteed; although it would take away the non-dangling 1s along as well: http://docs.Oracle.com/javase/8/docs/api/java/util/WeakHashMap.html#clear--
• A more realistic use for gc() is when we delete a big batch of objects in order to create another new big batch next.
• On that case, gc() serves as way to guarantee we got memory available right way before the big operation.
• A 3rd 1 is what you're trying to achieve. That is, aggressively attempting to release memory b/c you're creating new objects faster than JVM's GC can get rid of them.
• Although it's a losing battle, for eventually memory is gonna run out. Although much later... 8-|

Bhargav

@GoToLoop thanks for the description! I'll try to use clear also. And as you mentioned in the last point, i too realized that as the program runs further after a point memory is eventually gonna run out for sure, somehow able to extend some more time with gc() and removeCache().

This makes me conclude, that I wont be using raspberrypi and processing for a system with such requirement of memory to be cleaned, actually it is not requiring too much memory but due to memory clearance issues it is affecting the whole system setup. Just because of this I am changing my platform to OFx, in future I wont use processing on pi for a setup which has to run on its on. I might only use for testing small things in my presence.

@Gohai @Koogs Thanks for your support. I found processing easy to code, as I am not a proper programmer so I didn't try managing memory through JVM level tools or core java or linux on pi. In beginning I didn't realize that such issue may come over time. Further versions of processing should can automatically organize this and make processing more reliable to be used on Pi. Thanks :)