Processing 1.0 - Processing Discourse - A question about memory

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

‹ Previous Topic | Next Topic ›

Pages: 1

A question about memory (Read 1701 times)

homesick_alien

A question about memory
Feb 9^th, 2009, 4:52pm

Hello,

I'm working on a project where we need to load many (around 100) jpg images (1600x1200) and possibly some videos. The user will be able to manipulate them all, not at the same time, but we thought it would be better to load everything in the memory at startup, in order to have the media quickly available at any time.

As you can guess, we get a memory heap error - in our case when loading around 25 images. I read several posts here and I know that this is not the best practise, however, I would like to understand why this doesn't work.

We give 1.5Gb memory to processing (available system memory is 4Gb, windows vista sp1). These images are all together something like 10Mb only - where does the rest of the memory go?? Do you see any possible solution/hack for my case, apart from loading the images at the runtime?

Thanks!

PhiLho YaBB Moderator Offline Posts: 4190 Near Paris (France)	Re: A question about memory Reply #1 - Feb 9^th, 2009, 5:44pm Quote: These images are all together something like 10Mb only Not really. That's at least 732MB: 160012004*100/1024/1024 Images are stored uncompressed in memory.

homesick_alien YaBB Newbies Offline Posts: 26	Re: A question about memory Reply #2 - Feb 9^th, 2009, 6:37pm Oh, I never thought about that, thanks! But again, even with 700MB, shouldn't it work, as far as we provide 1.5G?

PhiLho

Re: A question about memory
Reply #3 - Feb 10^th, 2009, 12:02am

Not knowing your sketch, nor very well the internals of Processing (or Java!), I can't answer for sure, even less as you are stopped at a quarter of this figure.
But I suppose there are some expensive operations in Processing, eg. loadPixels() calls or get()/image(), and such. And if the garbage collector lags a bit, the memory might be saturated before GC gets a chance to clean it.

homesick_alien

Re: A question about memory
Reply #4 - Feb 10^th, 2009, 6:30pm

The thing is that apart from loading the images, I don't do anything special.. Now I exported as application and things seem to be better in terms of memory - I guess that a part of the RAM was reserved to processing itself before.

Anyway, I was just curious about that, thanks for bothering

Ofcourse, if someone has a deeper understanding of the whole thing, or any techniques for better performance, I would be happy to learn

Cheers!

kungfoo

Re: A question about memory
Reply #5 - Feb 16^th, 2009, 10:10am

You could read the compressed JPEG files to memory and only decompress the real image data, when an image has to be shown on screen.
You could do this in several ways, but the easiest, is probably to hold the data in memory and use ImageIO to decompress the jpegs only when you need them. Loading the images is probably slow, because it's disk access based, not because your processor is having a hard time deciding the images. Remember to null out the decompressed image data when you're done.
However this'll only work, if you don't need all the images fully on screen (or in memory to manipulate them) at any time.

Johnny Ow YaBB Newbies Offline Posts: 49	Re: A question about memory Reply #6 - Feb 19^th, 2009, 2:42am I some times call System.gc() before I load any large files in memory-hungry sketches to trigger a garbage collection. It's not guaranteed to force a garbage collection, isn't a recommended practice, and causes about a half second pause every time I call it, but it's worked for the most part.

kungfoo

Re: A question about memory
Reply #7 - Feb 19^th, 2009, 10:52am

Johnny Ow wrote on Feb 19^th, 2009, 2:42am:

I some times call System.gc() before I load any large files in memory-hungry sketches to trigger a garbage collection. It's not guaranteed to force a garbage collection, isn't a recommended practice, and causes about a half second pause every time I call it, but it's worked for the most part.

If you go in that deeply you might also want to have a look at: http://www.petefreitag.com/articles/gctuning/

You can tell the JVM how to do garbage collection. Some ways are more obtrusive to GUI-heavy applications, like the ones you are probably writing in Processing, since user actions should yield immediate feedback. Otherwise users will experience the system as "sluggish" or "lagging".

Johnny Ow YaBB Newbies Offline Posts: 49	Re: A question about memory Reply #8 - Feb 21^st, 2009, 9:47am kungfoo, thx. for the link. Will check it out in the morning when I'm a little more attentive.

depth

Re: A question about memory
Reply #9 - Feb 26^th, 2010, 11:04am

@kungfoo, thx for this link. i started this thread a couple days ago:
http://processing.org/discourse/yabb2/num_1267079727.html

i have a program that textures some polys with a Capture instance, and it's pausing every couple seconds for the GC (i'm pretty certain, but really just guessing). i'm trying to find a way to ameliorate this, and will dig through this link soon. but curious to hear if you have any suggestions (you seem to have some knowledge in this area....)

thanks!

Pages: 1

‹ Previous Topic | Next Topic ›