Processing 1.0 - Processing Discourse - java.lang.OutOfMemoryError

alphachapmtl

java.lang.OutOfMemoryError
Feb 6^th, 2008, 8:05am

I get this error:
java.lang.OutOfMemoryError
when executing this line (creating a big array):
int[] plist = new int[count];

Is my computer OutOfMemory or only Java? Can I make Processing/Java automatically use all available memory?
Should I change something in file preferences.txt?
What values should I put for
run.options.memory=
run.options.memory.initial=
run.options.memory.maximum=

as

Re: java.lang.OutOfMemoryError
Reply #1 - Feb 6^th, 2008, 1:39pm

i also get a similar error (java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space) and been trying to increase java heap size through processing preferences file without success.

wondering on the net i found people with similar problems just boosting a bit the java heap size works for them( ex: http://www.informix-zone.com/node/46)

3 grids of 256x256 in 3d are enough to burst java heap memory.
is there a workaround to make those settings persistent in the launched sketch and in the exported sketches/apps? this is because i tried to change the values (min heap 1024, max heap 4096) and couldnt go beyond grids of 150x150, the same as not changing those memory settings.

as YaBB Newbies Offline Posts: 30	Re: java.lang.OutOfMemoryError Reply #2 - Feb 6^th, 2008, 3:08pm hi again i just rechecked and it worked. this is on processing0135. i just went to preferences in the processing app, there is a max memory pane. in my case i put 1024mb and it worked. also worked as exported applet. thanks for prompting this, had this problem before and didnt know how to do this.. hope you can too. a

alphachapmtl

Re: java.lang.OutOfMemoryError
Reply #3 - Feb 6^th, 2008, 7:17pm

-- i just went to preferences in the processing app,
there is a max memory pane. --

What if the box for "Set maximum available memory to .. MB" is not checked? What would that mean? Is checking the box limiting the available memory? If my computer has 2Gig memory, should I put 2048 in there and check the box?

-- Java heap space error --

I had that problem in a previous program, so I programmed my own stack class with putin() and getout() methods, and changed my code to use that stack instead of relying on Java's heap.

as

Re: java.lang.OutOfMemoryError
Reply #4 - Feb 6^th, 2008, 8:28pm

>What if the box for "Set maximum available memory to .. MB" is not checked?

i guess you would have default memory java virtual machine defaultly allocates.

>What would that mean? Is checking the box limiting the available memory? If my computer has 2Gig memory, should I put 2048 in there and check the box?

you are not limited by actual ram amount. modern systems use empty disk space as ram space, so you can allocate more depending on your needs.
i checked and setting the value in the processing preferences puts a var in the preferences file (i have 2 on osx, one on processing/lib and other on ~/documents/processing/preferences.txt, where this value gets put)

run.options.memory.maximum=256

-- Java heap space error --

>I had that problem in a previous program, so I programmed my own stack class with putin() and getout() methods, and changed my code to use that stack instead of relying on Java's heap.

what i described is just encountering heap allocation erros, which is what i couldnt do, and boosting the memory in the preference pane above fixes that.. if you use your own stack method, dont you have to allocate memory for it somewhat also?

best regards
a

JohnG Ex Member	Re: java.lang.OutOfMemoryError Reply #5 - Feb 6^th, 2008, 9:16pm Just because you call a class a stack, doesn't mean it goes on stack memory I'm afraid. The only thing that goes on the stack (from really old memories of other languages) are ints, float and other built-in types. And object goes into the heap, no matter how you use it.

alphachapmtl

Re: java.lang.OutOfMemoryError
Reply #6 - Feb 6^th, 2008, 9:32pm

-- Java heap space error --
... so I programmed my own stack class with putin() and getout() methods, and changed my code to use that stack instead of relying on Java's heap.

>>> My Stack class did not replace Java's heap. I had a recursive method which could not go deep enough cause of Java heap overflow. So I replace the automatic recursion with a manual recursion using my own stack. Of course my stack (implemented as an array) used memory, but I could allocate as much as I needed.
Java's heap overflow could also happens for other reasons, in which case my solution would not apply.

alphachapmtl

Re: java.lang.OutOfMemoryError
Reply #7 - Feb 7^th, 2008, 8:18pm

As suggested, in File/Preferences I tried "Set maximum available memory to 1088 MB" and checked the box.
After that, my primelist program went 23 times further, to
791000000,
instead of
33700000.

However, I could set to no more than 1088MB, even if my system has 2GB.
I also went to ControlPanel/System/Advanced/Performance/Settings/Advanced/VirtualMemory where I increased the (Total paging file size) to 3582MB, but this had no effect.
I still could not Set maximum available memory to more than 1088 MB in File/Preferences.

27ipunt

Re: java.lang.OutOfMemoryError
Reply #8 - Feb 8^th, 2008, 11:18am

I think that java heap size is limited by the physical memory and the memory used/reserved by windows (or any OS). I have not tested this in windows, but i think that the limit is something like 1.3g (in a 32bits windows with a 32bits jvm), in linux i was able to allocate more heap space.

http://www.unixville.com/%7Emoazam/2004/06/03.html

So i guess there is no easy solution ... unless you have a 64 bits windows.

alphachapmtl

Re: java.lang.OutOfMemoryError
Reply #9 - Feb 8^th, 2008, 2:16pm

Quote:

This is a limitation of the Windows 32bit OS. 32bit processes can only use a max of 4GB memory address space. Windows further splits that into half by allocating 2GB to the kernel and 2GB to the application.

This mean Windows XP could actually use 4GB, with a full 2GB for the applications?
So I could upgrade my memory from 2GB to 4GB with some benefit?

27ipunt

Re: java.lang.OutOfMemoryError
Reply #10 - Feb 8^th, 2008, 3:07pm

alphachapmtl wrote on Feb 8^th, 2008, 2:16pm:

This mean Windows XP could actually use 4GB, with a full 2GB for the applications
So I could upgrade my memory from 2GB to 4GB with some benefit

I'm sorry, but no, from what i have read some of these 2Gb are used for other parts of the memory of your program, the jvm itself and some shared windows components. So what people say (from experience) is that the maximum is about 1.5G and something normal is about 1.3G. Your 1088Mb is a bit on the low side, but it depends on installed libraries, so I think that with a fresh windows install you may get closer to those 1.5G, but not sure. Also i have just read that another implementation of java (JVM) doesn't have this problem so maybe switching to this one may help, i have not tried it but in case you want to try you can find it here http://commerce.bea.com/products/weblogicjrockit/jrockit_prod_fam.jsp
You should get version 5.0 or 6 cause 1.4.2 doesn't support this feature. Also you have to make processing use this jvm, so i guess you should download the expert version without java and add this new java to the path before any other java versions you have installed (if you don't know how just ask).

Of course you could also try to make your program less memory hungry Tongue

what kind of thing do you think is the bottleneck recursion

JohnG

Re: java.lang.OutOfMemoryError
Reply #11 - Feb 8^th, 2008, 3:34pm

alphachapmtl wrote on Feb 8^th, 2008, 2:16pm:

This mean Windows XP could actually use 4GB, with a full 2GB for the applications
So I could upgrade my memory from 2GB to 4GB with some benefit

Unfortunately XP 32-bit uses some of the 2-4Gb memory space to talk to your video card, so you'd lose an amount proportional to the memory on your video card.

fry

Re: java.lang.OutOfMemoryError
Reply #12 - Feb 8^th, 2008, 8:37pm

lots of this is covered here on the troubleshooting page:
http://processing.org/reference/troubleshooting/index.html#memory

it's not a good idea to modify run.options.memory in the preferences file, just use the preferences dialog.

i'm also adding this paragraph to the memory section linked above:
32-bit operating systems and software are usually limited to addressing
somewhere around either 2GB and 4GB of RAM at a time. This means that
even if you have 8GB of RAM installed in your machine, you may only be
able to use shy of 2GB per application. Most Windows systems seem to be
limited to 2GB, while Mac OS X and Linux can usually access shy of 4GB.
Upcoming 64-bit operating systems get around this restriction and increase
the amount of available memory significantly. However, even if you're
running a 64-bit OS like Mac OS 10.5, or a 64-bit Linux, a combination of
factors (version of Java, etc.) may mean that you're still limited by the
32-bit boundary of 2GB or 4GB.

alphachapmtl

Re: java.lang.OutOfMemoryError
Reply #13 - Feb 9^th, 2008, 1:10am

Thanks for all that info. I wish those memory issues would be simpler, transparent, automatic. A simple user should not have to deal with it.

>> I'm using Processing-Java, which only support Java 1.4 and earlier (too bad for me):
http://www.processing.org/faq.html#java

>> I also found this which may be useful:
http://www.processing.org/reference/troubleshooting/index.html#memory
Quote:

To check the amount of memory that's used so far, or how much is available, use Java's Runtime object:

// The amount of memory allocated so far (usually the -Xms setting)
long allocated = Runtime.getRuntime().totalMemory();

// Free memory out of the amount allocated (value above minus used)
long free = Runtime.getRuntime().freeMemory();

// The maximum amount of memory that can eventually be consumed
// by this application. This is the value set by the Preferences
// dialog box to increase the memory settings for an application.
long maximum = Runtime.getRuntime().maxMemory();