Problem with java heap size and rendering large images.

Programming Questions

lukas.arv..

Problem with java heap size and rendering large images.

in Programming Questions • 1 month ago

Hi!

I am currently rendering a quite large image (to an offscreen PGraphics) and once i hit 10000px * 7000px I get the following error:

Exception in thread "Animation Thread" java.lang.OutOfMemoryError: Java heap space

at java.awt.image.DataBufferInt.<init>(DataBufferInt.java:75)

at java.awt.image.Raster.createPackedRaster(Raster.java:467)

at java.awt.image.DirectColorModel.createCompatibleWritableRaster(DirectColorModel.java:1032)

at java.awt.image.BufferedImage.<init>(BufferedImage.java:331)

at processing.core.PImage.saveImageIO(PImage.java:3117)

at processing.core.PImage.save(PImage.java:3297)

at OffScreenRender.stopRender(OffScreenRender.java:38)

at MainVecField.draw(MainVecField.java:93)

at processing.core.PApplet.handleDraw(PApplet.java:2305)

at processing.core.PGraphicsJava2D.requestDraw(PGraphicsJava2D.java:243)

at processing.core.PApplet.run(PApplet.java:2176)

at java.lang.Thread.run(Thread.java:724)

If I have the image at 9000px * 7000px everything is fine.

I am using IntelliJ idea as my IDE and I have increased the standard heap space from 512 to 2048mb. The image with the resolution 9000px * 7000px renders fine with 512mb heapspace, but the one with 10000px * 7000px does not (even with 2048 mb heap space).

This is my offScreenRender class:

import processing.core.*;
/**
* Renders the ProcessingSketch to a ".jpg" file at specified resolution.
*/
public class OffScreenRender
{
PApplet parent; // The parent PApplet
PGraphics render;
PGraphics original;
int width;
int height;
int resFactor;
OffScreenRender(PApplet p, int width_, int height_, int resFactor_)
{
//Initialize variables
parent = p;
width = width_;
height = height_;
resFactor = resFactor_;
render = parent.createGraphics(width, height);
}
//Render the image
void startRender()
{
parent.beginRecord(render);
render.scale(resFactor);
}
//Render the image
void stopRender()
{
parent.endRecord();
render.save("hej.jpg");
}
}

This is my draw() function:

public void draw()
{
frameRate(30);
stroke(255);
//Check if the render should be saved
if(offScreenRenderSwitch == true){screenRender.startRender();}
background(0,0,0);
//Run vector field with action input
vecField.run(action.get());
action.display(g); //Display the action input
//Load vector forces into drawGraphic
graphic.loadVectorGuide(vecField.getForceVectors());
//Render the background object
backgroundVisual.display();
//Draw the graphic from vectorForces
graphic.display();
if(offScreenRenderSwitch == true)
{
screenRender.stopRender();
offScreenRenderSwitch = false;
}
}

So is this a Java error or do you think it could be a problem with the structure of my program?

Thank you!

Replies(4)

dimkir

Re: Problem with java heap size and rendering large images.

1 month ago

It's kinda tricky when you start working with super-huge data structure.

Probalby the reason is that when you allocate those big images, there's something happens behind the scenes which sometimes requires more elements to be kept in memory at the same time than you expect. As these are inner workings of java, this may be hard to detect and debug.

There's this advice on calling to Garbage Collector in your code, to make sure all unused memory is freed.

https://forums.oracle.com/message/5448465#5448465

maybe this helps.

If not,

I found few threads which may shed some light on your problem:

http://www.coderanch.com/t/416485//java/Java-BufferedImage-OutOfMemoryError

Then this google search has may lead you to solution

https://www.google.ie/search?q=java.awt.image.databufferint+outofmemoryerror

lukas.arv..

Re: Problem with java heap size and rendering large images.

1 month ago

Thank you for your reply!

The problem seems to be with two things. First I cannot properly increase the heap size in IntelliJ IDE. Secondly processing consumes more memory (for some reason) when saving to .jpg. If I change to .tif I can mange to push for slightly higher resolutions.

When using the processing IDE I can manage resolution up to 16000px * 12000px and maybe more with setting the memory to 3gb. So now I have to understand how to correctly set the heap size using IntelliJ.

Thank you!

Best regards,

Lukas

dimkir

Re: Problem with java heap size and rendering large images.

1 month ago

This seems to be a doc on increasing memory heap in Idea.

http://www.jetbrains.com/idea/webhelp/increasing-memory-heap.html

As to

consumes more memory (for some reason) when saving to .jpg

there's nothing much surprising. To compress your image, JPG compression algorithm needs to create few extra buffers. (Because this is not simple linear algorithm). So JPG compression algorithm creates more buffers in memory, which are the size of your image or most likely twice or 4 times the size of your image (depends on implementation). So at the moment of JPG compression algorithm running, actual space taken directly or indirectly by your image may be up to 4 times larger than the size of the image. Of course after JPG compression function is over, most of that memory is freed, but the crash happens exactly in the moment of compression.

Looks like when you save TIF (which basically just dumps the bytes of the image into file almost the way it is) there's no much "algorithm stuff" happening and thus no extra buffers are allocated. So this is why saving as TIFF doesn't crash your sketch.

lukas.arv..

Re: Problem with java heap size and rendering large images.

1 month ago

Thank you!

I guessed it must have something to do with the compression of the jpg file. Thank you for the explanation! Regarding the IntelliJ memory settings there are three different, you have a memory setting for the build process, and you also set the xms and xmx values of IntelliJ iteself. But the option i was looking for was in run / debug configuration where you could set the memory for the JRE (by using the xmx command). So if anyone else has this problem that is the place too look for.

Top Reply