PShape Memory Leak - Garbage Collection Pauses

Sama

It seems that PShapes don't get removed from the memory and instead build up causing long garbage collection / relocation times.

Below is an example of this. The pauses are garbage collections.

void setup () {
   size (200,400,P2D); 
}

void draw() {

  PShape newCells;
     newCells = createShape();  
       newCells.beginShape(TRIANGLES);

         for (int i = 0; i <  2000; i++) {
                newCells.vertex(random(0,width),random(0,height));
                newCells.vertex(random(0,width),random(0,height));
                newCells.vertex(random(0,width),random(0,height));
         }
       newCells.endShape();   

       shape(newCells, 0,0);
 }

I have looked at the garbage collections in VisualVM and it builds up very quickly and moves memory to Old gen.

The same doesn't happen with immediate mode (no PShape)

My questions are:

1) Is this a bug?

2) Would changing GC options help, if so how do you set this up on mac, I am a total novice with Terminal

Edit: Just to clarify, it is clear this is not the way to use PShape, in the context immediate mode or PShape(Group) is appropriate. But my problem is I need to create the shapes in real time, with thousands of triangle each frames, and then repeat the geometry in a mandala (radial symmetry) form.

K2017

I am bumping this ancient topic because I believe there is still a problem here. I have noticed the same behavior in 3.2.3 using PShapes; The shapes are created and memory usage goes up, all fine and good. Except that, when the sketch is closed, the memory is not freed, i.e the PShapes are not GC'd properly. Running the sketch again then produces an OOME, which is to be expected.

jeremydouglass

@K2017 -- what is the test sketch you are using to reproduce this problem -- is it the exact sketch from the OP, above? How are you exiting the sketch?

If this is a reproducible way of creating an Out of Memory Error on re-running a Processing 3.2.3 sketch then you should check whether there is already an issue on the GitHub Processing Issues -- if not please open a new issue:

• https://github.com/processing/processing/issues

K2017

@jeremydouglass Hmm, the sketch I am using is a little on the 'massive' side code-wise, it's actually a game I am currently working on. I'll try to provide a snippet (I don't believe the code is the problem though). The way I am exiting the sketch involves overriding the stop() function and doing some cleanup (Removing all references to PImage for instance); I thought I would be able to mitigate the memory problem, but this hasn't helped.

jeremydouglass

@K2017 -- ideally what the devs would be looking for is a very minimal sketch like the example above that simple creates PShapes and then exits in a way that causes an OOME on sketch re-run. (Personally I'm surprised that this is even possible, as I would have thought that all memory would be reclaimed by the OS on JVM exit, regardless of garbage collection -- e.g. http://stackoverflow.com/questions/39500630/garbage-collection-after-system-exit -- and not persist in any way between sketch runs -- but this isn't my area of expertise.)

If you are able to quickly scale your massive sketch down to something smaller that still exhibits the same problem then it may also help you isolate where the error is introduced -- and if it is a problem that can be dealt with by refactoring the code. For example, did you encounter the problem before overriding stop(), or did that introduce the problem?

K2017

I'll see what I can throw together to recreate it. What's odd in my case is that the memory persists in the sketch's java process even when the sketch is closed @-)

K2017

Ok, got some code here:

def setup():
    size(200, 200, OPENGL)
    global MAP
    MAP = ShapeTest()
    MAP.CompileMap()

def draw():
    global MAP
    background(0)
    translate(width/2, height/2, 10)
    shape(MAP.Get(), 0, 0)

class ShapeTest(object):
    def __init__(self):
        self.Size = 200

        self.Map = createShape(GROUP)

    def CompileMap(self):

        S = 10

        for i in range(self.Size):
            for j in range(self.Size):

                TileSet = createShape()
                TileSet.beginShape(QUAD)

                TileSet.vertex(S + 2*S*i, S + 2*S*j, 0)
                TileSet.vertex(S + 2*S*i, -S + 2*S*j, 0)
                TileSet.vertex(-S + 2*S*i, -S + 2*S*j, 0)
                TileSet.vertex(-S + 2*S*i, S + 2*S*j, 0)

                TileSet.endShape()

                self.Map.addChild(TileSet)

    def Get(self):
        return self.Map

Running and closing this sketch multiple times increases memory usage on my system.

jeremydouglass

Interesting -- I didn't realize that you were using Processing.py / Python mode, so that adds the wrinkle that (if this is a Java memory problem) your Java output is being created by from Jython.

Does this simple translation of your sketch into Java also exhibit the same memory error when you test it on your system?

ShapeTest shape;

void setup() {
  size(200, 200, P3D);
  shape = new ShapeTest();
  shape.compileMap();
}

void draw() {  
  background(0);
  translate(width/2, height/2, 10);
  shape(shape.get(), 0, 0);
}

class ShapeTest {
  int size;
  PShape map;
  ShapeTest() {
    this.size = 200;
    this.map = createShape(GROUP);
  }
  void compileMap() {
    int S = 10;
    for (int i=0; i<size; i++) {
      for (int j=0; j<size; j++) {
        PShape TileSet = createShape();
        TileSet.beginShape(QUAD);
        TileSet.vertex(S + 2*S*i, S + 2*S*j, 0);
        TileSet.vertex(S + 2*S*i, -S + 2*S*j, 0);
        TileSet.vertex(-S + 2*S*i, -S + 2*S*j, 0);
        TileSet.vertex(-S + 2*S*i, S + 2*S*j, 0);
        TileSet.endShape(); 
        this.map.addChild(TileSet);
      }
    }
  }
  PShape get() {
    return map;
  }
}

K2017

It in fact does not exhibit the same issue in java mode..., The sketch process is properly disposed and the memory is made available.

In that case it looks like there is a major bug in python mode. Time for a bug report I suppose. :|

jeremydouglass

@K2017 -- Processing Mode / Processing.py issues should be reported here:

• https://github.com/jdf/processing.py/issues

Thank you!

P.S. one question -- I notice that you python mode code declares global MAP in both setup and draw. Is that necessary? Might it matter?

GoToLoop

@jeremydouglass, global is only necessary for when we use the assignment operator =.

In setup(), it was indeed needed. But in draw(), given = wasn't applied there, global was redundant.

GoToLoop

http://py.Processing.org/reference/globals.html

Lord_of_the_Galaxy

Line 8 (as GoToLoop explains) is therefore not required.

GoToLoop

https://GitHub.com/jdf/processing.py/issues/233

JonathanFeinberg

Keep in mind that the Java mode simply quits when a sketch exits, but the Python mode keeps the sketch runner alive between runs (otherwise it would be too slow). So this may be exposing an existing bug in PShape that simply never comes up in Java mode. The bug may well be in Python Mode itself, of course! But it will take some looking into. Bugs like this usually indicate a global static somewhere in the Java end.

Lord_of_the_Galaxy

@JonathanFeinberg

but the Python mode keeps the sketch runner alive between runs (otherwise it would be too slow).

Would you mind explaining a bit? I thought Jython just converts Python code into Java byte code.

neilcsmith_net

Interesting thread. I just found a similar memory leak with PFont. I'll have a look at this issue myself in the next few days and see if I can reproduce and find the cause with VisualVM. It might be a similar circular reference in a cache somewhere.

PFont issue for reference - https://github.com/processing/processing/issues/4875