My first quick attemp has been to pass a fixed number of images, five in this case. Using five uniform sampler2D, tex0 to tex4, managing them in the CPU and passing all of them to the GPU per frame didn't throw an interesting frame rate rise.
Next step was to pass all five textures once at the beggining and then update just one per frame together with an
index
to the shader
to indicate which texture is the newest .
- import SimpleOpenNI.*;
- SimpleOpenNI context;
- PGraphics pg;
- PShader kSmooth;
- int texIndex = 0;
- boolean ready = false;
- public void setup()
- {
- size(640 , 2 * 480 +10, P2D);
- context = new SimpleOpenNI(this);
- context.setMirror(true);
- if(context.enableDepth() == false)
- {
- println("Can't open the depthMap, maybe the camera is not connected!");
- exit();
- return;
- }
- if(context.enableRGB() == false)
- {
- println("Can't open the rgbMap, maybe the camera is not connected or there is no rgbSensor!");
- exit();
- return;
- }
- pg = createGraphics(640,480,P2D);
- kSmooth = loadShader("kinectsmooth.glsl");
- kSmooth.set("width", 640);
- kSmooth.set("height", 480);
- }
- public void draw()
- {
- // update the cam
- context.update();
- background(200,0,0);
- image(context.depthImage(),0,0);
- if(!ready){
- kSmooth.set("tex"+texIndex, context.depthImage().get());
- texIndex++;
- if(texIndex==5)
- {
- texIndex = 0;
- ready=true;
- }
-
- }
- else if(ready)
- {
- kSmooth.set("tex"+texIndex, context.depthImage().get());
- pg.shader(kSmooth);
- pg.beginDraw();
- pg.rect(0,0,pg.width,pg.height);
- pg.endDraw();
- texIndex = (texIndex+1) % 5;
- }
- image(pg,0,context.depthHeight()+ 10,pg.width,pg.height);
- frame.setTitle(frameRate+" fps");
- }
When running this code I get an important boost to the frame rate, although the results are not as expected as there must be an error somewhere so the mean values are not correctly calculated. But the funny thing is that the memory used by the process keeps increasing until reaching the max memory available set in processing's preferences, then the program freezes for a few seconds, the memory is released and the process begins again, increasing the memory usage once again. So I guess the memory is not being managed correctly, but where?
By the way, this is an attemp to smooth output from the kinect by doing a weighted mean of previous frames.