In dire need of optimization help with Android opengl and ketai cam

bashardouba

Can someone suggest a modification to optimize performance in terms of the following? I'm currently running the following code to superimpose a box onto my cam stream (android) using Ketai with OpenGL. I've tried using individual pixels with little gain. Also, I can't utilize image(..) because I have gyroscope data manipulating a 3D cam. Please help: the performance is waaaaaaaay to slow. The issue seems to be related to the background(..) method.

import ketai.sensors.*;
import processing.opengl.*;

int ty=0,tx=0,tz=0;
KetaiSensor sensor;
float rotX, rotY, rotZ;

///Video Cam begin
import ketai.camera.*;
KetaiCamera cam;
///Video Cam end

void setup()
{
  size(1920,1080,OPENGL);
  sensor = new KetaiSensor(this);
  sensor.start();
  orientation(LANDSCAPE);
//////////CAM
cam = new KetaiCamera(this, 1920,1080, 25);//480*800 original
//////////CAM

}

void draw()
{

background(cam);
ty+=rotY;
tx+=rotX;

 lights();

  // Change height of the camera with mouseY
  camera(0, 0, 220.0, // eyeX, eyeY, eyeZ
         -tx, ty, 0, // centerX, centerY, centerZ
        0, 1.0, 0.0); // upX, upY, upZ
  box(90);
}

void onGyroscopeEvent(float x, float y, float z)
{
rotX=x*4;
rotY=y*4;
rotZ=z*4;
}



////CAM BEGIN
void mousePressed()
{
  if (cam.isStarted())
    cam.stop();
  else
    cam.start();
}
void onCameraPreviewEvent()
{
  cam.read();
}

bashardouba

Perhaps there is a way to utilize lower camera resolution but then stretch the image out to 1920x1080?

bashardouba

Any help would be appreciated. Really stuck here...

calsign

I don't see why you can't use image(). If I had to guess, a large portion of the lag comes from the image being re-sized to the size of the screen every time you call background(). I don't know how to change the resolution of the camera's output, but that sounds like another potential option worth pursuing.

bashardouba

Hey man!

The issue with image() was that I had a camera manipulated via gyro data in the sketch. Hence it was difficult to get the image to stick in place. However, with a little beginCamera, endCamera, push and pop matrix, everything worked out :)

design24439

bashardouba can you let me now how you solve the problem with background(cam), my app on the phone continue closign every time when the cam start, i also whant to put some 3dshapes and i want the cam like a bacground so don interfer with the shapes.

appreciate your help.

bashardouba

/Camera
import ketai.camera.*;
KetaiCamera cam;
//Camera
void setup() {
  /////3D
  size(1920,1080,OPENGL); //revert back to 480*800 for original
  sensor = new KetaiSensor(this);
  sensor.start();
  orientation(LANDSCAPE); //revert back to PORTRAIT
  /////3D

  t1=new disObject(0,0);
  t2=new disObject(0,900);

//////////CAM
cam = new KetaiCamera(this, 640,480, 25);//480*800 original
cam.start();
//////////CAM

}//close setup

void draw() {  
  tx+=rotX;
  ty+=rotY;
background(0);
  pushMatrix();
  hint(DISABLE_DEPTH_TEST);
  lights();
  textMode(MODEL);
  textMode(MODEL);
    scale(3,2.25);
  image(cam,0,0);

  hint(ENABLE_DEPTH_TEST);
  popMatrix();

beginCamera();
pushMatrix();
camera(width/2.0, height/2.0, (height/2.0) / tan(PI*30.0 / 180.0), -tx+width/2.0, ty+height/2.0, 0, 0, 1, 0);
translate(eyex,0,eyez);//for testing
t1.display();//these t's are my 3d objects
t2.display();
popMatrix();
endCamera();
}

/////3D
void onGyroscopeEvent(float x, float y, float z)
{
rotX=x*12;//1.5 test
rotY=y*12;//1.5 test
rotZ=z*4;
}
/////3D


////CAM BEGIN

void onCameraPreviewEvent()
{
  cam.read();
}
////CAM END

bashardouba

There are some extra bits in there I can't quite remember why I used (it's been a long time). But the general concept is to:

pushMatrix
disable depth test
image(cam)
enable depth test
popMatrix
beginCamera
pushMatrix
camera(... settings for the opengl cam ..)
draw 3d objects
popmatrix
endCamera