Using Capture() with G4P. Urgent Help Please!

nsathiya

Hi,

Thanks in advance for your help. I am a beginner in Processing and in need of urgent advice. I need webcam to output frames into a second window, which I am using G4P library for. However, myCapture.read() either gets stuck in a frame or the fps count is very low so the video feed is not smooth. The draft code I tried is below:

import processing.video.*;
import g4p_controls.*;

Capture myCapture;
private GWindow window2;

GWinData data2;

void setup()
{
  size(640, 480);
  // The name of the capture device is dependent on
  // the cameras that are currently attached to your
  // computer. To get a list of the
  // choices, uncomment the following line
  println(Capture.list());

  data2 = new GWinData();

  window2 = new GWindow(this, "web cam", 0,0, width,height, true, OPENGL);
  window2.setActionOnClose(G4P.CLOSE_WINDOW);
  //window2.setResizable(true);
  //window2.addData(data2);
  window2.addDrawHandler(this, "Window2draw");


  print("debug");
  // To select the camera, replace "Camera Name"
  // in the next line with one from Capture.list()
  myCapture = new Capture(this, width, height, "FaceTime HD Camera (Built-in)", 30);

  // This code will try to use the camera used
  //myCapture = new Capture(this, width, height, 30);
  print("debug1");
  myCapture.start();
}

void Window2draw(GWinApplet b, GWinData d){

  if(myCapture.available())
  {
  myCapture.read();
  b.image(myCapture, 0, 0);
  }
}

void draw() {
  if(myCapture.available()){
    myCapture.read();
    image(myCapture, 0, 0);
  }
}

When I use 3 windows, the problem only gets worse and the camera freezes on the first frame. This is true for both the internal (Built in) and external logitech webcam. Im using Processing 2.0+ on Mac btw.

Thanks. BTW, G4P has been very helpful for this application.

quark

I have never tried using Capture with Video library so I have had a play with your code and I have a number of comments.

Comment 1

The second window is defined with the OPENGL renderer but since you are only displaying a bitmap image this is over the top and I would use JAVA2D.

Comment 2

Do not make a GWindow resizeable, I am pretty certain it won't work properly but by all means try it. Also does the second window size have to be the same size as the main window or the same size as the video capture 8->

Comment 3

If you want each GWindow to have its own set of attributes then you need to create a class that extends GWinData and add attributes to that. See the G4P_WindowStarter example and visit this webpage to find out more. If your GWindows don't need their own data then don't use GWinData - simply ignore it.

Comment 4

This is the biggy, the one you have been waiting for. You have defined a capture myCapture which will refresh 30 times a second. Now when myCapture.available returns true then it will read the image. While reading the image myCapture is no longer available. It means that the 30fps are shared by the two windows and since the draw method has priority the main window gets most of them and the GWindow gets very few. This will be made worse when another GWindow is added. There are two solutions - the first (see code below) simply displays the captured video in the GWindow so its gets all 30fps. The second approach (not discussed here) is only needed if you want the same video to be displayed in more than one window then you need a more sophisticated approach.

When I ran the code below on my iMac I got 60fps in both windows :)

import processing.video.*;
import g4p_controls.*;

Capture myCapture;
GWindow window2;

int x = 0;  

void setup() {
  size(700, 600);
  // The name of the capture device is dependent on
  // the cameras that are currently attached to your
  // computer. To get a list of the
  // choices, uncomment the following 2 lines
  for (String cam : Capture.list ())
    println(cam);

  window2 = new GWindow(this, "web cam", 0, 0, 640, 512, true, JAVA2D);
  window2.setActionOnClose(G4P.CLOSE_WINDOW);
  // Prevent G4P from clearing the background automatically
  window2.papplet.autoClear = false;
  window2.addDrawHandler(this, "window2draw");
  // I have changed name and size to match my computer
  myCapture = new Capture(this, 640, 512, "Built-in iSight", 30);
  myCapture.start();
}

void window2draw(GWinApplet b, GWinData d) {
  if (myCapture.available()) {
    b.background(200, 200, 255);
    myCapture.read();
    b.image(myCapture, 0, 0);
  }
}

void draw() {
  background(255);
  fill(0);
  text((int)frameRate, 20, 40);
  text((int)window2.papplet.frameRate, 20, 60);
  fill(255, 200, 255);
  ellipse(x, height/2, 20, 20);
  x = (x + 1)%width;
}

nsathiya

Comment 1- Yea I was just playing around with the other settings to see if they would have an impact. I will change it to JAVA2D.

Comment 2/ 3- The resizeable option wasn't working to begin with so I will remove that. I will also remove the unneeded data. The two windows will not have different attributes.

Comment 4- Thank you. So I better understand how it works now. What I need basically is for one window to play a slideshow of pics, and another window to display photos of from webcam so I can screenshot it at certain times. I currently has a workable draft. draw() uses webcam feed and the GWindow plays the slideshow. Do you think this is the most reliable way of going about it? i need to make sure the webcam and slideshow matches up and no delay is created.

Also, just out of curiosity, what would be your second approach? you dont need to code it if it will take time, just a brief explanation would do :)

Thanks.

quark

What I need basically is for one window to play a slideshow of pics, and another window to display photos of from webcam so I can screenshot it at certain times.

I assume you mean that one window is used to display a series of separate individual images (e.g. PImage objects) and the other window to display the webcam output (e.g. video)

I currently has a workable draft. draw() uses webcam feed and the GWindow plays the slideshow. Do you think this is the most reliable way of going about it? i need to make sure the webcam and slideshow matches up and no delay is created.

In my example code I display the webcam output in the secondary window (GWindow) and the animation in the main window but I see no reason why you can't swap these. I suspect that it is important that the framerate for the webcam window is greater than the capture speed for the webcam i.e. > 30fps. The harder part will be to time the slide show so it syncs with the video.

Also, just out of curiosity, what would be your second approach? you dont need to code it if it will take time, just a brief explanation would do

The trick here is to store the captured image in a buffer (PImage) outside of any draw method and then display the buffer in any window we like.

The code below shows the video output in 3 windows (each window is running at ~60fps at least on my machine)

import processing.video.*;
import g4p_controls.*;

Capture myCapture;
GWindow window2, window3;
// Bufer for captured image
PImage buffer;

int x = 0;  

void setup() {
  size(700, 600, JAVA2D);
  // The name of the capture device is dependent on
  // the cameras that are currently attached to your
  // computer. To get a list of the
  // choices, uncomment the following 2 lines
  for (String cam : Capture.list ())
    println(cam);

  window2 = new GWindow(this, "web cam", 0, 0, 640, 512, true, JAVA2D);
  window2.setActionOnClose(G4P.CLOSE_WINDOW);
  // Prevent G4P from clearing the background automatically
  window2.papplet.autoClear = false;
  window2.addDrawHandler(this, "window2draw");
  // Third window
  window3 = new GWindow(this, "web cam", 640, 0, 640, 512, true, JAVA2D);
  window3.setActionOnClose(G4P.CLOSE_WINDOW);
  // Prevent G4P from clearing the background automatically
  window3.papplet.autoClear = false;
  window3.addDrawHandler(this, "window3draw");
  // I have changed name and size to match my computer
  myCapture = new Capture(this, 640, 512, "Built-in iSight", 30);
  myCapture.start();

  // Use the pre method to capture the image
  registerMethod("pre", this);
}

// This is always executed just before the draw() method is executed
void pre() {
  if (myCapture.available()) {
    myCapture.read();
    buffer = myCapture.get();
  }
}

void window2draw(GWinApplet b, GWinData d) {
  if (buffer != null) b.image(buffer, 0, 0);
}

void window3draw(GWinApplet b, GWinData d) {
  if (buffer != null) b.image(buffer, 0, 0);
}

void draw() {
  background(255);
  if (buffer != null) image(buffer, 0, 0);
  fill(0);
  text((int)frameRate, 20, 40);
  text((int)window2.papplet.frameRate, 20, 60);
  text((int)window3.papplet.frameRate, 20, 80);
  fill(255, 200, 255);
  ellipse(x, height/2, 20, 20);

  x = (x + 1)%width;
}

nsathiya

Yes, the hardest part has been getting the timing right. I have been trying it with flags and delay() so just a bit of tweaking is necessary. I think for the most part I got the pics to synchronize.

And I haven't looked at pre() but thanks! I will check up on the documentation to see if it fits my project.

Appreciate all the help :)

quark

I wouldn't use delay() - there are numerous discussions on the forum using millis() to create a timer/watch type class for timing events. I suggest you search for a few.

Although this is not exactly what you need it shows how millis can be used.

nsathiya

Yea my timer uses millis() but delay() helps to synchronize it a bit. But I also read on another forum not to use it so i will tweak it with millis(). Your post was helpful. Thanks.

nsathiya

a good Timer class that I came across, if anyone reading this is interested-

http://processing.org/discourse/beta/num_1265681408.html