We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpVideo Capture,  Movie Playback,  Vision Libraries › The secret how to improve Capture speed...
Page Index Toggle Pages: 1
The secret how to improve Capture speed... (Read 1602 times)
The secret how to improve Capture speed...
Sep 15th, 2005, 6:05pm
 
...would be interesting to know!
I find it hard to believe that capturing a 640x480 video@25 fps brings processing as much to heavy breathing as it does.

What are the sources of this problem - and what could be improved. Is this running on Macs as bad as on Pcs?

Are there better VDIGs then the free WinVdig?
Any option for hardware support?

Here I get maybe 5-10 fps if I dont run in OpenGL, whereas OpenGL is causing some other problems as we know.
Black and white also seems to get some relief.
Re: The secret how to improve Capture speed...
Reply #1 - Sep 16th, 2005, 3:27pm
 
There's a chance it might be a hardware issue. Post the names of the devices and somone might know of any issues.

I had some excellent speed from a Creative Live Pro web cam, a Logitech Communicator ground the whole system to a halt, and the oh so dodgy Trust Web cam was tricky to set up what with it over writing other drivers but ran really fast once you got it going because it was so light on system resources (and it was the cheapest cam about).

WinVDig has rarely been the main cause of any issues I've had. Plus the guy who maintains it was reasonably approachable.

If you've got some spare cash you might want to nip out and get a cheap-as-chips capture device to check the performance differences.
Re: The secret how to improve Capture speed...
Reply #2 - Sep 16th, 2005, 6:33pm
 
Hey Aaron,

I am using mainly a Philips ToUcam 840K which has a nice CCD sensor, but I also tried with a Logitech Quickcam pro - no big difference.
As soon as I simply capture 640x480 @ 25 fps, the delay is horrible and CPU is @ 100%.

If I check these cams in other proggs liek EyesWeb Wink they have perfect speed, almost no dropped frames, so it is Processing specific.


Can you tell me that you get really good speeds with Processing at 640*480?
Maybe check the script below (from a Forum member, dont remember who it was, thanks anyway), and tell me if its fast (check CPU as well).
If thats the case, I see some light on the end of the very dark tunnel Im in right now Smiley

Code:

import processing.opengl.*;
import processing.video.*;  

PGraphics3 tmp;
PImage img, oldCam;
Capture cam;

boolean src = false, shiftdown = false;
int sens = 70;
int[] ax,ay;
int num = 0;

void setup(){
 size(320, 240, OPENGL);
 framerate(20);
 String s =  Capture.list()[0];
 cam = new Capture(this, width, height, 20);
 tmp =  new PGraphics3(width,height,null);
 img = new PImage(width, height);
 oldCam = img;
}

void draw(){
 image(renderDifference(cam.pixels, oldCam.pixels, sens),0,0);
}  

void captureEvent(Capture camera) {  
 cam.read();  
}  

PGraphics3 renderDifference(color[] a, color[] b, int sens){
 
 color trans = color(255,255,255); //the colour of the pixels w/ movement
 color ch = color(0,0,0); //the colour of the pixels that haven't changed
 
 for(int i = 0; i < width; i++)
   for(int j = 0; j < height; j++)
   {
    float bA = brightness(a[j*width+i]);
    float bB = brightness(b[j*width+i]);

    if (sens<abs(bA-bB))  
      tmp.pixels[j*width+i] = ch;
    else  
      tmp.pixels[j*width+i] = trans;
   }
 }
 tmp.updatePixels();
 arraycopy(cam.pixels, oldCam.pixels);
 
 return tmp;
}


Another thing: which capture devices or cameras can you recommend for getting a good image at 720*576 or at least 640*480?
Re: The secret how to improve Capture speed...
Reply #3 - Sep 17th, 2005, 12:46pm
 
Hi beachmeat,

your renderDifference method needs some serious optimization. There are several unnecessary calculations that can be replaced:

Code:

PGraphics3 renderDifference(color[] a, color[] b, int sens){  
 /*
color trans = color(255,255,255); //the colour of the pixels w/ movement  
color ch = color(0,0,0); //the colour of the pixels that haven't changed  
*/    

// your version maybe nicer to read, but you can simply replace this by:

int trans = 0xffffffff;
int ch= 0xff000000;
 
// though that will not make much of a difference

// this is where the horrible part starts - two nested
// loops? Why? You are just traversing through the pixels
// array, in order to check every pixel. But the pixels
// array is 1-dimensional, so you adding extra effort here
// because two loops will force you to use multiplication

/*
 for(int i = 0; i < width; i++)  
   for(int j = 0; j < height; j++)
   {  
    float bA = brightness(a[j*width+i]);  
    float bB = brightness(b[j*width+i]);  
 
    if (sens<abs(bA-bB))  
      tmp.pixels[j*width+i] = ch;  
    else    
      tmp.pixels[j*width+i] = trans;  
   }  
 }  
*/

// this is already smaller:

// use a reference to the pixels array
int[] p = tmp.pixels;

for (int i = width*height; --i>-1;)
{
    float bA = brightness(a[i]);  
    float bB = brightness(b[i]);  
 
    if (sens<abs(bA-bB))  
      p[i] = ch;  
    else    
      p[i] = trans;
}

// it could be further improved, though. The two
// brightness() calls should be at least unrolled.
// Though maybe you could just compare the red channel.
// Also the abs() is expensive if you use it 640*480 times
// in a loop.

 tmp.updatePixels();
 arraycopy(cam.pixels, oldCam.pixels);
 
 return tmp;  
}

Re: The secret how to improve Capture speed...
Reply #4 - Sep 17th, 2005, 1:00pm
 
I did not test this yet, but you might also be able to use the XOR function to compare the two bitmaps:

Code:

int[] p = tmp.pixels;

for (int i = width*height; --i>-1;)
{
float b = brightness(a[i]^b[i]);
p[i] = ( sens<b ? ch : trans );
}
Re: The secret how to improve Capture speed...
Reply #5 - Sep 17th, 2005, 5:21pm
 
since performance seems to come up a lot, i've added a note to the faq about it (at the end under "all platforms")
http://processing.org/faq/bugs.html#video
Re: The secret how to improve Capture speed...
Reply #6 - Sep 17th, 2005, 6:04pm
 
On a side note, I have found that if I double the fps (run at 60fps instead of 30), even if the source stream is only 30fps, it runs much much smoother.   However, it seems with updated Quicktime, I can't run anything to back up my comments right now (java.lang.UnsatisfiedLinkError: lockDrawingSurface
).

Marcello
Re: The secret how to improve Capture speed...
Reply #7 - Sep 17th, 2005, 6:32pm
 
Thanks for your improvements Sensei Quasimondo because they offer a lot learning possibility, but this was just some copy paste from another code from a user here of the forum. I was already checking quite some posts of you here, since you seem to be the master of image bit-calculations

But the thing is that you could as well just display any direct output of camera capture, and it would not be the speed one would dream of. 640/480 @ 30fps are impossible for me here.

I completly understand that Processing is not ment to be used for this kind of video analysis, and its doing a great job for what its good at - but I simply dont know what else to use for making some realtime image analysis with background subtraction and in the end have a processing output on it, because thats the tricky part where the proggs I have looked into failed - I couldnt send stuff to Processing in the required way (like a very big 1bit matrix, or).

But anyway I always guess there is room for improvement, be it hardware or drivers or vdig. But if you, Fry, say that maybe double speed could be reached by pushing the quicktime integration further, then there is room for hope Smiley
There are very very many projects that use video input from webcams or whatever, so its really a great idea to extend Processings power there I believe...I hope the right person will come across our way nd be able to spend some time on that, because I definitly suck at this.

@cello I really like your flyingplanes sketch! So simple yet so fun to play around with
Page Index Toggle Pages: 1