We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
Index › Programming Questions & Help › OpenGL and 3D Libraries › OPENGL and frame differencing
‹ Previous Topic | Next Topic ›
Page Index Toggle Pages: 1
OPENGL and frame differencing (Read 2237 times)
JHT83
OPENGL and frame differencing
Nov 17th, 2009, 1:02pm  
Hi,

I just need to get a simple motion detection working whereby an event occurs over a threshold of motion.

The very simple frame differencing sketch in the processing examples folder is ideal. However, it does not seem to work with OPENGL.

Does anybody know an alternative that does this as simply?

I have included the code below. If you remove OPENGL from the size () method you will see the sketch working and producing a white ellipse when the motion goes over a certain threshold.

thanks

Code:

 * Frame Differencing 
 * by Golan Levin. 
 * 
 * Quantify the amount of movement in the video frame using frame-differencing.
 */ 


import processing.video.*;

int numPixels;
int[] previousFrame;
Capture video;

void setup() {
  size(640, 480); // Change size to 320 x 240 if too slow at 640 x 480
  // Uses the default video input, see the reference if this causes an error
  video = new Capture(this, width, height, 24);
  numPixels = video.width * video.height;
  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();
}

void draw() {
  if (video.available()) {
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available
    
    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
	color currColor = video.pixels[i];
	color prevColor = previousFrame[i];
	// Extract the red, green, and blue components from current pixel
	int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
	int currG = (currColor >> 8) & 0xFF;
	int currB = currColor & 0xFF;
	// Extract red, green, and blue components from previous pixel
	int prevR = (prevColor >> 16) & 0xFF;
	int prevG = (prevColor >> 8) & 0xFF;
	int prevB = prevColor & 0xFF;
	// Compute the difference of the red, green, and blue values
	int diffR = abs(currR - prevR);
	int diffG = abs(currG - prevG);
	int diffB = abs(currB - prevB);
	// Add these differences to the running tally
	movementSum += diffR + diffG + diffB;
	// Render the difference image to the screen
    
    
    //  pixels[i] = color(diffR, diffG, diffB);
    
    
	// The following line is much faster, but more confusing to read
	pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
	// Save the current color into the 'previous' buffer
	previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 0) {
	updatePixels();
	println(movementSum); // Print the total amount of movement to the console
    }
    background(0);
  if (movementSum > 9657343) {
	ellipse(56, 46, 55, 55); // Print the total amount of movement to the console
    }
 // if (movementSum > 7010223) {
  //  rect(100, 100, 10, 10);
   // }
  }
}
Back to top
 
 
JHT83
Re: OPENGL and frame differencing
Reply #1 - Nov 17th, 2009, 1:19pm  
My apologies,

I should have said I think that I need to use OPENGL as it is with the toxi verlet physics libraries. I have noticed that other motion detection sketches have used P3D instead. Is there a way of converting OPENGL to P3D? Or a way of getting any motion detection to work with OPENGL?

Thanks
Back to top
 
 
Cedric
Re: OPENGL and frame differencing
Reply #2 - Nov 17th, 2009, 2:03pm  
there is not reason why you shouldnt be able to work with toxiclibs and P3D... whats the problem there ?
Back to top
 
... twitter.com/CedricKiefer
WWW CedricKiefer 9570393 CedricKiefer  
JHT83
Re: OPENGL and frame differencing
Reply #3 - Nov 17th, 2009, 2:17pm  
Hi Cedric,

Thank you very much for getting back to me. I have the soft body demo from Toxi and when I replace the OPENGL with P3D, nothing comes up on the screen. I realise that there must be something in the syntax making it only work with OPENGL. However, I don't know what this is.

I have included the code below in case you can spot it.  Thank you again Cedric.

Code:

/**
 * Softbody square demo
 */
import processing.opengl.*;

import toxi.physics2d.constraints.*;
import toxi.physics2d.*;

import toxi.geom.*;
import toxi.math.*;

int DIM=10;
int REST_LENGTH=20;
float STRENGTH=0.125;
float INNER_STRENGTH = 0.2;

VerletPhysics2D physics;
VerletParticle2D head,tail;

void setup() {
  size(1280,720,OPENGL);
  smooth();
  physics=new VerletPhysics2D();
  physics.gravity=new Vec2D(0,0.1);
  physics.setWorldBounds(new Rect(0,0,width,height));
  for(int y=0,idx=0; y<DIM; y++) {
    for(int x=0; x<DIM; x++) {
	VerletParticle2D p=new VerletParticle2D(x*REST_LENGTH,y*REST_LENGTH);
	physics.addParticle(p);
	if (x>0) {
	  VerletSpring2D s=new VerletSpring2D(p,physics.particles.get(idx-1),REST_LENGTH,STRENGTH);
	  physics.addSpring(s);
	}
	if (y>0) {
	  VerletSpring2D s=new VerletSpring2D(p,physics.particles.get(idx-DIM),REST_LENGTH,STRENGTH);
	  physics.addSpring(s);
	}
	idx++;
    }
  }
  VerletParticle2D p=physics.particles.get(0);
  VerletParticle2D q=physics.particles.get(physics.particles.size()-1);
  float len=sqrt(sq(REST_LENGTH*(DIM-1))*2);
  VerletSpring2D s=new VerletSpring2D(p,q,len,INNER_STRENGTH);
  physics.addSpring(s);
  p=physics.particles.get(DIM-1);
  q=physics.particles.get(physics.particles.size()-DIM);
  s=new VerletSpring2D(p,q,len,INNER_STRENGTH);
  physics.addSpring(s);
  head=physics.particles.get((DIM-1)/2);
  head.lock();
}

void draw() {
  background(0);
  stroke(255);
  head.set(mouseX,mouseY);
  physics.update();
  for(Iterator i=physics.springs.iterator(); i.hasNext();) {
    VerletSpring2D s=(VerletSpring2D)i.next();
    line(s.a.x,s.a.y,s.b.x,s.b.y);
  }
}
Back to top
 
 
JHT83
Re: OPENGL and frame differencing
Reply #4 - Nov 17th, 2009, 2:26pm  
Actually,

My apologies. It does work in P3D. But I have a sketch that uses multiple mesh classes and so for one P3D seems to be really really slow. However, the main problem is still that for some reason the frame differencing code does not seem to work in this sketch even though the frame differencing code on it's own works with P3D.

Do you have any ideas on how to speed up the P3D and how to incorporate the frame differencing into the soft body demo?

Thank you
Back to top
 
 
JHT83
Re: OPENGL and frame differencing
Reply #5 - Nov 17th, 2009, 2:34pm  
Hi Cedric,

My apologies for this. I am quite stressed due to time limit and so I have rushed into sending a lot of posts.

I have now got frame differencing working with my more complex sketch using Toxi's soft body demo.

However, the P3D is running really really slowly still. I wondered if there was a way to somehow speed this up or use OPENGL with frame differencing. (that's my last question Smiley

Thank you
Back to top
 
 
Cedric
Re: OPENGL and frame differencing
Reply #6 - Nov 17th, 2009, 2:44pm  
ok like you just realized. the code you posted works with both, openGL and P3D. so what is the problem now?
that in your example P3D is really slow but you need it for the frame differencing?  where did you find that frame differencing example?
Back to top
 
... twitter.com/CedricKiefer
WWW CedricKiefer 9570393 CedricKiefer  
JHT83
Re: OPENGL and frame differencing
Reply #7 - Nov 17th, 2009, 2:49pm  
Hi Cedric,

Thank you again for the reply. The frame differencing code is below:

Code:
/**
 * Frame Differencing 
 * by Golan Levin. 
 * 
 * Quantify the amount of movement in the video frame using frame-differencing.
 */ 


import processing.video.*;

int numPixels;
int[] previousFrame;
Capture video;

void setup() {
  size(640, 480, P3D); // Change size to 320 x 240 if too slow at 640 x 480
  // Uses the default video input, see the reference if this causes an error
  video = new Capture(this, width, height, 24);
  numPixels = video.width * video.height;
  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();
}

void draw() {
  if (video.available()) {
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available
    
    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
	color currColor = video.pixels[i];
	color prevColor = previousFrame[i];
	// Extract the red, green, and blue components from current pixel
	int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
	int currG = (currColor >> 8) & 0xFF;
	int currB = currColor & 0xFF;
	// Extract red, green, and blue components from previous pixel
	int prevR = (prevColor >> 16) & 0xFF;
	int prevG = (prevColor >> 8) & 0xFF;
	int prevB = prevColor & 0xFF;
	// Compute the difference of the red, green, and blue values
	int diffR = abs(currR - prevR);
	int diffG = abs(currG - prevG);
	int diffB = abs(currB - prevB);
	// Add these differences to the running tally
	movementSum += diffR + diffG + diffB;
	// Render the difference image to the screen
    
    
    //  pixels[i] = color(diffR, diffG, diffB);
    
    
	// The following line is much faster, but more confusing to read
	pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
	// Save the current color into the 'previous' buffer
	previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 0) {
	updatePixels();
	println(movementSum); // Print the total amount of movement to the console
    }
    background(0);
  if (movementSum > 9657343) {
	ellipse(56, 46, 55, 55); // Print the total amount of movement to the console
    }
 // if (movementSum > 7010223) {
  //  rect(100, 100, 10, 10);
   // }
  }
}



Back to top
 
 
Cedric
Re: OPENGL and frame differencing
Reply #8 - Nov 17th, 2009, 2:55pm  
you are posting and posting and i just cant find any problems Smiley
this sketch runs great using P3D and OPENGL. No difference again... What kind of error do you get ?
Back to top
 
... twitter.com/CedricKiefer
WWW CedricKiefer 9570393 CedricKiefer  
JHT83
Re: OPENGL and frame differencing
Reply #9 - Nov 17th, 2009, 3:19pm  
Hi Cedric,

My apologies again as I am posting a lot tonight Smiley

Strangely I don't get any error messages when I try to use OPENGL with the frame differencing sketch. I just get a blank (gray) sketch window.

Perhaps the code for the OPENGL version you did is different to mine. Here is mine incase it is different:

Code:


/**
 * Frame Differencing 
 * by Golan Levin. 
 * 
 * Quantify the amount of movement in the video frame using frame-differencing.
 */ 


import processing.video.*;
import processing.opengl.*;

int numPixels;
int[] previousFrame;
Capture video;

void setup() {
  size(640, 480, OPENGL); // Change size to 320 x 240 if too slow at 640 x 480
  // Uses the default video input, see the reference if this causes an error
  video = new Capture(this, width, height, 24);
  numPixels = video.width * video.height;
  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();
}

void draw() {
  if (video.available()) {
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available
    
    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
	color currColor = video.pixels[i];
	color prevColor = previousFrame[i];
	// Extract the red, green, and blue components from current pixel
	int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
	int currG = (currColor >> 8) & 0xFF;
	int currB = currColor & 0xFF;
	// Extract red, green, and blue components from previous pixel
	int prevR = (prevColor >> 16) & 0xFF;
	int prevG = (prevColor >> 8) & 0xFF;
	int prevB = prevColor & 0xFF;
	// Compute the difference of the red, green, and blue values
	int diffR = abs(currR - prevR);
	int diffG = abs(currG - prevG);
	int diffB = abs(currB - prevB);
	// Add these differences to the running tally
	movementSum += diffR + diffG + diffB;
	// Render the difference image to the screen
    
    
    //  pixels[i] = color(diffR, diffG, diffB);
    
    
	// The following line is much faster, but more confusing to read
	pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
	// Save the current color into the 'previous' buffer
	previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 0) {
	updatePixels();
	println(movementSum); // Print the total amount of movement to the console
    }
    background(0);
  if (movementSum > 9657343) {
	ellipse(56, 46, 55, 55); // Print the total amount of movement to the console
    }
 // if (movementSum > 7010223) {
  //  rect(100, 100, 10, 10);
   // }
  }
}
Back to top
 
 
Cedric
Re: OPENGL and frame differencing
Reply #10 - Nov 17th, 2009, 3:23pm  
nope, works fine here. i get the white dot in the upper left corner as soon as i start moving. the same i got earlier with the P3D sketch.
Cant tell you whats wrong.
Back to top
 
... twitter.com/CedricKiefer
WWW CedricKiefer 9570393 CedricKiefer  
JHT83
Re: OPENGL and frame differencing
Reply #11 - Nov 17th, 2009, 3:25pm  
Hi Cedric,

Thank you very much for trying. I can't work out why it is not working. At least I know that it should work. I will try somebody elses machine tomorrow and see what happens.

Thanks again,
Jamie
Back to top
 
 
jaylfk
Re: OPENGL and frame differencing
Reply #12 - Nov 17th, 2009, 7:55pm  
I had the very same problem and finally found out a working solution for my Macbook Pro, ATI Radeon Mobility 1600xt on Snow Leopard 10.6.2

Add this right after size() in your setup().

Code:
hint(ENABLE_OPENGL_4X_SMOOTH);
Back to top
 
 
JHT83
Re: OPENGL and frame differencing
Reply #13 - Nov 18th, 2009, 1:02am  
Wow, wow!!

Thank you so much Jayflk,

That is really kind of you. And thank you to Cedric too for helping so much. I can't even describe how much of a massive help that is!  Grin
Back to top
 
 
jaylfk
Re: OPENGL and frame differencing
Reply #14 - Nov 18th, 2009, 2:37am  
You're welcome  Wink
Back to top
 
 
Page Index Toggle Pages: 1
‹ Previous Topic | Next Topic ›