We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexDiscussionExhibition › Optical Flow Particles Source Code
Page Index Toggle Pages: 1
Optical Flow Particles Source Code (Read 4437 times)
Optical Flow Particles Source Code
Sep 27th, 2008, 3:23am
 
Hi

Someone asked me to release the source code to a demo I made with Optical Flow and a Particle engine and I thought that it would be an idea to post it here in case it would be useful to anyone.

http://www.andybest.net/2008/09/optical-flow-particles-source-code/

A video of it working can be seen here

http://www.andybest.net/2008/07/particle-engine-optical-flow/
Re: Optical Flow Particles Source Code
Reply #1 - Sep 27th, 2008, 2:10pm
 
thanks a lot. this is great, will definitely have a look at the code Smiley
Re: Optical Flow Particles Source Code
Reply #2 - Sep 30th, 2008, 12:18am
 
This is so awesome, I have been developing a project using the same optical flow technique, which I believe you hinted to in another post. I was heading to the board to post my current progress with it to get feedback, tips, etc. and I find this.. Now I gotta poor over your code and get some better ideas...

Basically the project that I am doing takes a dancers movements to generate sound and manipulate it. I am using three different tracking techniques from three different cameras at different angles(top, bottom , side). The optical flow portion is done from the front, and controls certain dsp effects via osc. Here is my current work on the optical flow (2 weeks in to the project)

*code posted below*

Any suggestions on optimization would be awesome... I would like the movement to happen once it detects movement and move till momentum is gone, ignoring any new detection.

Thank you so much for all the insight you have already given me into this project. Awesome Work!
Re: Optical Flow Particles Source Code
Reply #3 - Sep 30th, 2008, 12:19am
 
Code:

import JMyron.*;


final int NROWS = 50;
final int NCOLS = 50;
final float LIMIT = 20.0;

JMyron m;
Region reg1;
Region reg2;

PImage img1, img2;
int w, h;

void setup() {
 size(320,240);
 m = new JMyron();
 m.start(width,height);
 m.findGlobs(0);
 img1 = createImage(width,height,ARGB);
 img2 = createImage(width,height,ARGB);
 noFill();
 stroke(255,200,0);
 smooth();
 frameRate(15);
 w = 4;
 h = 4;

 reg1 = new Region(0,0,20,20);
 reg2 = new Region(20,20,40,40);

 
 
 m.update();
 m.imageCopy(img1.pixels);
 m.imageCopy(img2.pixels);
 img1.updatePixels();
 img2.updatePixels();
}

void draw() {
 background(0);
 img2.copy(img1,0,0,width,height,
   0,0,width,height);
 img2.updatePixels();
 m.update();
 m.imageCopy(img1.pixels);
 img1.updatePixels();
 image(img1,0,0);
 
 reg1.render();
 reg1.update();
 reg2.render();
 reg2.update();

}

boolean matchCol(color c1, color c2) {
 float d = dist(red(c1),green(c1),blue(c1),
   red(c2),green(c2),blue(c2));
 return (d<LIMIT);
}



void stop() {

 m.stop();

 super.stop();

}

class Region {
             
//  left, top, right, bottom define the region rectangle.
//  xOff, yOff are half the size of the boundary for the neighbours.
//  vx, vy define the x and y components of the velocity.
//  rW, rH are the width and height of the region.

 int left, top, right, bottom;
 int xOff, yOff;
 float vx, vy;
 int rW, rH;
 int centerX, centerY;
 int xdirection = 1;
 int ydirection = 1;

 Region(int _l, int _t, int _r, int _b) {
   left = _l;
   top = _t;
   right = _r;
   bottom = _b;
   xOff = w/2;
   yOff = h/2;
   vx = 0.0;
   vy = 0.0;
   rW = right - left;
   rH = bottom - top;
   centerX = (right/2) + (left/2);
   centerY = (bottom/2) + (top/2);
   xdirection = 1;
   ydirection = 1;
 }

 void update() {

   centerX = (right/2) + (left/2);
   centerY = (bottom/2) + (top/2);
   
   //  Calculate the flow information within the region.

   if (left < 1){
     left += 1;
     right += 1;
   }
   if (top < 1) {
     bottom += 1;
     top += 1;
   }
   for (int r=top;r<bottom;r+=h) {
     for (int c=left;c<right;c+=w) {
       Point p1 = new Point(0,0);
       p1.x = constrain(c,0,img1.width-1);
       p1.y = constrain(r,0,img1.height-1);
       Point p2 = findPoint(p1);
 
// p1 is the pixel in img1 - current image
// p2 is the corresponding point in img2 - previous image
// vx, vy - velocity is computed by the sum of difference
// between the current position and previous position.

       vx += (p1.x-p2.x);
       vy += (p1.y-p2.y);
       drawFlow(p1,p2);
 
     }
   }
   moveRegion();
 }

 Point findPoint(Point _p) {
   Point p = new Point(_p.x,_p.y);
   color col1 = img1.pixels[_p.y*img1.width+_p.x];
   color col2 = img2.pixels[_p.y*img2.width+_p.x];
   if (!matchCol(col1,col2)) {
     float minDiff = 9999999999.0;
     int l = constrain(_p.x-xOff,0,img2.width-1);
     int t = constrain(_p.y-yOff,0,img2.height-1);
     int r = constrain(_p.x+xOff,0,img2.width-1);
     int b = constrain(_p.y+yOff,0,img2.height-1);
     for (int j=t;j<b;j++) {
       for (int i=l;i<r;i++) {
         col2 = img2.pixels[j*img2.width+i];
         float d = dist(red(col1),green(col1),blue(col1),
           red(col2),green(col2),blue(col2));
         if (d<minDiff) {
           minDiff = d;
           p.x = i;
           p.y = j;
         }
       }
     }
   }
   return p;
 }

 void drawFlow(Point p1, Point p2) {
   stroke(0);
   if (p1.x!=p2.x || p1.y!=p2.y) {
     line(p1.x,p1.y,p2.x,p2.y);
     float ang = atan2(p2.y-p1.y,p2.x-p1.x);
     float ln = w/3.0;
     float tx = p1.x + ln*cos(ang-PI/6);
     float ty = p1.y + ln*sin(ang-PI/6);
     line(p1.x,p1.y,tx,ty);
     tx = p1.x + ln*cos(ang+PI/6);
     ty = p1.y + ln*sin(ang+PI/6);
     line(p1.x,p1.y,tx,ty);
   } else {
     line(p1.x,p1.y,p2.x,p2.y);
   }
 }

 void moveRegion() {
   println(vx+ " " +vy);
   if (vx < 1 && vy < 1){
     top += 1;
     bottom += 1;
   }
   left += vx * xdirection;
   right += vx * xdirection;
   top += vy * ydirection;
   bottom += vy * ydirection;

   if (left<0) {
     xdirection *= -1;
     left = 0;
     right = rW;
   }
   if (right>width) {
     xdirection *= -1;
     right = width;
     left = right - rW;
   }
   if (top<0) {
     ydirection *= -1;
     top = 0;
     bottom = rH;
   }
   if (bottom>height) {
     ydirection *= -1;
     bottom = height;
     top = height - rH;
   }
  centerX = (right/2) + (left/2);
  centerY = (bottom/2) + (top/2);
 
   vx = vx*0.482;
   vy = vy*0.482;
   
 }
 
 void render() {
   rectMode(CENTER);
   stroke(255,0,0);
   fill(255,255,0,100);
   pushMatrix();
   translate((left+right)/2,(top+bottom)/2);
   rect(0,0,rW,rH);
 
   popMatrix();
 }
}

class Point {
 int x, y;
 
 Point(int _x, int _y) {
   x = _x;
   y = _y;
 }
}
Re: Optical Flow Particles Source Code
Reply #4 - Oct 5th, 2008, 2:57am
 
Hi psykel,

I'm not sure exactly how you plan to use the optical flow data in context, but I feel that you may be better off perhaps using the data from the entire flow field (i.e. the whole screen) rather than using movable flow regions. This is what I did (crudely) in the particle example.
Re: Optical Flow Particles Source Code
Reply #5 - Oct 7th, 2008, 1:55am
 
well each region is connected via OSC to Max/MSP to control some thing. I noticed that you were doing that and us it, and I find better control, plus using a modified version of your particle system attaching OSC messages to each, which has helped greatly.  I am curious about the flowAlternate that you use in the main ParticleSystem. If I understand correctly you are alternating frames to track because of the flow technique utilizing the previous frame? This all seems to make control a little better, but I want to make sure that this why.

I will post more code shortly, working on incorporating all the tracking techniques into one sketch. I will probably have some threading questions once I get all cameras running into one program.

Once again, thank you for the help you have provided.
Re: Optical Flow Particles Source Code
Reply #6 - Oct 8th, 2008, 4:16am
 
I can't remember for definite (it was a while ago that I did it) but I believe that it was because I wanted to only do detection every other frame so that I could increase the framerate somewhat.
Page Index Toggle Pages: 1