Problem with motion capture using movieEvent();

Gildash

Hey guys, so I'm writing up a basic motion detection program that looks at changes in colors between one frame and the next. I got it to work perfectly with a live stream from my computer's webcam, but when I import a video and use movieEvent() instead of captureEvent(), I get a null entry when trying to compare the second image of the imported movie to the first one. I'm not sure what the problem is considering I can do it with a live feed vs. an imported movie. It would make sense that it gets to the end of the movie and no longer has a next frame to compare, but i've set it on loop and the error occurs at the very beginning, not the end. Anyways, everything works perfectly with a live video feed so I'm pretty confused. Any help would be appreciated, thanks guys!:

  import processing.video.*;


  Movie myMovie;
  color trackColor;
  PImage prevFrame;
  // How different must a pixel be to be a "waggle" pixel
  float frequency = 150;
  float prevX = 0;
  float prevY = 0;  

  float  avgX = 0;
  float  avgY = 0;


  void setup() {
    size(640, 360);
    myMovie = new Movie(this, "C:\\Users\\Michael\\Downloads\\bee dance 2 24.06.15.MTS");

    myMovie.loop();
    // Create an empty image the same size as the video

    prevFrame = createImage(myMovie.width, myMovie.height, RGB);
    // Start off tracking for black

    trackColor = color(0);

  }

  void movieEvent(Movie myMovie) {

      // Save previous frame for motion detection!!


    prevFrame.copy(myMovie, 0, 0, myMovie.width, myMovie.height, 0, 0, myMovie.width, myMovie.height); 

  //Before we read the new frame, we always save the previous frame for comparison!


    prevFrame.updatePixels();  // Read image from the movie


    myMovie.read();


  }

  void draw() {

    image(myMovie, 0, 0, width, height);
    loadPixels();
    myMovie.loadPixels();
    prevFrame.loadPixels();
      // These are the variables we'll need to find the average X and Y
  float sumX = 0;
  float sumY = 0;
  int motionCount = 0; 
  float ravg = 0; //average colors of the detected movement
  float gavg = 0;
  float bavg = 0;

    // Begin loop to walk through every pixel    

    for (int x = 0; x < myMovie.width; x ++ ) {
      for (int y = 0; y < myMovie.height; y ++ ) {

        int loc = x + y*myMovie.width;            // Step 1, what is the 1D pixel location
        color current = myMovie.pixels[loc];      // Step 2, what is the current color
  color previous = prevFrame.pixels[loc];   // Step 3, what is the previous color
  int currentx = x;
  int currenty = y; 
  int previousx = x-1; 
  int previousy = y-1;

  // Step 4, compare colors (previous vs. current)
  float r1 = red(current); 
  float g1 = green(current); 
  float b1 = blue(current);
  float r2 = red(previous); 
  float g2 = green(previous); 
  float b2 = blue(previous);
  float diff = dist(r1, g1, b1, r2, g2, b2);


  // If the color at that pixel has changed, then there is motion at that pixel.
  if (diff > frequency) { 
    // If motion, display black
    pixels[loc] = color(0);
    sumX += x;
    sumY += y;
    motionCount++;

    ravg = ravg + r1;
    gavg = gavg + g1;
    bavg = bavg + b1;

  } else {
    // If not, display original colors
    pixels[loc] = color(r1,g1,b1);
  }
}
    }

  }

The code above doesn't work, however the following code using a live stream does:

  import processing.video.*;
  // Variable for capture device
  Capture video;
  // Previous Frame
  PImage prevFrame;
  // How different must a pixel be to be a "motion" pixel
  float threshold = 50;

  void setup() {
    size(320, 240);
    video = new Capture(this, width, height, 30);
    video.start();
    // Create an empty image the same size as the video
    prevFrame = createImage(video.width, video.height, RGB);
  }

  void captureEvent(Capture video) {
    // Save previous frame for motion detection!!
    prevFrame.copy(video, 0, 0, video.width, video.height, 0, 0, video.width, video.height); // Before we read the new frame, we always save the previous frame for comparison!
    prevFrame.updatePixels();  // Read image from the camera
    video.read();
  }

  void draw() {

    loadPixels();
    video.loadPixels();
    prevFrame.loadPixels();

    // Begin loop to walk through every pixel
    for (int x = 0; x < video.width; x ++ ) {
      for (int y = 0; y < video.height; y ++ ) {

  int loc = x + y*video.width;            // Step 1, what is the 1D pixel location
  color current = video.pixels[loc];      // Step 2, what is the current color
  color previous = prevFrame.pixels[loc]; // Step 3, what is the previous color

  // Step 4, compare colors (previous vs. current)
  float r1 = red(current); 
  float g1 = green(current); 
  float b1 = blue(current);
  float r2 = red(previous); 
  float g2 = green(previous); 
  float b2 = blue(previous);
  float diff = dist(r1, g1, b1, r2, g2, b2);

  // Step 5, How different are the colors?
  // If the color at that pixel has changed, then there is motion at that pixel.
  if (diff > threshold) { 
    // If motion, display black
    pixels[loc] = color(0);
  } else {
    // If not, display white
    pixels[loc] = color(255);
  }
}
    }
    updatePixels();
  }

akenaton

@Gildash===

when you create prevFrame with Capture the size (width, height) is known; when you do the same (in setup()) with a movie the movie size is not known. solution 1:: (code below)= you "hardcode" the size for prevFrame according to the size of the movie. solution 2:: you create a method && some boolean and createImage() as soon as the movie size is known. Note that the black pixels are seen (human eye) only when the movie images are very different.

code for 1==

     import processing.video.*;

      Movie myMovie;
      color trackColor;
      PImage prevFrame;
      // How different must a pixel be to be a "waggle" pixel
      float frequency = 10;//i exagerate the threshold
      float prevX = 0;
      float prevY = 0;  

      float  avgX = 0;
      float  avgY = 0;


      void setup() {
        size(800, 600);
        myMovie = new Movie(this, "station.mov", 10);//movie rate = 10 in order that //the blackpixels can be seen
      myMovie.play();
        myMovie.loop();

        // Create an empty image the same size as the video

        prevFrame = createImage(160, 120, RGB);//hardcoding!!!

//with your code:: size = 1; you can verify:: println(PrevFrame.width);

        // Start off tracking for black

        trackColor = color(0);

      }

      void movieEvent(Movie myMovie) {




        myMovie.read();


      }

      void draw() {

     //be sure that the movie is really ready

     if (myMovie.available() && myMovie.width>1 && myMovie.height>1) {
        // Save previous frame for motion detection!!
        image(myMovie, 0, 0, myMovie.width,myMovie.height);
        prevFrame.copy(myMovie, 0, 0, myMovie.width, myMovie.height, 0, 0, myMovie.width, myMovie.height); 
        prevFrame.updatePixels();



      }else{

        return;
      }
        //loadPixels();// this is not useful

        myMovie.loadPixels();
        prevFrame.loadPixels();
          // These are the variables we'll need to find the average X and Y
      float sumX = 0;
      float sumY = 0;
      int motionCount = 0; 
      float ravg = 0; //average colors of the detected movement
      float gavg = 0;
      float bavg = 0;

        // Begin loop to walk through every pixel    

        for (int x = 0; x < myMovie.width; x ++ ) {
          for (int y = 0; y < myMovie.height; y ++ ) {

            int loc = x + y*myMovie.width;            // Step 1, what is the 1D pixel location
            color current = myMovie.pixels[loc];      // Step 2, what is the current color
      color previous = prevFrame.pixels[loc];   // Step 3, what is the previous color
      int currentx = x;
      int currenty = y; 
      int previousx = x-1; 
      int previousy = y-1;

      // Step 4, compare colors (previous vs. current)
      float r1 = red(current); 
      float g1 = green(current); 
      float b1 = blue(current);
      float r2 = red(previous); 
      float g2 = green(previous); 
      float b2 = blue(previous);
      float diff = dist(r1, g1, b1, r2, g2, b2);


      // If the color at that pixel has changed, then there is motion at that pixel.
      if (diff > frequency) { 
        // If motion, display black
        myMovie.pixels[loc] = color(0);
        sumX += x;
        sumY += y;
        motionCount++;

        ravg = ravg + r1;
        gavg = gavg + g1;
        bavg = bavg + b1;

      } else {
        // If not, display original colors
        myMovie.pixels[loc] = color(r1,g1,b1);
      }
    }
        }
       myMovie.updatePixels();//added

       prevFrame.updatePixels();//added

}

      }

Gildash

First of all, thanks so much for your help! So I've placed your code (and replaced the movie with my own), and two problems showed up:

1:

myMovie = new Movie(this, "station.mov", 10);

Placing the frame rate within this isn't allowed for some reason, although I can manually set the frame rate if it comes to that. It returns the error: "The Constructor Movie(FileName, String, int) is Undefined". Instead the only thing allowed is:

myMovie = new Movie(this, "station.mov");

2: When I run your program, I get the following error: "ArrayIndexOutOfBoundsException: 19200" on the following line of code:

color previous = prevFrame.pixels[loc];   // Step 3, what is the previous color

This was also the line of code giving me the error before, except that it had "0" instead of 19200, which is what 160*120 is in the hardwired width and height that we declare.

Below is my new code with your changes implemented, maybe I've missed something?:

import processing.video.*;

Movie myMovie;
color trackColor;
PImage prevFrame;
// How different must a pixel be to be a "waggle" pixel
float frequency = 150;
float prevX = 0;
float prevY = 0;  

float  avgX = 0;
float  avgY = 0;

void setup() {
  size(800, 600);
  myMovie = new Movie(this, "C:\\Users\\Michael\\Downloads\\bee dance 2 24.06.15.MTS");
  myMovie.play();
  // Create an empty image the same size as the video
  prevFrame = createImage(160, 120, RGB);

}
  void movieEvent(Movie myMovie) {
    myMovie.read();
  }

  void draw() {

 //be sure that the movie is really ready

 if (myMovie.available() && myMovie.width>1 && myMovie.height>1) {
    // Save previous frame for motion detection!!
    image(myMovie, 0, 0, myMovie.width,myMovie.height);
    prevFrame.copy(myMovie, 0, 0, myMovie.width, myMovie.height, 0, 0, myMovie.width, myMovie.height); 
    prevFrame.updatePixels();



  }else{

    return;
  }
//loadPixels();// this is not useful

myMovie.loadPixels();
prevFrame.loadPixels();
      // These are the variables we'll need to find the average X and Y
  float sumX = 0;
  float sumY = 0;
  int motionCount = 0; 
  float ravg = 0; //average colors of the detected movement
  float gavg = 0;
  float bavg = 0;

    // Begin loop to walk through every pixel    

    for (int x = 0; x < myMovie.width; x ++ ) {
      for (int y = 0; y < myMovie.height; y ++ ) {

        int loc = x + y*myMovie.width;            // Step 1, what is the 1D pixel location
        color current = myMovie.pixels[loc];      // Step 2, what is the current color
  color previous = prevFrame.pixels[loc];   // Step 3, what is the previous color
  int currentx = x;
  int currenty = y; 
  int previousx = x-1; 
  int previousy = y-1;

  // Step 4, compare colors (previous vs. current)
  float r1 = red(current); 
  float g1 = green(current); 
  float b1 = blue(current);
  float r2 = red(previous); 
  float g2 = green(previous); 
  float b2 = blue(previous);
  float diff = dist(r1, g1, b1, r2, g2, b2);


  // If the color at that pixel has changed, then there is motion at that pixel.
  if (diff > frequency) { 
    // If motion, display black
    myMovie.pixels[loc] = color(0);
    sumX += x;
    sumY += y;
    motionCount++;

    ravg = ravg + r1;
    gavg = gavg + g1;
    bavg = bavg + b1;

  } else {
// If not, display original colors
    myMovie.pixels[loc] = color(r1,g1,b1);
  }
}
    }
   myMovie.updatePixels();//added

   prevFrame.updatePixels();//added
  }

akenaton

@Gildash== 1)i have used p5 1.5.1== Constructor

Movie(parent, filename) Movie(parent, filename, fps) Movie(parent, url) Movie(parent, url, fps)

probably they changed that in P5 2...

2) of course 160/120== size of station.mov is it the ize of your movie "dance...." if not you have to put here width & height from your movie

Gildash

It works! Thanks!