Blank/Grey screen (frame differencing)

Luke12345

My code seems like a little bit buggy. Sometimes, i need to refresh few times to get the ideal results. This is very annoying and you need to keep on restart the processing and execute the code again. And there is no message in console. 3 problems happen in my code:

a) Grey screen (can hear video sound)
b) Normal video with sound (No frame differencing function)
c) Frame differencing + video (ideal)

import processing.video.*;

int numPixels;
int[] previousFrame;
Movie video;

void setup() {
  size(1280, 770);

  video = new Movie(this, "human.mp4");

  // Start capturing the images from the camera
   video.loop(); 
   video.read(); 
  numPixels = video.width * video.height;
  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();

}

void draw() {

  if (video.available()) {
 image(video, 0, 0);
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available

    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
      color currColor = video.pixels[i];
      color prevColor = previousFrame[i];
      // Extract the red, green, and blue components from current pixel
      int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
      int currG = (currColor >> 8) & 0xFF;
      int currB = currColor & 0xFF;
      // Extract red, green, and blue components from previous pixel
      int prevR = (prevColor >> 16) & 0xFF;
      int prevG = (prevColor >> 8) & 0xFF;
      int prevB = prevColor & 0xFF;
      // Compute the difference of the red, green, and blue values
      int diffR = abs(currR - prevR);
      int diffG = abs(currG - prevG);
      int diffB = abs(currB - prevB);
      // Add these differences to the running tally
      movementSum += diffR + diffG + diffB;
      // Render the difference image to the screen
      pixels[i] = color(diffR, diffG, diffB);
      // The following line is much faster, but more confusing to read
      //pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
      // Save the current color into the 'previous' buffer
      previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 0) {
      updatePixels();
      println(movementSum); // Print the total amount of movement to the console
    }
  }
}

CrimsonCrow98

Maybe by using another renderer? The default renderer isn't really suited for a lot of frame operations.. Maybe try size(1280, 770, P2D); or size(1280, 770, FX2D); ?

kfrajer

I believe when he calls size with only two arguments, the system is using P2D rendering as default. Did you tr running examples available in the video library? Do you get the same behavior?

Kf

Luke12345

@CrimsonCrow98 Same problem. Nothing happens. Could you help me debug my code? Appreciate :) @kfrajer i did. It works fine.

GoToLoop

println(getGraphics());

Luke12345

@GoToLoop I tried with P2D, P3D, JAVA2D, and FX2D (Lag) . Still the same. Random results. It is either grey screen or normal video. Ideal result is very odd.

GoToLoop

@Luke12345, I was replying to @kfrajer's assertive that size()'s default renderer is P2D. 8-|

Luke12345

No solution for this? Anyone? :(

koogs

do you need to call loadPixels()?

you are doing it for the camera but not for the screen and the reference says

"This function must always be called before reading from or writing to pixels[]."

https://processing.org/reference/loadPixels_.html

(you are calling it in setup(), but that only runs once. try moving it to draw())

candanerim

Hello, I met the same error with the same code. How did you fix it? My code gives the sounds of my video with the just gray screen or just the video without FrameDifferencing. I've tried with example videos on the processing library and it works fine but not my videos.

candanerim

UpdatePixels(); is working for me. The code was working with the example code in the library and also footage from my webcam. So I fixed the problem by using the Magic Camera software. It helps to make my videos like they are coming from my webcam.

But I have another question about this code. I want to visualize as a line graph of this frameDifference outputs. I tried the code below but there is a problem and do you have any idea how can I make a line graph with this output or how can I fix the code below?

ArrayList<Points> poop = new ArrayList();
import processing.video.*;

int numPixels;
int[] previousFrame;
Capture video;

void setup() {
  size(320, 240);
  video= new Capture(this, width, height, "MagicCamera Device" );
  // This the default video input, see the GettingStartedCapture 
  // example if it creates an error
  // video = new Capture(this, width, height);

  // Start capturing the images from the camera
  video.start(); 

  numPixels = video.width * video.height;

  // Create an array to store the previously captured frame
  previousFrame = new int[numPixels];
  loadPixels();

}

void draw() {
  if (video.available()) {
    // When using video to manipulate the screen, use video.available() and
    // video.read() inside the draw() method so that it's safe to draw to the screen
    video.read(); // Read the new frame from the camera
    video.loadPixels(); // Make its pixels[] array available

    int movementSum = 0; // Amount of movement in the frame
    for (int i = 0; i < numPixels; i++) { // For each pixel in the video frame...
      color currColor = video.pixels[i];
      color prevColor = previousFrame[i];
      // Extract the red, green, and blue components from current pixel
      int currR = (currColor >> 16) & 0xFF; // Like red(), but faster
      int currG = (currColor >> 8) & 0xFF;
      int currB = currColor & 0xFF;
      // Extract red, green, and blue components from previous pixel
      int prevR = (prevColor >> 16) & 0xFF;
      int prevG = (prevColor >> 8) & 0xFF;
      int prevB = prevColor & 0xFF;
      // Compute the difference of the red, green, and blue values
      int diffR = abs(currR - prevR);
      int diffG = abs(currG - prevG);
      int diffB = abs(currB - prevB);
      // Add these differences to the running tally
      movementSum += diffR + diffG + diffB;
      // Render the difference image to the screen
      pixels[i] = color(diffR, diffG, diffB);
      // The following line is much faster, but more confusing to read
      //pixels[i] = 0xff000000 | (diffR << 16) | (diffG << 8) | diffB;
      // Save the current color into the 'previous' buffer
      previousFrame[i] = currColor;
    }
    // To prevent flicker from frames that are all black (no movement),
    // only update the screen if the image has changed.
    if (movementSum > 50) {

      updatePixels();

      float t = movementSum/2000;
  Points P = new Points(width, t );
  poop.add(P);

      println(movementSum); // Print the total amount of movement to the console

    }
  }
      beginShape();
  for (int i=0;i<poop.size();i++) {
    Points P = (Points)poop.get(i);
    //vertex(P.x, P.y);
    if (P.x<0)poop.remove(i);
    P.x--;
  }
  endShape(CLOSE);    
}
class Points {
  float x, y;
  Points(float x, float y) {
    this.x = x;
    this.y = y;
  }
}

koogs

moved here: https://forum.processing.org/two/discussion/22041/making-a-graph-with-frame-differencing-outputs