Avergage Brightness of frames of video

samsamhailey

How would I take the average brightness of a video file and whilst its playing back use that brightness level to control another part of the program e.g. the frequency of a sine wave? This is something that I managed to create in Max but I want to recreate it in Processing as I cannot afford to buy max. I know that I would have to add all the values of the pixels and divide by the amount of pixels, but I cannot get it to work. Any help would be greatly appreciated.

GoToLoop

https://forum.Processing.org/two/discussion/15050/errors-when-cycling-through-pixels

koogs

post code. without it we can only guess what your problem is.

koogs

that said, if you are adding pixel values you have to take into account the format of the pixels, ie 8 bits each of A, R, G and B data. you can't just treat them as a single int as that won't make sense.

read the color tutorials on the processing website.

samsamhailey

here is my attempt at the code, i am new to java and processing so excuse any stupid mistakes

import processing.sound.*;
import processing.video.*;

Movie dotsFilm;

void setup() {
  size(320, 180);
  dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");
  loadPixels();
}

void draw() {
  background(150);

  float frameBright = 0;

  image(dotsFilm, 0,0, width, height);
  dotsFilm.loop();

  for(int i = 0; i < (width*height/2)-width/2; i++) {
    color c = pixels[i];
    float pixelVal = (red(c)+green(c)+blue(c));
    frameBright = pixelVal + frameBright;
  }
  updatePixels(); 
  frameBright = 0;
}

void movieEvent(Movie m) {
  m.read();
}

koogs

ok, the summing looks ok but i see no averaging. and i see you doing nothing with the average.

samsamhailey

when i got to this point the application spotted even opening so I didn't add anything else. saying that, here it is with averaging and a sine wave

import processing.sound.*;
import processing.video.*;

Movie dotsFilm;
SinOsc sine;

float freq=400;
float amp=0.5;
float frameBright = 0;

void setup() {
 size(320, 180);
 dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");

sine = new SinOsc(this);
sine.play();
}


void draw() {
  background(150);
  
loadPixels();
  
  image(dotsFilm, 0,0, width, height);
  dotsFilm.loop();
  
  for(int i = 0; i < (width*height/2)-width/2; i++) {
  color c = pixels[i];
  float pixelVal = ((red(c)+green(c)+blue(c))/3);
  frameBright = pixelVal + frameBright;
  sine.freq(frameBright);

}

frameBright = frameBright/(width * height);
updatePixels(); 
frameBright = 0;
}

void movieEvent(Movie m) {
  m.read();
}

koogs

ok, line 32 is a problem...

it's using the sum of the brightnesses so far.

you only work out the average on line 36 (which you then never use)

you can use the average so far by changing that to sine.freq(framebright / i); and it'll change as you progress.

also, do yourself a favour and use ctrl-t in the editor to indent the code properly because it's a bit messy at the moment.

when i got to this point the application spotted even opening

sounds bad. but you don't explain the problem any further so...

samsamhailey

thank you so much for your help, i'm new to this forum and processing in general and appreciate the help.

samsamhailey

I have been out of action the past couple of days and haven't had much progress, this is where I am. When I open the program I here a high pitch beep and then it changes to a low pitch beep that plays indefinatly

import processing.sound.*;
import processing.video.*;

Movie dotsFilm;
SinOsc sine;

float freq=400;
float amp=0.5;
float frameBright = 0;

void setup() {

  size(320, 180);
  dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");
  sine = new SinOsc(this);
  sine.play();
}


void draw() {

  loadPixels();

  image(dotsFilm, 0, 0, width, height);

  for (int i = 0; i < (width*height/2)-width/2; i++) {
    color c = pixels[i];
    float pixelVal = ((red(c)+green(c)+blue(c))/3);
    frameBright = pixelVal + frameBright;
    updatePixels();
  }

  frameBright = (frameBright / (width*height));
  sine.freq(frameBright);

  frameBright = 0;
  dotsFilm.loop();
}

void movieEvent(Movie m) {
  m.read();
}

koogs

println(frameBright); after line 33

When I open the program I here a high pitch beep and then it changes to a low pitch beep that plays indefinatly

what are you expecting?

and what's that on line 26? i < (width*height/2)-width/2 ???

samsamhailey

ok I have finally got it to work, I had a few days away from it and realised I made some really dumb mistakes! Thanks for your help again

import processing.sound.*;
import processing.video.*;

Movie dotsFilm;
SinOsc sine;

float freq=1000;
float amp=0.5;
float frameBright = 0;

void setup() {

  size(1280, 720);
  dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/merriweather.mov");
  sine = new SinOsc(this);
  sine.play();
}


void draw() {
  background(150);


  image(dotsFilm, 0, 0, width, height);
  loadPixels();
  for (int i = 0; i < pixels.length; i++) {
    color c = pixels[i];
    float pixelVal = ((red(c)+green(c)+blue(c))/3);
    frameBright = pixelVal + frameBright;
  }

  frameBright = (frameBright / (width*height));

  sine.freq((frameBright * 1.5));

  frameBright = 0;
  dotsFilm.loop();
}

void movieEvent(Movie m) {
  m.read();
}