Avergage Brightness of frames of video

How would I take the average brightness of a video file and whilst its playing back use that brightness level to control another part of the program e.g. the frequency of a sine wave? This is something that I managed to create in Max but I want to recreate it in Processing as I cannot afford to buy max. I know that I would have to add all the values of the pixels and divide by the amount of pixels, but I cannot get it to work. Any help would be greatly appreciated.

Answers

  • post code. without it we can only guess what your problem is.

  • that said, if you are adding pixel values you have to take into account the format of the pixels, ie 8 bits each of A, R, G and B data. you can't just treat them as a single int as that won't make sense.

    read the color tutorials on the processing website.

  • edited February 2016

    here is my attempt at the code, i am new to java and processing so excuse any stupid mistakes

    import processing.sound.*;
    import processing.video.*;
    
    Movie dotsFilm;
    
    void setup() {
      size(320, 180);
      dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");
      loadPixels();
    }
    
    void draw() {
      background(150);
    
      float frameBright = 0;
    
      image(dotsFilm, 0,0, width, height);
      dotsFilm.loop();
    
      for(int i = 0; i < (width*height/2)-width/2; i++) {
        color c = pixels[i];
        float pixelVal = (red(c)+green(c)+blue(c));
        frameBright = pixelVal + frameBright;
      }
      updatePixels(); 
      frameBright = 0;
    }
    
    void movieEvent(Movie m) {
      m.read();
    }
    
  • ok, the summing looks ok but i see no averaging. and i see you doing nothing with the average.

  • when i got to this point the application spotted even opening so I didn't add anything else. saying that, here it is with averaging and a sine wave

    
    import processing.sound.*;
    import processing.video.*;
    
    Movie dotsFilm;
    SinOsc sine;
    
    float freq=400;
    float amp=0.5;
    float frameBright = 0;
    
    void setup() {
     size(320, 180);
     dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");
    
    sine = new SinOsc(this);
    sine.play();
    }
    
    
    void draw() {
      background(150);
      
    loadPixels();
      
      image(dotsFilm, 0,0, width, height);
      dotsFilm.loop();
      
      for(int i = 0; i < (width*height/2)-width/2; i++) {
      color c = pixels[i];
      float pixelVal = ((red(c)+green(c)+blue(c))/3);
      frameBright = pixelVal + frameBright;
      sine.freq(frameBright);
    
    }
    
    frameBright = frameBright/(width * height);
    updatePixels(); 
    frameBright = 0;
    }
    
    void movieEvent(Movie m) {
      m.read();
    }
    
    
  • Answer ✓

    ok, line 32 is a problem...

    it's using the sum of the brightnesses so far.

    you only work out the average on line 36 (which you then never use)

    you can use the average so far by changing that to sine.freq(framebright / i); and it'll change as you progress.

    also, do yourself a favour and use ctrl-t in the editor to indent the code properly because it's a bit messy at the moment.

    when i got to this point the application spotted even opening

    sounds bad. but you don't explain the problem any further so...

  • thank you so much for your help, i'm new to this forum and processing in general and appreciate the help.

  • I have been out of action the past couple of days and haven't had much progress, this is where I am. When I open the program I here a high pitch beep and then it changes to a low pitch beep that plays indefinatly

    
    import processing.sound.*;
    import processing.video.*;
    
    Movie dotsFilm;
    SinOsc sine;
    
    float freq=400;
    float amp=0.5;
    float frameBright = 0;
    
    void setup() {
    
      size(320, 180);
      dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/Grad Film/test.mp4");
      sine = new SinOsc(this);
      sine.play();
    }
    
    
    void draw() {
    
      loadPixels();
    
      image(dotsFilm, 0, 0, width, height);
    
      for (int i = 0; i < (width*height/2)-width/2; i++) {
        color c = pixels[i];
        float pixelVal = ((red(c)+green(c)+blue(c))/3);
        frameBright = pixelVal + frameBright;
        updatePixels();
      }
    
      frameBright = (frameBright / (width*height));
      sine.freq(frameBright);
    
      frameBright = 0;
      dotsFilm.loop();
    }
    
    void movieEvent(Movie m) {
      m.read();
    }
    
    
  • println(frameBright); after line 33

    When I open the program I here a high pitch beep and then it changes to a low pitch beep that plays indefinatly

    what are you expecting?

    and what's that on line 26? i < (width*height/2)-width/2 ???

  • ok I have finally got it to work, I had a few days away from it and realised I made some really dumb mistakes! Thanks for your help again

    import processing.sound.*;
    import processing.video.*;
    
    Movie dotsFilm;
    SinOsc sine;
    
    float freq=1000;
    float amp=0.5;
    float frameBright = 0;
    
    void setup() {
    
      size(1280, 720);
      dotsFilm = new Movie(this, "/Users/samuelhailey/Movies/merriweather.mov");
      sine = new SinOsc(this);
      sine.play();
    }
    
    
    void draw() {
      background(150);
    
    
      image(dotsFilm, 0, 0, width, height);
      loadPixels();
      for (int i = 0; i < pixels.length; i++) {
        color c = pixels[i];
        float pixelVal = ((red(c)+green(c)+blue(c))/3);
        frameBright = pixelVal + frameBright;
      }
    
      frameBright = (frameBright / (width*height));
    
      sine.freq((frameBright * 1.5));
    
      frameBright = 0;
      dotsFilm.loop();
    }
    
    void movieEvent(Movie m) {
      m.read();
    }
    
Sign In or Register to comment.