Advice on how to convert sound into camera distortion?

edited October 2016 in Library Questions

I'm working on a program that distorts camera feed using audio input and I was just wondering if people had advice on libraries/concepts/necessary to do this? Or if they'd be willing to share their experience with past programming projects?

here's what I've gotten to so far:

/*
Be Careful with your speaker volume, you might produce a painful 
feedback. We recommend to wear headphones for this example.

*/

import processing.sound.*;
import processing.video.*;

AudioIn input;
Amplitude rms;
Capture cam;

int scale=1;

void setup() {
    size(1200,800);
    background(255);

    //Create an Audio input and grab the 1st channel
    input = new AudioIn(this, 0);

    // start the Audio Input
    input.start();

    // create a new Amplitude analyzer
    rms = new Amplitude(this);

    // Patch the input to an volume analyzer
    rms.input(input);
  String[] cameras = Capture.list();

  if (cameras == null) {
    println("Failed to retrieve the list of available cameras, will try the default...");
    cam = new Capture(this, 640, 480);
  } if (cameras.length == 0) {
    println("There are no cameras available for capture.");
    exit();
  } else {
    println("Available cameras:");
    printArray(cameras);

    cam = new Capture(this, cameras[0]);

    cam.start();

    background(0,0,0);
      noStroke();
  }
}



void draw() {

    input.amp(map(mouseY, 0, height, 0.0, 200.0));

    scale=int(map(rms.analyze(), 0, 0.5, 1, 350));
    noStroke();

      if (cam.available() == true) {
    cam.read();
  }
  image(cam, 0, 0, width, height);

 }

Answers

  • This example plays with the alpha value of the image by using tint. You could modify it for your purpose. Instead of photo, use your "cam" variable:

    import processing.sound.*;
    Amplitude amp;
    AudioIn in;
    float ampt;
    PImage photo;
    
    void setup() {
      size(1200, 800);
      background(255);
    
      // Create an Input stream which is routed into the Amplitude analyzer
      amp = new Amplitude(this);
      in = new AudioIn(this, 0);
      in.start();
      amp.input(in);
      photo = loadImage("fig.jpg");
    }      
    
    void draw() {
      background(255);
      ampt = amp.analyze();
      println(ampt);
    
      float myColor = ampt*256; 
      tint((myColor), (myColor), (myColor));
      image(photo, 0, 0, width, height);
    }
    

    Are you familiar with loadPixel function call?

    https://processing.org/reference/loadPixels_.html https://processing.org/reference/pixels.html

    One thing you could do in the previous draw method:

    void draw() {
      background(255);
      ampt = amp.analyze();
      println(ampt);
    
      image(photo, 0, 0, width, height);
      loadPixels(); 
      int halfImage = width*height/2;
      for (int i = 0; i < halfImage; i++) {
        pixels[i+halfImage] = color(constrain((pixels[i]>>16)*round(random(ampt)*100),0,255) & 0xff,pixels[i]>>8 & 0xff,pixels[i]&0xff);
      }
      updatePixels();
    }
    

    In the line of code of interest, color() accepts three integers based on RGB definitions. I play with the red value. The red value of the half of the image is copied to the other half of the input image. This value is multiplied by a random value related to the amplitude recorded by the mic. Last, I make sure the resulting red value is constrained in the color range definition, between 0 and 255. I hope this helps,

    Kf

  • Awesome it does. Ty!

Sign In or Register to comment.