Advice on how to convert sound into camera distortion?

kitschen

I'm working on a program that distorts camera feed using audio input and I was just wondering if people had advice on libraries/concepts/necessary to do this? Or if they'd be willing to share their experience with past programming projects?

here's what I've gotten to so far:

/*
Be Careful with your speaker volume, you might produce a painful 
feedback. We recommend to wear headphones for this example.

*/

import processing.sound.*;
import processing.video.*;

AudioIn input;
Amplitude rms;
Capture cam;

int scale=1;

void setup() {
    size(1200,800);
    background(255);

    //Create an Audio input and grab the 1st channel
    input = new AudioIn(this, 0);

    // start the Audio Input
    input.start();

    // create a new Amplitude analyzer
    rms = new Amplitude(this);

    // Patch the input to an volume analyzer
    rms.input(input);
  String[] cameras = Capture.list();

  if (cameras == null) {
    println("Failed to retrieve the list of available cameras, will try the default...");
    cam = new Capture(this, 640, 480);
  } if (cameras.length == 0) {
    println("There are no cameras available for capture.");
    exit();
  } else {
    println("Available cameras:");
    printArray(cameras);

    cam = new Capture(this, cameras[0]);

    cam.start();

    background(0,0,0);
      noStroke();
  }
}



void draw() {

    input.amp(map(mouseY, 0, height, 0.0, 200.0));

    scale=int(map(rms.analyze(), 0, 0.5, 1, 350));
    noStroke();

      if (cam.available() == true) {
    cam.read();
  }
  image(cam, 0, 0, width, height);

 }

kfrajer

This example plays with the alpha value of the image by using tint. You could modify it for your purpose. Instead of photo, use your "cam" variable:

import processing.sound.*;
Amplitude amp;
AudioIn in;
float ampt;
PImage photo;

void setup() {
  size(1200, 800);
  background(255);

  // Create an Input stream which is routed into the Amplitude analyzer
  amp = new Amplitude(this);
  in = new AudioIn(this, 0);
  in.start();
  amp.input(in);
  photo = loadImage("fig.jpg");
}      

void draw() {
  background(255);
  ampt = amp.analyze();
  println(ampt);

  float myColor = ampt*256; 
  tint((myColor), (myColor), (myColor));
  image(photo, 0, 0, width, height);
}

Are you familiar with loadPixel function call?

https://processing.org/reference/loadPixels_.html https://processing.org/reference/pixels.html

One thing you could do in the previous draw method:

void draw() {
  background(255);
  ampt = amp.analyze();
  println(ampt);

  image(photo, 0, 0, width, height);
  loadPixels(); 
  int halfImage = width*height/2;
  for (int i = 0; i < halfImage; i++) {
    pixels[i+halfImage] = color(constrain((pixels[i]>>16)*round(random(ampt)*100),0,255) & 0xff,pixels[i]>>8 & 0xff,pixels[i]&0xff);
  }
  updatePixels();
}

In the line of code of interest, color() accepts three integers based on RGB definitions. I play with the red value. The red value of the half of the image is copied to the other half of the input image. This value is multiplied by a random value related to the amplitude recorded by the mic. Last, I make sure the resulting red value is constrained in the color range definition, between 0 and 255. I hope this helps,

Kf

kitschen

Awesome it does. Ty!