Making sound from an Image. But some problems!

abeythomas

Hello!

I wrote this code to make sounds according to the pixel colour value. I also want the pixels to be drawn simultaneously in sync with the sound.

Please help me as I'm pretty new to Processing

Link to the picture used http://theblackliberalboomer.com/wp-content/uploads/2013/01/President-Obama-Official-Pic-2013.jpg

PImage obama;

import ddf.minim.*;
import ddf.minim.ugens.*;

Minim minim;
AudioOutput out;
Oscil wave;

void setup(){

  size(612,612);
  obama = loadImage("obama.jpg");

  minim = new Minim(this);
  out = minim.getLineOut();
  wave = new Oscil( 440, 0.5f, Waves.SINE );
  wave.patch( out );
  noLoop();
}

void draw(){

  obama.loadPixels();
  //image(obama,0,0);


for(int x = 0; x<612; x++){
    for(int y=0; y<612; y++){

      int index = x+y*612;
      color c  = obama.pixels[index];
      //stroke(c);

      float amp = map( c, 0, 255, 1, 0 );
      wave.setAmplitude( amp );

      float freq = map( c, 0, 255, 110, 880 );
      wave.setFrequency( freq );

      stroke(c);
      point(x,y);
      //delay(500);


    }
  }
}

prince_polka

Cool stuff, I think your problem has this simple solution.
Move the looping variables outside of draw,
loop through a portion of them every frame (speed),
and remove noLoop.

PImage obama;
    int x=0;
    int y=0;
    int speed=2000;
    import ddf.minim.*;
    import ddf.minim.ugens.*;

    Minim minim;
    AudioOutput out;
    Oscil wave;

    void setup(){
      size(612,612);
      obama = loadImage("obama.jpg");
      minim = new Minim(this);
      out = minim.getLineOut();
      wave = new Oscil( 440, 0.5f, Waves.SINE );
      wave.patch( out );
      obama.loadPixels();
    }

    void draw(){
      for(int s = 0; s<speed; s++){
      x=(x+1)%612;if(x==611){y++;}
      if(y>612){exit();}
          // int index = x+y*612;
          int index = min(x+y*612,(612*612)-1); // this line might not be needed just got an out of bound error
          color c  = obama.pixels[index];
          float amp = map( c, 0, 255, 1, 0 );
          wave.setAmplitude( amp );
          float freq = map( c, 0, 255, 110, 880 );
          wave.setFrequency( freq );
        stroke(c);
          point(x,y);
      }
}  

Hope this helps

abeythomas

Hi prince, Thanks a lot! This works perfect. Even though I couldn't get the whole idea/logic completely in the start of the draw(). Sorry to bother. I'm new to processing and have been trusting on this one 'for' loop for all my codes :)

for(int s = 0; s<speed; s++){
      x=(x+1)%612;

      if(x==611){
          y++;
      }
      if(y>612){
        exit();
      }

Abey

abeythomas

Also, if add strokeWeight(3); towards the end it doesn't work. or instead a small rectangle instead of the point(). Any idea?

kfrajer

Two suggestions.

1. I would keep the amplitude fix in order to observe solely the effect of the frequency based on color. Otherwise, lower frequencies will not be played loud ever.

2. @prince_polka Related to mapping color to amplitude (or frequency), one needs to remember color ranges from 0x0 to 0xfffff in RGB mode (there is also the alpha channel). The mapping in the above example is limited to the range of 0 to 255 (aka 0x0 to 0xff). You could change it by setting the frequency based on a single primary color (either red,blue,green as in red(obama.pixels[index]) or you do the mapping across the full range or you do the mapping using hue() function and the HSB mode (check colorMode in the reference).

Kf

prince_polka

@ krajfer, color is a bit tricky,
Transparent colors goes from 0x0 to 0xffffff,
If it's opaque then black is #ff000000 aka 0xFF<<24 aka -16777216
And white is #ffffffff
Only 6digits is allowed with the hashtag, but you get my point

colorMode(RGB);
color c = color(0);
println(c); // -16777216
color c = color(0,0,0,0);
println(c); // 0

Unsure whether it has any significance in this color to sound case, but it's something to keep in mind.
https://processing.org/reference/colorMode_.html
[https://processing.org/reference/map_.html(https://processing.org/reference/map_.html)

xxMrPHDxx

This hurts my brain...

import ddf.minim.ugens.*;
import ddf.minim.*;

Minim minim;
AudioOutput out;
Oscil wave;

PImage flowers;

ImageTrack it;

void setup() {
  size(600, 600);
  flowers = loadImage("whatever.jpg");

  minim = new Minim(this);
  out = minim.getLineOut();
  wave = new Oscil(440, 0.5f, Waves.SINE);
  wave.patch(out);

  it = new ImageTrack(flowers);
}

void draw() {
  it.play();
}

class ImageTrack {
  PImage image;
  int[] c;
  int current = 0;

  boolean stop = false;

  ImageTrack(PImage image) {
    this.image = image;
    c = image.pixels; 
    image.loadPixels();
  }

  void play() {
    if (stop)return;

    color col = c[current];

    float amp = map(col, 0, 255, 1, 0);
    wave.setAmplitude(amp);

    float freq = map(col, 0, 255, 300, 1000);
    wave.setFrequency(freq);

    stroke(col);
    point(current % image.width, current / image.width);

    current++;
    if (current >= c.length) {
      stop = true;
      stop();
    }
  }

  void stop() {
    wave.setAmplitude(0);
  }
}

kfrajer

Sounds like if the sound was generated by a quantum computer. Neat! @xxMrPHDxx

Kf

abeythomas

Thanks a lot guys! Since I'm new to processing, I'm gonna take some days to understand the whole code. But never mind, I love these!

xxMrPHDxx

Sounds like if the sound was generated by a quantum computer.

Yeah. It depends on the image you are using :D @kfrajer

Thanks a lot guys!

No problem @abeythomas