Visualising sound

General Discussion Other

ollena

in General Discussion • Other • 1 year ago

Hi Guys,

here I am asking for some programming help again.
I am visualising sound again.

Lovely Chrisir did already help me on this subject see here:
http://forum.processing.org/topic/please-help-to-change-the-existing-coding-so-that-it-visualises-colour#25080000001576085

This is an amaizing visualising tool, but now I want to achieve accurate visual of every second of the sound.
And I have no Idea how to do that (I'm still very much a beginner in Processing)
My tutor told was going to help me with that but he is obviously too busy now, and also I'm eager to surprise him with any result I can ;)

But what he said was that an easier solution would be to just examine the buffer as the sound is playing, which would mean that to make the image would take 90 minutes which is absolutely fine for me!

If I may ask for your help in programming this process it would be amazing!

So here is the short outline of what I'm trying to achieve:

I have sound files *.mp3 they are very long soundtracks: 90-120 minutes.
what I need to do is to convert sound intensity (volume) into grayscale colour to visualize each second of the soundtrack.

for example if a soundtrack is 90 minutes = 5400 sec
than the artwork is a square of 5400 frames (sides: 73x73 frames)

(I make each frame 9x9 pixels)

every frame represents the volume of the soundtrack by grayscale colour:
the louder the sound the lighter the frame.
So silence will create black frames, extremely loud will appear white.

If my description needs more clarity, please let me know.

Please guys, let me know, if it is at all possible for you to help me with this project?
I will really appreciate, if it is.
Thanks

And yes, the final image should look something like this:

Replies(16)

Chrisir

Re: Visualising sound

1 year ago

hi,

I guess my Code is a good starting point for this:

https://forum.processing.org/topic/please-help-to-change-the-existing-coding-so-that-it-visualises-colour#25080000001571295

you need to:

delete line 45 (or 39-45)
remove the green color and replace with grayscale (line 46)
remove the start of a new page since you want to slowly fill the screen (lines 60-65)
change the grid values in 50, 57
since you want to have it per second you need an idea to add up the values until one second is over (use millis()). Probably do line 50 only when one second (1000 millis) is over (and continue the for-loop before)
later you want a real 2D-Array for the grid or even a class maybe since the code now is now a little hard to read

Greetings, Chrisir

ollena

Re: Visualising sound

1 year ago

Thanks, Chrisir,

I think I've already tried most of what you advised,
and I couldn't really achieve the result I need,
anyway, I'm working on it now.
I'll let you know when I will succeed in it.
Thanks for your help Chrisir.

Alena x

Chrisir

Re: [WatchList] Visualising sound

1 year ago - via EMail-to-forum

Hi, if you like post your current Code and I have a look.

ollena

Re: Visualising sound

1 year ago

Thanks Chrisir

The solution i tried for my code: i amended the frame rate, not sure if it's correct, but it slows down the visual,

but it looks like it converts sound into lines of squares and what I need is to convert every second into one little square.

Can you please have a look if it's possible to amend your code or it's worth creating a new one from scratch.
(which i can't do yet on my stage of knowledge unfortunately :(

I will really appreciate if you could lead me in this one
Any any help or advise will be very very helpful

Thanks
Alena

import ddf.minim.*;
// import processing.opengl.*;
AudioPlayer track;
Minim minim;
float bpm = 50.2;
float mspb = 60000/bpm;
float x = 0;
float y = 0;
float x1 =0;
float x2 =0;
float y2 =0;
float y1 =0;
//
float horizontal_scaling = 180; // or 160
float line_height = 1;
int time = 0;
//
//
void setup()
{
size(860, 700);
background(0);
stroke(255);
strokeWeight(0.5);
frameRate(20);
smooth();
minim = new Minim(this);

track = minim.loadFile("IN_BRUGES.mp3", 2048);
track.loop();
}
//
//
void draw()
{
int lapse = millis() - time;
//
for (int i = 0; i < track.left.size()-1; i++)
{
    float column_norm = line_height/1*(0+1.5*y);
    x1 = x     / horizontal_scaling;
    x2 = (x+1) / horizontal_scaling;
    y1 = track.left.get(i) *line_height/2 + (column_norm);
    y2 = track.left.get(i+1)*line_height/2 + (column_norm);
    // line(x1, y1, x2, y2);
    fill(0, map( abs(track.left.get(i)), 0, .2, 0, 255), 0);
    // stroke(222, 2, 2);
    noStroke();
    rect(x, y, 5, 5);
    x+=6;
} // for
//
if (lapse > mspb || x > width )
{
    // start a new line on the left
    time = millis();
    y += 6; // or 1
    x = 0;
    //if (y2 > height-40 )
} // if
} // func
//
//
void stop()
{
// always close Minim audio classes when you are done with them
track.close();
minim.stop();
super.stop();
}
//

Chrisir

Re: Visualising sound

1 year ago

hi,

this Code delivers something that at least looks similar to what you mean.

But I am not sure if it's mathematical correct and precise regarding one second, mainly because I don't understand the minim library or the function of track.left.get(i)).

But the general thinking is: While not 1 second is over, add up the color values (based on the sound volume) and count how often you do this. Take the average of the colors added up for this second and display it. Thus the current rect changes during the second, because he keeps on adding up his colors based on the volume.

When 1 second is over, he skips to the next rect and resets the values for color.

You can discuss this with your tutor.

Great movie btw.

Greetings, Chrisir

// visualizes sound in 1-second-blocks
//
import ddf.minim.*;
AudioPlayer track;
Minim minim;
float x = 0;
float y = 0;
int lastTime=0;
float oldCol=0.0;
int count=0;
//
//
void setup()
{
size (73*9, 73*9); // (860, 700);
background(0);
noStroke();
minim = new Minim(this);
track = minim.loadFile("IN_BRUGES.mp3", 2048);
track.loop();
lastTime=millis();
}
//
void draw()
{
//
for (int i = 0; i < track.left.size(); i++)
{
// oldCol sums up the colors for one rect
oldCol=oldCol+map( abs(track.left.get(i)), 0, .2, 0, 255);
// "count" counts how many colors for one rect have been count
count++;
// the average of all count colors is the color
fill( oldCol/count );
rect(x, y, 8, 8);
// has 1 second passed?
if (1000 <= millis() - lastTime) {
// next rect
x+=9;
lastTime=millis();
// reset our colors
count=0;
oldCol=0;
} // if
//
if ( x > width ) {
// start a new line on the left
y += 9;
x = 0;
} // if
} // for
} // func
//
//
void stop()
{
// always close Minim audio classes when you are done with them
track.close();
minim.stop();
super.stop();
} // for
//

ollena

Re: Visualising sound

1 year ago

Hi Chrisir,

it's amazing how you do it! it works exactly how i need it. I'm enjoying it now, experimenting with time and different soundtracks.
oh, and i'm glad you like 'in bruge' it's a great film indeed.

I'm very grateful for the code, Chrisir,
can't thank you enough really

Alena x

Chrisir

Re: Visualising sound

1 year ago

hi,

I am glad to hear that.

Chrisir

Re: Visualising sound

1 year ago

oh, and try
if ( x > = width ) {
with a = sign

in line 46

ollena

Re: Visualising sound

1 year ago

ok, I've changed 46 line.
thanks again, you are a star!

Chrisir

Re: Visualising sound

1 year ago

Thank you...

Chrisir

Re: Visualising sound

1 year ago

Hello,

when you think further about it - as I accidentally did - you'll find some interesting things.

Either the for loop is done before the second is over, that's fine, he starts all over in draw() within the same rect. Or the for loop is not over before the second is over, he goes to the next rect and works on within the for loop - fine as well.

But: Processing always draws only once at the end of draw() (60 times per second or so). Thus the rect() within the loop is not necessary. Place it at the end of draw, after the loops }. But then you would loose a rect when he goes to next rect within the for loop. So in fact you need rect when 1 second is passed and then before changing x. But you see no change of a rect anymore. See code below.
Or remove the for loop and leave the handling of "i" to draw() itself (you then have to take care of i yourself). The latter should give you more changes in the rect but I am not sure about the handling of track.left.get(i) etc.
I also noted that the moving to the next second is not so precise (offset of some 1/100 second). That could sum up when you play 1,5 hours...

You see, a lot to ponder about...

Greetings, Chrisir

// visualizes sound in 1-second-blocks
//
import ddf.minim.*;
AudioPlayer track;
Minim minim;
float x = 0;
float y = 0;
int lastTime;
float rectColor=0.0;
int count=0;
//
//
void setup()
{
size (73*9, 73*9); // (860, 700);
background(0);
noStroke();
minim = new Minim(this);
track = minim.loadFile("test.mp3", 2048);
track.loop();
lastTime=millis();
}
//
void draw()
{
fill(0);
rect(width-200, height-200, 200, 200);
fill(111);
text(millis(), width-100, height-100);
text(lastTime, width-100, height-70);
//
for (int i = 0; i < track.left.size(); i++)
{
// "rectColor" sums up the colors for one rect
rectColor=rectColor+map( abs(track.left.get(i)), 0, .2, 0, 255);
// "count" counts how many colors for one rect have been count
count++;
// has 1 second passed?
if (1000 <= millis() - lastTime) {
// the average of all count colors is the color
fill( rectColor/count );
rect(x, y, 8, 8);
// next rect
x+=9;
lastTime=millis();
// reset our colors
count=0;
rectColor=0;
//
if ( x > width ) {
// start a new line on the left
y += 9;
x = 0;
} // if
} // if
//
} // for
//
} // func
//
void stop()
{
// always close Minim audio classes when you are done with them
track.close();
minim.stop();
super.stop();
} // for
//

ollena

Re: Visualising sound

1 year ago

Hi Chrisir,

May I kindly ask you? Is it is possible at all to change this code so that it operates a life sound playing at the moment rather than mp3?

I've found this code that creates another picture, but when I try to combine your code using this FFT fft instead of the track it doesn't work :(

Can you please advice how I can change it?
Is it easy to do it actually?

Thank you xxx

/**
* LiveSpectrogram_window
* Takes successive FFTs and renders them onto the screen as grayscale, scrolling left.
* This version includes variable window length. Drag the mouse left and right to change the
* length of the Hanning window applied before the FFT is taken.
*
* Dan Ellis dpwe@ee.columbia.edu 2010-01-15
*/

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;
AudioInput in;
FFT fft;

// Configuration: spectrogram size (pixels)
int colmax = 500;
int rowmax = 250;
// Sample rate
float sampleRate = 22050;
// buffer size (= FFT size, must be power of 2)
int bufferSize = 1024;

// Variables
int[][] sgram = new int[rowmax][colmax];
int col;
int leftedge;
int window_len = bufferSize;

void setup()
{
size(colmax + 80, rowmax, P3D);
textMode(SCREEN);
textFont(createFont("SanSerif", 12));

minim = new Minim(this);

// setup audio input
in = minim.getLineIn(Minim.MONO, bufferSize, sampleRate);

fft = new FFT(in.bufferSize(), in.sampleRate());

// suppress windowing inside FFT - we'll do it ourselves
fft.window(FFT.NONE);
}

void draw()
{
background(0);
stroke(255);

// grab the input samples
float[] samples = in.mix.toArray();

// apply windowing
for (int i = 0; i < samples.length/2; ++i) {
   // Calculate & apply window symmetrically around center point
   // Hanning (raised cosine) window
   float winval = (float)(0.5+0.5*Math.cos(Math.PI*(float)i/(float)(window_len/2)));
   if (i > window_len/2) winval = 0;
   samples[samples.length/2 + i] *= winval;
   samples[samples.length/2 - i] *= winval;
}
// zero out first point (not touched by odd-length window)
samples[0] = 0;

// perform a forward FFT on the samples in the input buffer
fft.forward(samples);

// fill in the new column of spectral values
for(int i = 0; i < rowmax /* fft.specSize() */; i++) {
   sgram[i][col] = (int)Math.round(Math.max(0,2*20*Math.log10(1000*fft.getBand(i))));
}
// next time will be the next column
col = col + 1;
// wrap back to the first column when we get to the end
if (col == colmax) { col = 0; }

// Draw points.
// leftedge is the column in the ring-filled array that is drawn at the extreme left
// start from there, and draw to the end of the array
for (int i = 0; i < colmax-leftedge; i++) {
   for (int j = 0; j < rowmax; j++) {
     stroke(sgram[j][i+leftedge]);
     point(i,height-j);
   }
}
// Draw the rest of the image as the beginning of the array (up to leftedge)
for (int i = 0; i < leftedge; i++) {
   for (int j = 0; j < rowmax; j++) {
     stroke(sgram[j][i]);
     point(i+colmax-leftedge,height-j);
   }
}
// Next time around, we move the left edge over by one, to have the whole thing
// scroll left
leftedge = leftedge + 1;
// Make sure it wraps around
if (leftedge == colmax) { leftedge = 0; }

// Add frequency axis labels
int x = colmax + 2; // to right of spectrogram display
stroke(255);
line(x,0,x,height); // vertical line
// Make text appear centered relative to specified x,y point
textAlign(LEFT,CENTER);
for (float freq = 0.0; freq < in.sampleRate()/2; freq += 500.0) {
   int y = height - fft.freqToIndex(freq); // which bin holds this frequency?
   line(x,y,x+3,y); // add tick mark
   text(Math.round(freq)+" Hz", x+5, y); // add text label
}

// Report window size
text("Window length = "+window_len+" points", 5, 10);
}

void mousePressed()
{
mouseDragged();
}

void mouseDragged() {
// map mouse position within window to -1..1
float proportion = map(mouseX, 0, colmax, -1, 0);
// convert to window length, log scale, 2^8 range
window_len = (int)Math.round(Math.pow(2.0, 4.0*proportion)*(float)bufferSize);
}

void stop()
{
// always close Minim audio classes when you finish with them
in.close();
minim.stop();

super.stop();
}

Chrisir

Help still needed...

1 year ago

hello,

the code doesn't work for me.
Does it need audio input on the "line-in" of the Hardware of the PC? I don't have that.

I suppose the merging of the code should work. Maybe somebode else can help?

Greetings, Chrisir

ollena

Re: Visualising sound

1 year ago

Thank you Chrisir, I'll try to work it out.
xxx

minazs86

Re: Visualising sound

10 months ago

Hey, I am lucky to find you guys. I am working on a different project that requires taking audio from a microphone at one end and have a visual effect at the other end. I have no background in processing but after comparing a couple of other codes and yours, I ended up having the live effect work. The only problem is I don't know how to have this keep going rather than stop having any visual effect when the page is full. also is there a way that instead of a line it would make the pixels in a slower rate. I am trying to point at noise pollution, so time is important to show the effect I want.

this is the code I have right now after editing yours. thank you.

import ddf.minim.*;

// import processing.opengl.*;

AudioInput in;

Minim minim;

float x = 0;

float y = 0;

float x1 =0;

float x2 =0;

float y2 =0;

float y1 =0;

float horizontal_scaling = 180; // or 160

float line_height = 1;

void setup()

{

size(860, 700);

background(0);

stroke(255);

strokeWeight(0.5);

frameRate(20);

smooth();

minim = new Minim(this);

in = minim.getLineIn(Minim.STEREO, 512);

}

void draw()

{

for (int i = 0; i < in.bufferSize()-1; i++)

{

float column_norm = line_height/1*(0+1.5*y);

x1 = x / horizontal_scaling;

x2 = (x+1) / horizontal_scaling;

y1 = in.left.get(i) *line_height/2 + (column_norm);

y2 = in.left.get(i+1)*line_height/2 + (column_norm);

// line(x1, y1, x2, y2);

fill(2, map( abs(in.left.get(i)), 0, .2, 0, 255), 0);

// stroke(222, 2, 2);

noStroke();

rect(x, y, 5, 5);

x+=6;

} // for

{

// start a new line on the left

y += 6; // or 1

x = 0;

//if (y2 > height-40 )

} // if

} // func

void stop()

{

// always close Minim audio classes when you are done with them

in.close();

minim.stop();

super.stop();

}

Chrisir

Re: [WatchList] Visualising sound

10 months ago - via EMail-to-forum

Search the name ollena in the forum - she did amazing visualizations.

In theory you have to check whether the y is > height and if so set y to 1 or 20 or so and then background(0) once and x=0. this will start the page all over again.

Sorry, I am not on my computer.

Good Luck!

Chrisir

Top Reply