Why does it work this way? Need help for a schoolproject

beladrum

Hey Guys,

first of all, sorry if my englisch isnt perfect , because im from Germany but i will try my best :D

Im working on a school project and i need to create a Musicvisualizer in 3D. Now ive wrote a code thats works good and i also like how it looks, but i dont know why it work in this way. the Code is creating many Spheres if i use the translate command to place it in the middel of the screen. In my opinion it looks very cool with more of these Spheres but i dont understand why there are so many. If I just write the Sphere command without the Translate , it only creates one Sphere thats moving to the analysed float from the FFT.

Can someone maybe tell me why this happens ? because i need to present and also explain how my code works in front of my teacher. hope you can help me , thx and greetings.

---

CODE :

import shapes3d.*;
import shapes3d.animation.*;
import shapes3d.utils.*;

import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import ddf.minim.signals.*;
import ddf.minim.spi.*;
import ddf.minim.ugens.*;  



import peasy.*;
int i;
PeasyCam cam;
Minim minim; 
AudioPlayer player;
FFT fft;
void setup() {
  size(800, 800, P3D);
  cam = new PeasyCam(this, 0, -100, 200, 1000);
  cam.setMinimumDistance(20);
  cam.setMaximumDistance(5000);

  minim= new Minim(this);
  player= minim.loadFile("bmth.mp3");
  player.play();

  fft = new FFT(player.bufferSize(), player.sampleRate());
}

void draw() {
  rotateX(-.5);
  rotateY(-.5);
  background(0);
  //fill(255,0,0);
  //sphere(i);
  keyPressed();

  fft.forward(player.mix);// used to analyze the frequency coming from the mix 
  for (int i = 0; i < fft.specSize(); i += 50)// specSize is changing the range of analysis
  {
    noFill();

    ellipse(width/2, height/2, 500, fft.getFreq(i/2)*2.5);

    ellipse(width/2, height/2, 200+fft.getFreq(i/2)*2.5, 300+fft.getFreq(i/2)*2.5);

    ellipse(width/2, height/2, fft.getFreq(i/2)*2.5, 400);

    fill(255, 0, 0);

    stroke(255, 255, 255);

    pushMatrix();
    translate(400, 400, 5);

    sphereDetail(20, 3);

    sphere(50+ fft.getFreq(i/2)*0.2);

    popMatrix();

    sphereDetail(3, 20);

    translate(400, 50, 5);

    sphere(50+ fft.getFreq(i/2)*0.2);

    pushMatrix();

    translate(0, 750, 5);
    sphere(50+ fft.getFreq(i/2)*0.2);
    popMatrix();
  }
}

void keyPressed () {

  if ( key=='s') { 
    player.pause();
  }
  if ( key=='d') { 
    player.play();
  }
}

koogs

Edit post, highlight code, press ctrl-o to format code

prince_polka

do you mean like on line 42 ?
if you remove that loop you only get one "thing"

kfrajer

If you add this line right in line 41 and 42, you get some values to keep in mind: println(player.bufferSize()+" "+ player.sampleRate() + " " +fft.specSize()); The reason you get that many spheres is because of your for loop, since you increment your loop by steps of 50.

For example, using my dummy song for testing, I get "1024 44100.0 513"

Now, when you do a FFT, you start with an initial data buffer. In your case it is coming from the mp3. How much sound is stored in this data buffer? We need to check the sample rate: 44100 Hz or 44.1 KHz so your data buffer holds 1024*(1/44100) seconds of music , just about 0.023 seconds each time draw is called. When you call fft.forward, it will return your data buffer (which is in time domain) in frequency domain. How much do you know about time and freq domain.... ay caray... this requires a sort of technical explanation that other websites can do better than me but here I'll give it a try.

Remember that when you manage your data, you have a data buffer with is 1024 in size and in total it holds 23 msecs of your music at each time. Time domain means that the music in this buffer starts at position 0 and the end of the 23 msec ends at position 1023, so you will listen to your song in that order. The data chunk of 23msec is a small part of your song, but because it is processed in draw, you are processing many and different chunks in order. For each chunk, you do a FFT meaning it gets transformed from time domain to a frequency domain.

Frequency domain technicality: If you start with a buffer of size N, your processed FFT returns a frequency buffer of size N/2. So in your case, this explain why we start with 1024 and the FFT spectrum size is 513 (ok... maybe is N/2+1... close enough)

What does the freq buffer hold? it holds the frequency equivalent of your initial time buffer. If you imagine a music made with a sine wave of a single frequency, your freq spectrum will be a flat line except at that frequency position which will be non-zero. That non-zero value is the amplitude of that frequency in that time buffer. This amplitude is important. It tells how much this frequency contributes to the 0.023 seconds of your sound piece.

If you load an opera song, the tenors can play different frequencies. If you analyze their songs, some buffer will have more high frequency contribution and some buffers will have more components in the low frequency. Some buffers will have spikes all along the spectrum, or a mix of low and high frequency components. If there is a silence, then of course the buffer will be flat silent.

How is it possible to translate time domain to frequency domain. The Fourier theorem says, without going into many details, that your sound buffer in time domain can be represented as the sum of multiple waves of different frequencies and amplitudes. If you say "A-E-I-O-U" and lets assume every letter fits in a time buffer, then each corresponding letter will have a fft signature in the frequency domain. This signature is similar between every human but not the same. This is how one could use voice recognition to execute a task [Off topic...]

Back to the FFT theorem. In other words, the Fourier theorem promises that our sound data can be represented by overimposing waves of defined frequencies and amplitudes. When you provide data to an FFT engine, the FFT engine uses an algorithm to try to guess what is this sets of frequencies and amplitudes that could represent your sound. Of course this does not work all the time but in general it does a good job.

One note. Waves are usually represented using sine and cosine functions. Because we are dealing with real data (not complex values), then the resulting FFT has some characteristics related to the input data. The resulting FFT is real and odd, meaning that the spectrum repeats after N/2. This is the reason why only, for initial time data of N, we only consider N/2 values. The other N/2 values is a copy of the first part and it can be discarded.

You can read somebody try to explain this using other words, directly from the minim creators: http://code.compartmental.net/minim/javadoc/ddf/minim/analysis/FFT.html

In your case, if in the for loop you change your step size to either 100 or 20, you will have opposite effects. For 20, you are increasing the number of spheres presented in your sketch. Each sphere is representing a frequency. The more spheres you have, the more frequency components you will be able to see. If you choose 100 instead, then 513/100 means you will get about 5 spheres and you can only see about 5 frequency components.

I will suggest to choose a value not too big. 100 could work. I think 50 is not to bad. Maybe you should consider displaying music that contains different frequency information. Then you can show how your spheres represent the frequency contribution of the different time buffers of your song. To keep things simple, you should stick to playing songs with the same sample rate. I believe you can define the buffer size in minim to 1024 in your constructor. You can check the documentation for more info.

Not sure if this is clear and I ca only hope this helps,

Kf

beladrum

Wow >:D< kfrajer.... Thank you sooo much !

youre answer is awesome. Now i know how my code works the right way und why this happend to the Spheres :D
Thank youre for your detailed explaination , it was good to understand and know im able to finish my project with some details :)

if im done with my presentation i will let you know what grade i got .

Have a nice day O:-)

kfrajer

@beladrum Good to hear it was clear and good luck!

Kf

kfrajer

Additional relevant post: https://forum.processing.org/two/discussion/comment/78678/#Comment_78678

Kf