We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpSound,  Music Libraries › Minim: real-time sample array to sound card output
Page Index Toggle Pages: 1
Minim: real-time sample array to sound card output (Read 2683 times)
Minim: real-time sample array to sound card output
Oct 24th, 2009, 12:37pm
 
To all,

I have a question about sending a continuously changing array of samples to the output of my sound card using Minim.  Basically, a buffer of samples is created each time through my draw() loop.  The last step in the signal generation process is an inverse FFT, which updates a float[] buffer.  This is very similar to the example at the bottom of the Minim User Manual's page on FFT.

The Minim manual mentions that this example could be used to modify a real audio signal.  Now, I would like to send this buffer of samples to the sound card.  It appears that I can not directly write to any of AudioOutput's AudioBuffers (left, right, or mix).  Could anyone help me find a way to write "buffer" to the sound card output?

I'm not sure that I can write a user defined AudioSignal (like the MouseSaw example) because I create the samples in real-time from microphone data and several sawtooth signals.  Can I create several sawtooth waveforms and getLineIn() within a user defined AudioSignal class?

Does anybody have any suggestions?

Thanks in advance,
Adam
Re: Minim: real-time sample array to sound card output
Reply #1 - Oct 28th, 2009, 3:31am
 
hm... interesting
Re: Minim: real-time sample array to sound card output
Reply #2 - Oct 28th, 2009, 6:59am
 
Have fun !!

it take signal on the left and plays back on the right .

not surpisingly the sound is crap, but it can give you an idea

h


import processing.core.PApplet;
import ddf.minim.*;
import ddf.minim.analysis.*;


       private FFT fft;
     private FFT fft2;
     private AudioInput input;
     private AudioOutput output;
     private Minim minim;

     private int SAMPLERATE = 44100;
     private int BUFFERSIZE = 512;
      void setup() {
             size(300,200);
             
             minim = new Minim(this);
             
     
             
             fft = new FFT(BUFFERSIZE, SAMPLERATE);
           //fft.window(FFT.HAMMING);
             
   
             
           

           input = minim.getLineIn(Minim.STEREO, BUFFERSIZE, SAMPLERATE);
           input.addListener(new Listener());

           output = minim.getLineOut(Minim.STEREO, BUFFERSIZE, SAMPLERATE);
           output.addSignal(new MySignal());
     }

     class Listener implements AudioListener {
           public void  samples(float[] sample){
             fft.forward(sample);
           }
           public void samples(float[] left, float[] right) {
             samples(left);
           }
     }

     
     private class MySignal implements AudioSignal{
           public void generate(float[] out) {

      fft2 = new FFT(BUFFERSIZE, SAMPLERATE);

              // out = new float[BUFFERSIZE];
               int u=2;
               if(u==1){//method 1
                     //FFT imaginary part used to reconstruct time info i think.
                     fft.inverse(out);
               }else{
                     //only use real part of the FFT. no time information i think.
                   fft2.window(FFT.HAMMING);
                   for(int i=0;i<BUFFERSIZE/2;i++){
                     fft2.setBand(i, fft.getBand(i));
                   }                
                   fft2.inverse(out);
               }
                                       
               for(int i=0;i<out.length;i++){
                     out[i]=out[i];
               }
             }

           public void generate(float[] left, float[] right) {
               generate(right);
           }
     }
Re: Minim: real-time sample array to sound card output
Reply #3 - Oct 30th, 2009, 4:35am
 
Thanks very much mots,

In short, method 2 is exactly the approach I was hoping somebody would come up with.  In my final application I do quite a bit more stuff with the microphone data before I do an inverse FFT though.  Before you say that the sound is crap, I should give away the secret and explain that I'm trying to build a Vocoder!

Basically it works like this:  Microphone data comes in and an FFT is applied.  Let's call that input A, the modulator.  It turns out that most of the features of human language (vowel sounds, consonants, etc) have a pretty unique frequency spectrum shaped mostly by the mouth and tongue.  We would like to encode this information into a carrier signal to create a kind of singing robot voice.

There is a second audio signal which is the output of a synthesizer (hopefully rich in harmonic content).  Let's call that input B, the carrier.  In my implementation, this synthesizer is also created in Minim using a set of SawWave generators.  By using the computer keyboard you can decide the musical root note and chord structure (maybe an A major, or F minor 7th, etc.).  Then we take the FFT of input B to get it's spectrum also.

Now if we scale the spectrum of the carrier by the spectrum of the modulator, the result is a spectrum that has all of the pitch and chord information from the synthesizer but is encoded by the consonant/vowel information from the microphone.  All that remains is to take the inverse FFT of the scaled signal and put it out to the speakers.  Singing Robot!

Whenever I'm finished developing the whole application, I will append it to this thread.

I do have a few comments about your code:

  • The new Listener and MySignal classes are the bridges that I was missing.  These handle the refilling of the audio output buffers properly without all of the clicking and popping I was getting using a more naieve approach.
  • The input to your fft.forward call is an array of REAL numbers that represent samples with respect to time indices.  The output of fft.forward is generally an array of COMPLEX numbers that represent frequency content with respect to frequency indices (sometimes called bins).  If you look at the Minim source code though, you will see that fft.forward with one argument actually returns the MAGNITUDE of those complex numbers using sqrt(REAL2 + IMAG2).  This does throw away the phase information (which is like time alignment) as you say.
  • In the MySignal() class, generate() function: I don't think the for() { out[i] = out[i]; } loop is needed at the end.  fft2.inverse(out) writes directly to the buffer called out.


Thanks again for the help, and stay tuned for the final product!
Adam
Re: Minim: real-time sample array to sound card output
Reply #4 - Oct 30th, 2009, 6:42am
 
hello,
hi !

the last for loop with the dummy assignment was used in debug mode in eclipse to multiply the ouput by an amplifier to lower or higher the volume Smiley so that i did not loose my ears Smiley

i of course noticed the "vocoderish" nature of the inverse FFT without the phase information. kinda funny. Smiley
one problem however i faced is the junction between generated audio buffer which  did not had matching begin/end. so it would produce a click between every audio buffer. i suppose this can be somehow sorted out.

my DSP lessons were long time ago... i should read again about FFT.


happy i could help !
Re: Minim: real-time sample array to sound card output
Reply #5 - Oct 30th, 2009, 11:19am
 
I agree about the clicks between every audio buffer reload.  This happens at a rate of SAMPLERATE/BUFFERSIZE = 44,100/512 = 86.13 Hz or roughtly 11ms and could potentially be a buzzing sound. Here are some initial thoughts:

The severity of each click depends on the mismatch between the end sample of the previous buffer and the beginning sample of the current buffer.  Both of these samples depend on exactly what signal you're creating at that moment in time.

One option would be to only create audio signals which have an integer number of periods within each BUFFERSIZE.  That is, sine waves at any harmonic of 86.13 Hz.  You have to get the phase exactly equal to zero.  Actually, this is exactly what the inverse Fourier transform should do...  I can't see why there would be any clicks and pops using this method.  I'll have to look closely at two adjacent buffers and see what's going on sometime soon.

The alternative is probably to window the contents of each audio buffer so that is (fairly) smoothly approaches zero at the beginning and end.  This has the unfortunate side effect of introducing an annoying volume modulation at the same 86.13 Hz.
Re: Minim: real-time sample array to sound card output
Reply #6 - Oct 30th, 2009, 1:29pm
 
volume modulation at around 80hz will be ring modulation Smiley

i though this problem could be actually related to the loss of the phasing information.

to be experimented !
Page Index Toggle Pages: 1