Record audio, save FFT data to .txt - how to best approach?

edhebert

I hope to build an app that records and processes audio info sampled from the environment, and saves certain FFT data to a txt file. This would be my first p5.js project, but thought it might be a fun way to get my feet wet with it. Still, since I'm a noob to all this, I'm hoping I could get some general workflow advice from you folks.

In a perfect world, my user would interact with the app within a web browser, hit a "record" button, capture a 5 second event, see the visualized FFT spectrum and a table of frequency/amplitude values emerge. From there, the user hits a "save" button, and both an mp3 file of my recording and a text file of its analyzed frequencies and amplitudes are saved.

But I realize there's some stuff (like saving files) that isn't allowed in client-side JS. And I'm not thrilled that I have to allow permission to use the mic each time I want to record from within the browser.

What would you suggest as the best approach? Could I (should I?) use node.js to handle the recording and saving of the audio, and use p5.js just to process the audio? Or is this type of interactive app not really fit well to p5.js.

For the record, I built the visualizer / FFT table in about a minute with p5.js (logging data to the console), so I was very excited at its capabilities!

Thanks in advance for any thoughts for me!

GoToLoop

... there's some stuff (like saving files) that isn't allowed in client-side JS. And I'm not thrilled that I have to allow permission to use the mic...

In all honesty, if a web app could capture audio & video, load & save local files, collect geolocation, etc.,
w/o asking for permission 1st, we'd be spied left & right more than we already are! =;

I'm still a JS noob and not even touched those things yet. All I know is that in order to do things which "offline" apps can do, a web app needs to request those resources from a server. Node.JS is an excellent choice for it IMO! <):)

• http://p5js.org/reference/#/libraries/p5.sound
• http://nodejs.org/

talkscheap

This sounds great!

You can record and save files with a p5.AudioIn and a p5.Recorder: http://p5js.org/reference/#/p5.SoundRecorder

The recorder only saves .wav files for now.

p5.FFT can return spectrum data. http://p5js.org/reference/#/p5.FFT/analyze

You could save this data using the save() method, as JSON or just Strings http://p5js.org/reference/#/p5/save

The save methods will save to the client's computer, prompting a save dialog that depends on the client's browser.

If you want to save to your server instead, you could do that on the server-side, for example with node.js. p5 doesn't have great hooks for making HTTP requests yet, but it is on the horizon, so if you wanna go that route, let's figure out how to make it work!

-Jason

edhebert

Wow, thank you guys!

Jason, I didn't realize that save() had that versatility. Perfect. Sounds like p5 has most of the stuff I need after all! This should work without the server. That said, I'm also interested in messing with node and p5 together, so maybe I'll eventually check back with a workflow for http requests.

I really appreciate your input!

edhebert

OK folks, I tried your suggestion, but am getting an unexpected behavior when attempting to save my array of data. The data array looks as expected when I console.log it.

However, when I add the line with the save() function, multiple save dialogs are triggered, and the data array console.logs multiple times.

Any ideas?? Thx!

function mouseClicked(){

    // find maximum amplitude in a given range of frequencies
    var minFreq = 0;
    var maxFreq = 20000;

    var currentEnergy = 0;
    var peakAmplitude = 0;
    var peakFrequency = 0;

    var data = [];

    // grab the energy every 20Hz through the range of frequencies
    for (var i = minFreq; i < maxFreq; i=i+20) {
        // get energy at this frequency
        currentEnergy = fft.getEnergy(i);

        // add data to array
        data.push(i + ', ' + currentEnergy);

        if (currentEnergy > peakAmplitude) {
            // this is the new peak - assign current freq and amplitude as peaks
            peakAmplitude = currentEnergy;
            peakFrequency = i;
            // console.log('peak frequency: ' + peakFrequency + ',  peak amplitude: ' + peakAmplitude + '\n');
        }
    }

    // this line delivers as expected
    console.log(data);
    
    // this line is giving me grief! this line will cause multiple save dialogs and looping
    save(data, 'spectrum.txt');
}

GoToLoop

The mouseClicked() online example is working perfectly for me: /:)
http://p5js.org/reference/#/p5/mouseClicked

edhebert

@GoToLoop, not sure what I'm missing. For now, I want to do the analysis and save the file when the mouse is clicked. Am I missing something that you're seeing?

My code can be seen at http://edhebert.org/p5js/

edhebert

I think I've figured out the issue. Including return false; at the end of my mouseClicked() prevented the function from looping uncontrollably when saving the file!

Thanks @GoToLoop for pointing me to the reference. The example code chunk I'd modified didn't originally have that, and I wouldn't have known it's required otherwise.

GoToLoop

Congrats! I had no idea that returning some Boolean would have any effect on mouseClicked()! 8-|

P.S.: There's also mousePressed() btW. Which doesn't w8 for a mouse button to be released:
http://p5js.org/reference/#/p5/mousePressed