sound library to synchronize audio and video, using deferred time

pietroLama · November 2017

Hi guys, as many of you know, often exporting multimedia content from processing, which contains both audio and video, can be very difficult for synchronization issues. I've created a class capable of reading and saving wav files, copying the file to a float vector and implementing a method for fft analysis and various audio filters. What I would like to do now is to create from these classes a library whose operation would be similar to that of minim, only being deferred, we will be choosing fps and anything else useful to synchronize perfectly audio and video. Any libraries useful would like to give me a hand?

kfrajer · November 2017

You should check the VideoExport lib. More precisely the withAudioViz example as @hamoid makes some comments that are relevant to your post. I will paste the comment below for you to consider.

Kf

/*
   Example to visualize sound frequencies from
   an audio file.
    
   Producing a file with audio and video in sync
   is tricky. It gets easily out of sync.
    
   One approach, used in this example, is:
   
   Pass 1. Analyze the sound in a Processing sketch 
           and output a text file including the FFT 
           analysis data.
   Pass 2. Load the data from pass 1 and use it to 
           output frames for a video file, including 
           the right frames to match the sound 
           precisely at any given time.
            
   Using this technique it does not matter how fast
   or slow your second program is, and you know that
   no frames will be dropped (as may happen when
   recording live).
   
   The difficulty of recording live graphics with
   sound is that the frame rate is not always stable.
   We may request 60 frames per second, but once in
   a while a frame is not ready on time. So the
   "speed of frames" (the frameRate) is not constant
   while frames are produced, but they are probably
   constant when played back. The "speed of audio",
   on the other hand, is often constant. If audio
   is constant but video is not, they get out of 
   sync.
*/

pietroLama · November 2017

thank you for the reply @kfrajer, but i've already done these part, i've solved this problem. what i need now is someone that collaborate with me to create a library of my classes. :D

hamoid · November 2017

Hi! What if I added an example to the VideoExport library that used your library to analyze the sound? So step 1 would be using your library and step 2 using VideoExport. Is that close to what you want to achieve?

koogs · November 2017

it's the library creation he's having problems with...

hamoid · November 2017

I see! Did you explore https://github.com/processing/processing-library-template? Do you have experience with Eclipse?

pietroLama · November 2017

Yes i have experience with Eclipse, but i don't want use video library, my library is for deferred time, so the sketch Will generate a lot of frame at fps choosed by you and you Can Accord audio buffer to your sketch parameter.

Howdy, Stranger!

Categories

In this Discussion

sound library to synchronize audio and video, using deferred time

Answers