Processing 1.0 - Processing Discourse - Synchronizing generative video + audio

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

‹ Previous Topic | Next Topic ›

Pages: 1 2 3 4

Synchronizing generative video + audio (Read 29944 times)

fry YaBB Moderator Offline Posts: 2623 mass, usa	Re: Synchronizing generative video + audio Reply #30 - Aug 29^th, 2007, 12:42am afpAccessDenied is a quicktime/macos error. it means that it can't get to the file for some reason, which could be things like 1) the file isn't there 2) you don't have permissions to read it 3) it's in use by another process, etc..

error

Re: Synchronizing generative video + audio
Reply #31 - Oct 7^th, 2007, 5:30pm

Hi,

I need some advice on compression and codecs for an audio-reactive audio-visual project that I made with processing. I use Mr. Bollinger's code to write the movie file while analyzing the audio file off-line. Animation codec seems to be the best choice to get the highest quality video file. However, the data rates are so high, normally, I can't even play it properly on my mac. The thing is, I need to get this animation on a DVD, which is restricted to 9 Mbps, and I need to find the right compression type and data rates to do this.

I tried (using moviemaker) cinepak, sorenson, h263/264 to get small files but the quality is considerably reduced. Animation and raw types are great, but I need to compress them anyway in order to make a DVD-video disc.

So what would you suggest? I couldn't get the animation codec compressed with acceptable quality and the others don't work very well. What is the procedure that you would follow in order to do what I want to do? Say, would you write the animation file in a lossless codec in the first place, and then compress it (with which compression method?) to put it on a DVD, or would you save it in a lossy codec while writing it in the first place from processing? Also, is it possible to limit the data rate when writing the file with moviemaker?

Thanks,

Emre.

davbol

Re: Synchronizing generative video + audio
Reply #32 - Oct 8^th, 2007, 12:00am

I posted the "moviemaker" version of the code simply as a convenience for those who use that approach. I myself don't use it, and prefer to write out individual uncompressed frames (tif, tga, png, bmp, whatever) and then do ALL compression as a post-processing (pun intended! Cheesy

) step.

As to "HOW" you do that, it will depend on what software you have available. And as to what compression format you want to use for compressed files, again, depends on target. Just as a point of reference, I've had great success using the DivX codec when the target is computer playback or online playback transcoding (like youtube, vimeo, google, etc).

Your conversion path to get the best DVD output may vary - it is SO dependent on the authoring software you use (and I'm a Windows guy, so can't address Mac questions). Generally though, (for most software), you want to feed it the highest quality uncompressed video and have IT do the transcoding for you as part of the authoring process. Check the manuals, or other references for your authoring software, as to what's the best input material format.

Hope that helps.

moka

Re: Synchronizing generative video + audio
Reply #33 - Jan 9^th, 2008, 9:33pm

does anyone know how to get the levels in ESS for each frame?

in the example on the first page you get the spectrum for each frame through

myFFT.getSpectrum(myChannel.samples, (int)(frameNumber * myChannel.sampleRate / framesPerSecond));

is there a similar way for the levels? I couldnt figure it out on my own.

« Last Edit: Jan 9^th, 2008, 10:52pm by moka »

anachronik Junior Member Offline Posts: 80	Re: Synchronizing generative video + audio Reply #34 - Jan 10^th, 2008, 10:45am Hi, It seems that you can do that with getLevel function. From the sonia reference "returns the peak value for the specified channel." Have a look at http://sonia.pitaru.com/ in the reference page. regards, a.

moka Junior Member Offline Posts: 66	Re: Synchronizing generative video + audio Reply #35 - Jan 10^th, 2008, 12:25pm first of all i am using ess not sonia. Ess also has a getLevel function but you can only get the levels if you actually play the sound. Since there is no level array (as there is for the spectrum), i dont really know how to handle it.

anachronik Junior Member Offline Posts: 80	Re: Synchronizing generative video + audio Reply #36 - Jan 10^th, 2008, 1:11pm Sorry for the bad answer was reading too quickly... Sorry for that. (brain overflow).

moka

Re: Synchronizing generative video + audio
Reply #37 - Jan 10^th, 2008, 4:15pm

okay anyways, I was wondering if there is also a way to grab the levels for each frame in ESS without actually playing the sound.(for rendering porpuses)

You can do it with the spectrum through this function:

myFFT.getSpectrum(myChannel.samples, (int)(frameNumber * myChannel.sampleRate / framesPerSecond));

is there something similar in ESS for the levels?

peer

Re: Synchronizing generative video + audio
Reply #38 - May 10^th, 2008, 11:35pm

davbol wrote on May 9^th, 2007, 6:50pm:

Thanks Chris. Timestamp -- clever approach.

However, I've just re-RTFM and realized that (for my purposes anyway) I'm making this wayyy more difficult than it has to be. With ESS, the FFT's getSpectrum() method will take a float array (um, samples) and an offset (um, timecode) - how did I miss that!. IOW, offline analysis is already built in, you don't even need to play the audio at all. Demo usage for others who might've missed it also:

Now just assemble the frames, overlay the audio, and viola! perfect sync.

----------------------------------
----------------------------------
hello,

i'm working on fft and have problem with saving sequencial spectrum images. Btw, i've tried davbol's code. It really helps but i don't know what's P3D (in page 1) here i extract:

void setup() {
size(320,240,P3D);

}
can anyone pls explain

thanks

V Senior Member Offline Posts: 430 portugal	Re: Synchronizing generative video + audio Reply #39 - May 11^th, 2008, 1:21am P3D is processing software 3d rendering engine. you can choose from p3d or opengl
	Victor Martins \| www.pixelnerve.com/v

peer YaBB Newbies Offline Posts: 2	Re: Synchronizing generative video + audio Reply #40 - May 11^th, 2008, 4:09pm thanks crmx. if you dont mind i have more questions about "try-catch" methods. After reading from many threads, i've come acroos that "try-catch" are only needed if the application works on the internet? or i misunderstood?
	« Last Edit: May 11^th, 2008, 5:27pm by peer »

phi.l

Re: Synchronizing generative video + audio
Reply #41 - Jun 12^th, 2008, 11:55am

lol, u're quite on the wrong track there

try-catch is also called error handling and is used to CATCH errors, so that the application does not quit when errors occur.

for example, a division by zero will throw an error and quit your application:

int test = 10 / 0;

to avoid that the application quits you have to TRY the code first and CATCH the errors:

try{
int test = 10 / 0;
}catch(Exception e){
println("look! i caught folling error: " + e);
}

however, this is the wrong thread to ask this.
just as it is the wrong thread to answer this. (;

beckel YaBB Newbies Offline Posts: 16 LA	Re: Synchronizing generative video + audio Reply #42 - Sep 27^th, 2008, 1:45am Does anybody know if this process would be simplified by taking the draw() Thread out of the picture? For instance, Let's say you tell the PApplet "noLoop()" and then do everything in one call of draw(), treating it like a main() function. Does that make any sense?

beckel YaBB Newbies Offline Posts: 16 LA	Re: Synchronizing generative video + audio Reply #43 - Sep 27^th, 2008, 10:49pm I have done a little experimenting to find that the program that davebol suggested only works for a framerate of 25 fps. Still can't figure out why.

davbol

Re: Synchronizing generative video + audio
Reply #44 - Oct 30^th, 2008, 4:11pm

You're likely using the MovieMaker as output? I've recently found that MM actually creates a 30.3 fps when 30.0 is requested, so sync would fail when merging audio in post. (bug submitted) I myself don't typically use MM, so never really bothered to check it too closely. Yes, 25fps works fine with MM, as will 20fps, but not 30fps. (it's a divisor issue i believe)

That general code method for audio sync will work fine at 30.0fps as long as you instead render to still images and assemble to 30.0fps video in post, just avoid MM at 30fps for now.

hth

Pages: 1 2 3 4

‹ Previous Topic | Next Topic ›