I have an application where I'm looping a pair of short movies -- one a heavily-processed version of the other, so they're exactly the same length. I'm trying to keep them exactly synchronized (although being perhaps one frame, maybe two, off would be Ok). Right now, they're about a minute long.
If I start them at the same time and let them loop independently, they drift apart over time. Not a *huge* surprise, although it would've been nice if they'd stayed in sync. I added code to rewind them both whenever one finished, and this every-loop reset eliminates extremes in drift, but it's still not unusual to see 10 or so frames of offset at the end of the 1800 or so frames (and it's very wrong, visually).
I'm using movieEvent callbacks to load frames (check to see which movie caused the callback, load the appropriate image). I'm happy to use another method if it'd work better.
I tried a couple of brute-force ways to keep things aligned (like checking the frame numbers and seeking to one or the other if things drift), but performance suffers. Things get much, much worse if I run at anything other that 1X speed (in either direction, fast or slow....somehow, in this conditions, the two movies can get to be seconds off, even within the short minute duration).
Is there a good way to keep a pair of streamed movies synchronized? Should I be using the available() method for figuring out when to read a new frame? Right now, I read each frame and then copy a portion of each to visible. The two video streams are juxtaposed, so this drift is really problematic.
Suggestions? I guess I could read all 1800 frames of each stream into memory, but that seems pretty brutish (and I'm probably most interested in an approach that will work in a more general (unlimited length) case.
My movies are currently 1024x768. I'm only putting 1/4 of each one onto the screen.
The program seems to be keeping up fairly nicely, although it appears to be missing/dropping probably 1 frame every 6 or 8 or so. The videos each look pretty good, independently, but the drift is problematic.
I'm musing about how I could perhaps use textures and skip handling the image data in my program, but it's not immediately clear to me yet how to accomplish showing just the desired portion (even if it's just a simple horizontal or vertical stripe of the frame data).
I'm currently cranking out a smaller pair of movie files...I'll know soon if this helps matters any.
Ok, so if the files are smaller (about 8MB for the minute-long loops), the files definitely look like they're staying better synchronized (at this bit rate, the video is not particularly handsome....I'll see if it looks better when I turn off every frame being a keyframe (which I recently added in the hopes that it would help keep things synchronized during slow and fast playback), but I still see a frame drop every 8 or so (I'm printing *.frame() in the draw loop).
So is there really no way to keep two streams synchronized using this toolkit? Do the streams have any notion of "correct" time, or is it simply spewing the frames out there with best effort and actual time is irrelevant? Is there some encoding thing I can do to make things better/easier? I tried making every frame a keyframe, so it could skip ahead for slow/fast playback, but I'm not sure that has helped.
I guess I can put both of my videos into a single "double wide" file with Final Cut and guarantee that they're delivered in lock step, but it would seem like there should be *some* way to keep two streams (especially short ones like this) synchronized adequately. Things get really bad really quickly when I run the videos slow or fast (it can get as many as 100 frames off during the duration of a 1600 frame pair of movie clips).
I was also hoping I could advance to longer clips, at some point...but not if this behavior persists.
Sorry to get your hopes up with a reply but I have the exact same problem. I think it has to do with gsstreamer which doesn't support frame-by-frame iteration. I've been looking for alrernatives with eg. Xuggle but haven't got it to work, and it's heck of a lot more complicated than GSVideo.
Any input on how to solve this would be greatly appreciated!
Have you guys tried the GSVideo-GLGraphics integration mode that offloads some part of the frame color conversion and rendering to native code. This reduces CPU usage quite significantly, and might help in you situation.
Either the Movie and HDMovie examples included in the "Contributed Libraries/GSVideo/GLGraphics" folder show how to use this mode.
Leave a comment on andres's reply
Change topic type
Link this topic
Provide the permalink of a topic that is related to this topic