GSVideo: Playing h264 videos with Fluendo codecs

zaros.adl

GSVideo: Playing h264 videos with Fluendo codecs

in Contributed Library Questions • 2 years ago

Hi Andres,

I just bought and downloaded Fluendo's codec pack to try and improve h264 playback when using GSVideo/GLGraphics

Testing with "gst-launch -v playbin2 ..." reveals that it is working i.e. gstreamer automatically selecting the new codecs, but GSVideo doesn't show improvements.

Is there something I am missing here?

Replies(5)

andres

Re: GSVideo: Playing h264 videos with Fluendo codecs

2 years ago

Only the latest test package of GSVideo that I uploaded today uses PlayBin2 inside GSMovie, so this one should allow you to access the Fluendo codecs. Grab from here.

zaros.adl

Re: GSVideo: Playing h264 videos with Fluendo codecs

2 years ago

Thanks. I've tested it now.

Findings:

With GSMovie:

1. Using GSMovie opens a new window, not showing within the Processing canvas. But the 1080p playback is smooth.

2. MKV playback doesn't work -- this seems to be a Playbin2 problem, because constructing pipe with "gst-launch filesrc ... ! matroskademux ! ..." works but "gst-launch playbin2 uri=..." doesn't. -- got any ideas on how to fix this though?

With GSPipeline, this works:

pipe = new GSPipeline(this, "filesrc location=home/zaros/testvideos/1080p.mkv ! matroskademux ! fluh264dec");
pipe = new GSPipeline(this, "filesrc location=home/zaros/testvideos/1080p.mkv ! matroskademux ! ffdev_h264");

(and so does playback with mp4 files, using qtdemux) However, all tests show that GSVideo is dropping a lot of frames.

If you're wondering, there appears to be no significant difference in performance between fluh264dec and ffdec_h264. However, if I can use fluvadec (hardware acceleration) that would bring CPU usage down to less than 20% on my dual-core system -- that would need fluvasink to work though, and I am guessing we won't be able to do that with GSVideo? ...

... and these don't:

pipe = new GSPipeline(this, "playbin2 uri=file:///home/zaros/testvideos/1080p.mkv");

The canvas is gray. No video is displayed, but CPU is working. As I mentioned above, there seems to be a problem with playbin's mkv demux so this problem was expected.

pipe = new GSPipeline(this, "playbin2 uri=file:///home/zaros/testvideos/1080p.mp4");

A window opens, black background, no video is displayed, but CPU is working. It seems to be something to do with the fix for the previous color-swap issue we had.

Thanks,

zaros

andres

Re: GSVideo: Playing h264 videos with Fluendo codecs

2 years ago

So you can at least get smooth h264 playback at 1080p using GSMovie? If that's the case, sounds great.

As for the issues with playbin2/ mkv container , the fact that it doesn't work well even with gst-launch from the command line might indicate it is a problem somewhere in gstreamer or the fluendo codecs. You should contact fluendo about this.

About the problems with GSPipeline and the mp4 files... not sure about the reason (I don't have the fluendo codecs so i cannot test myself). Did you try lower resolution videos? Also, you could try with earlier versions of gsvideo that didn't include the color swap fix, to see if this really has something to do with the problem.

zaros.adl

Re: GSVideo: Playing h264 videos with Fluendo codecs

2 years ago

Thanks for the feedback. I'll ask Fluendo for support about the mkv issue.

I made a structured series of tests today. Here are the findings:

Base hardware: MacMini4,1 with Kubuntu 10.10, fully updated. I had desktop effects switched off.

1. I tested with GSVideo-0.7 Stable. + Fluendo

-- color swap was present when I play a .wmv video with GSMovie

-- with GSMovie was unable to play my 1080p.mkv test file

-- with GSMovie playing 1080p.m4a test file it opened a new window and I get hardware accelerated playback. The colors were correctly displayed.

-- The results for this test are actually identical with test #2, but with the color swap issue manifesting with videos shown in Processing window.

2. Test with Fluendo + GSVideo 20110116

-- when playing 1080p & 720p.mp4's a new window opens on top of Processing. The window title is "GSVideo" so I guess that's good news: can get smooth 1080p h264 playback with GSVideo. The CPU was running at <30% most of the time. There were some decompression artifacts.

-- not able to play 1080p.mkv, but was able to play 720p.mkv but it also it opens a new window. However, after that my program stopped responding.

-- when I close these windows, the last frame remains on my desktop and I have to restart X to clear it.

-- made an additional test with a 720p.wmv file . Interestingly, GSMovie played as normal (no windows opened). I can't tell if this is the fluwmvcodec being used though... playbin2 autoselects this.

3. I removed the Fluendo codecs and tested with GSVideo-20110116

-- with GSMovie was able to play 1080p.mkv (~20fps), 1080p.mp4, 720p.mkv, 720p.mp4, 720.wmv

-- it didn't open a new window on top of processing when playing with mp4

4. Test with Fluendo codes + GSVideo-20110111

-- can play all files mentioned

-- 1080p h264 playback is around 18-25fps (Full CPU)

-- 720p h264 playback is about 50fps (CPU at about 80%)

-- I am assuming that the fluendo codecs are being used automatically but can't be sure: see test #5

5. I set up an identical hardware & config and tested side by side with Setup #4 but without Fluendo codecs

-- I assume the decoder being used was ffdec_h264

-- Setup #4 use slightly more CPU and memory than #5 when playing back any h264 video

-- There is no significant difference of performance and perceived quality of the videos being played back. Both setups look equally not-so-great, actually.

6. Used gst-launch pipe to test between ffdec_h264 and fluh264dec shows that the fluendo codec is superior

7. Removed gstreamer0.10-ffmpeg so it's more likely that the Fluendo codecs are used and ran test #4 again. Got same results.

-----------------------------------------------------------

Questions

1. I'm quite baffled why Setup #4 works without the video window opening up, when Setup #1 didn't.

And why there is no improvement between tests #4 thru #7.

-- Not knowing exactly what GSVideo is doing, To test the assumptions, How can I find out which decoder GSVideo is using when I load a file with GSMovie, during or after runtime?

2. I suspect that the new window opens because Fluendo uses its own fluvasink when performing hardware accelerated h264 decoding. Is it possible to fix this issue?

Thanks for your time.

Regards,

zaros

andres

Re: GSVideo: Playing h264 videos with Fluendo codecs

2 years ago

1. I think the significant difference here between the GSVideo 20110116 and all the previous versions is that the former is using playbin2 for playback inside GSMovie, while the older versions including 0.7 are using playbin. It seems from what you describe that the fluendo codecs get selected only by playbin2 and not by playbin.

All the codec selection is handled automatically by the playbin/playbin2 object inside GSMovie, I need to check if there are methods in the gstreamer-java API that allow to inspect the pipeline generated by the bin element.

2. I agree, if fluendo uses its own video sink element, then this is probably the reason why a second window is opened and the processing output is bypassed completely. Solving this would probably involve writing a new wrapping class in gstreamer-java to grab the frames from the fluendo video sink.

Top Reply