GSPipeline refresh

akiersky

in Contributed Library Questions • 1 year ago

Hi All!

Working on a project with live video streams, came to decide that GSVideo GSPipeline and VLC's Gstreamer functionality would be a good workflow. Eventually ill set it up using vlc's command line tools to streamline the process.

After a lot of (somewhat blind) trial and error I found a solution that actually works! I start up a capture stream in vlc, grabbing my screen at 5fps, and sending through http on my local ip address, encapsulated in MPEG 1, with the video transcoded as mp2v 512kbps. This stream is then captured in processing with the GSPipeline Capture example with this string:

pipeline = new GSPipeline(this, "wininetsrc location=http://xx.xx.xx.xx:1234 ! mpegdemux ! mpeg2dec ! ffmpegcolorspace");

And I can see my mac screen on my pc!

However, it seems that the refresh rate is somewhat inconsistant sometimes I see changes within a half second, other times it is more like 6-8 seconds before updates, sometimes even longer.

Are there adjustments to either my vlc server or the processing client that could make for a more reliable solution? I think in the end I will put the pipeline reading and updating on a seperate thread to reserve power for other parts of the application, but I would like a closer to realtime video stream if possible.

Thanks!

Replies(2)

akiersky

Re: GSPipeline refresh

1 year ago

Ok,

So I tried a few other codecs and settings and found that the ogg format works much better!

My new settings for VLC are sending the stream via http, same as before, but the encapsulation method is OGG with theo (theora) transcode for the video at 32kb/s (kilo-BITS!) which streams at very nice quality 15-20 fps out of VLC.

Then my new pipeline string is:

pipeline = new GSPipeline(this, "wininetsrc location=http://xx.xx.xx.xx:1234 ! oggdemux ! theoradec ! ffmpegcolorspace");

and the stream is now in processing, and much more responsive overall, about 1second delay average, than the mpeg settings. Plus the quality is better! win win!

Now with further testing I am finding that if I close the vlc server connection, GSVideo sends an error:

2:41:04.037751000 5664 241D5A40 ERROR wininetsrc gstwininetsrc.c:370:gst_win_inet_src_create:<wininetsrc0> short read (eof?)

Which is expected because the stream source ended. But I would like to be able to catch this error, and respond to it. Right now it just seems to kill the stram, starting the server again does not update the client. Ideally I would be able to catch the error, display some kind of loading message, then restart the stream once it is found.

Do I need to set up the pipeline again? how would I catch the error? Is this possible or if I lose the connection I need to restart the app?

Thanks!

andres

Re: GSPipeline refresh

1 year ago

GSPipeline has a isPlaying() method, but it is not useful in this case because it just keeps track of calls to the play()/stop()/pause() methods. If the native gstreamer pipeline stops, this won't update the value returned by GSPipeline.isPipeline().

However, GSPipeline is basically a wrapper to a Pipeline object that interfaces directly with the native gstreamer library. Pipeline has its own isPlaying(), which might accurately report the playing status of the stream. This internal object is protected, but you could define a subclass of GSPipeline that just gives access to it:

import org.gstreamer.Pipeline;
CustomPipeline stream;
void setup() {
stream = new CustomPipeline(...);
}
void draw() {
...
if (!stream.getPipeline().isPlaying()) {
println("Stream is stopped");
}
}
class CustomPipeline extends GSPipeline {
CustomPipeline(PApplet parent, String pipelineString) {
super(parent, pipelineString);
}
Pipeline getPipeline() {
return gpipeline;
}
}

If this approach doesn't work, you could just keep track of the last time when the pipelineEvent() function was called, and if the time interval goes beyond some threshold, you could reinitialize your pipeline object.