Video Libraries and Formats

akiersky

in Contributed Library Questions • 2 years ago

Hi All,

Im working on a project that is going to use many components, one of which will be video. I have experimented on a few of the video libraries out there and it seems like GSVideo is the most effective (not resource intensive).

I would like to be able to play an HD video file (1080 would be great, but would settle for 720) but I can't seem to get better than 15 fps playback. I currently am using an h264 compressed mov file, but have experimented with mp4, avi, m4v and ogg formats with various bitrates with similar results.

Does anyone have recommendations on compression settings to use or a library that will work more consistently?

thanks!!
ak

Replies(2)

akiersky

Re: Video Libraries and Formats

2 years ago

ok, so i have got this playing fairly well with a 720 hd quicktime compressed as JPEG. Im still using the GSVideo lib, but also using the GLGraphics lib with the GLConstants.GLGRAPHICS as the renderer. Using a technique of copying the frames to a texture mentioned here:

http://forum.processing.org/topic/gsvideo-poor-performance-with-opengl
and demonstrated here:
http://mechanicape.googlecode.com/svn/trunk/projects/mobilitymodule/software/processing/libraries/glgraphics/examples/HDMovieFilter/HDMovieFilter.pde

here's my code if anyone else is looking:

import processing.opengl.*;
import codeanticode.gsvideo.*;
import codeanticode.glgraphics.*;

GSMovie myMovie;
GLTexture tex;
PImage img;

void setup() {
size(960, 540, GLConstants.GLGRAPHICS);
background(0);
// frameRate(60);
tex = new GLTexture(this);
myMovie = new GSMovie(this, "movie.mov" );
myMovie.loop();
}

void movieEvent(GSMovie myMovie) {
myMovie.read();
}

void draw() {
background(255);
myMovie.loadPixels();
myMovie.updatePixels();
if ((1 < myMovie.width) && (1 < myMovie.height)) {
    tex.putPixelsIntoTexture(myMovie);
    image(tex, 0,0, 960, 540);
    fill(200);
    text("fr: "+int(frameRate), 10, 10, 50, 50);
}
}

I would still love to get it working with a full hd video, but this will do for now. Any other tips would be appreciated!

thanks,
ak

andres

Re: Video Libraries and Formats

2 years ago

You can still get some more performance by avoiding unnecessary texture copies. Basically, you should call putPixelsIntoTexture() only when you are sure a new video frame has been read. Take a look at the following code:

import processing.opengl.*;
import codeanticode.gsvideo.*;
import codeanticode.glgraphics.*;
GSMovie movie;
GLTexture tex;
boolean newfr;
int rcount, dcount;
int lastrm, lastdm;
float readfr, drawfr;
int frint = 1;
void setup() {
size(960, 540, GLConstants.GLGRAPHICS);
tex = new GLTexture(this);
movie = new GSMovie(this, "movie.mov");
movie.loop();
}
void movieEvent(GSMovie mov) {
mov.read();
newfr = true;
rcount += 1;
int m = millis();
if (m - lastrm > 1000 * frint) {
readfr = float(rcount) / frint;
rcount = 0;
lastrm = m;
}
}
void draw() {
if ((1 < movie.width) && (1 < movie.height) && newfr) {
tex.putPixelsIntoTexture(movie);
image(tex, 0, 0, 960, 540);
newfr = false;
text("read fr: "+ int(readfr), 10, 10);
text("draw fr: "+ int(drawfr), 10, 25);
}
dcount += 1;
int m = millis();
if (m - lastdm > 1000 * frint) {
drawfr = float(dcount) / frint;
dcount = 0;
lastdm = m;
}
}

I also put manual framerate calculations, since the internal frameRate variable might not be accurate all the time. Also, there are actually two different framerates you need to consider: the rate at which the read() function is called (which should be same as the original video framerate if no frames are being dropped) and the rate at which the draw() function is called (which can be higher than the read rate).

On OSX, using the new frame trick with full HD video (1080p), I gain several frames, but the read framerate is still lower than the original video's framerate at some points. This is also dependent on the platform: on Linux I'm able to play 1080p with no frames dropped, performance on OSX is a little worse, followed by Windows. Hardware will also have an effect.

Top Reply