mixing multiple masked videos using GSVideo and GLGraphics

paolop

mixing multiple masked videos using GSVideo and GLGraphics

in Contributed Library Questions • 2 years ago

Hi everybody

I am using GSVideo and GLGraphics libraries to make a dynamic collage of videos that are masked using textures that are generated in real time...

I tried both on Ubuntu and Win 7 32 bit and I am using x264 encoded videos (576p).

but i don't understand why this piece of code won't work:

Copy co

import processing.opengl.*;
import codeanticode.glgraphics.*;
import codeanticode.gsvideo.*;
//number of video clips
int nclips = 3;
GSMovie [] clips = new GSMovie[nclips];
GLTexture [] textures = new GLTexture [nclips];
//GLTexture [] maskedTextures = new GLTexture [nclips];
GLGraphicsOffScreen [] masks = new GLGraphicsOffScreen [nclips];
//i'll use this to draw the mask
int [] fibonacci = {
1, 1, 2, 3, 5, 8, 13, 21, 34, 55
};
int x, y, l;
int rf, rx;
int module=10;
int f = 0;
GSMovieMaker mm;
void setup() {
size(1024, 576, GLConstants.GLGRAPHICS);
background(0);
// Use texture tex as the destination for the movie pixels.
for (int i =0; i<nclips; i++) {
textures[i] = new GLTexture(this);
clips[i] = new GSMovie(this, i+".mp4");
clips[i].setPixelDest(textures[i]);
clips[i].loop();
masks[i] = new GLGraphicsOffScreen(this, width, height);
// maskedTextures[i] = new GLTexture(this);
masks[i].beginDraw();
masks[i].background(0);
masks[i].endDraw();
}
mm = new GSMovieMaker(this, width, height, "field_video.ogg", GSMovieMaker.THEORA, GSMovieMaker.BEST, 10);
mm.setQueueSize(50, 10);
mm.start();
}
void movieEvent(GSMovie movie) {
clips[f].read();
}
void draw() {
// If there is a new frame available from the movie, the
// putPixelsIntoTexture() function will copy it to the
// video card and will return true.
//System.gc();
masks[f].beginDraw();
masks[f].noStroke();
rf = floor(random(0, fibonacci.length));
l = module*fibonacci[rf];
rx = floor(random(0, ((width)/5)));
x=rx*module;
y=0;
masks[f].fill(255, 255/(fibonacci[rf]*0.25));
masks[f].rect(x, y, l, height);
rf = floor(random(0, fibonacci.length));
l = module*fibonacci[rf];
rx = floor(random(0, ((width)/5)));
x=rx*module;
y=0;
masks[f].fill(0);
masks[f].rect(x, y, l, height);
masks[f].endDraw();
if (textures[f].putPixelsIntoTexture()) {
textures[f].updateTexture(); // copy texture to pixels
textures[f].mask(masks[f]);
textures[f].loadTexture(); // copy pixels to texture
image(textures[f], 0, 0);
}
loadPixels();
mm.addFrame(pixels);
println(frameCount);
if (frameCount == 5000) {
// Finish the movie
mm.finish();
// Quit running the sketch once the file is written
exit();
}
}
}

it gives me this error:

0:00:08.290474000 4532 7AD0DB90 ERROR ffvideoscale .:0:: no frame!

0:00:08.290474000 4532 7AD0DB90 ERROR ffvideoscale .:0:: get_buffer() failed (-1 0 0 00000000)

0:00:08.290474000 4532 7AD0DB90 ERROR ffvideoscale .:0:: decode_slice_header error

do you have any idea? maybe this is not the best way to mix videos dynamically?

thanks a lot

paolo

Replies(3)

andres

Re: mixing multiple masked videos using GSVideo and GLGraphics

2 years ago

The error seems to be originated from the video decoder in gstreamer. Does it still happen if you disable the masking functionality?

paolop

Re: Re: mixing multiple masked videos using GSVideo and GLGraphics

2 years ago

thanks for your answer!
commenting out the masking part:

if (textures[f].putPixelsIntoTexture()) {
// textures[f].updateTexture(); // copy texture to pixels
// textures[f].mask(masks[f]);
// textures[f].loadTexture(); // copy pixels to texture
image(textures[f], 0, 0);
}

it gives OutOfMemoryError

0:00:01.434082000 3568   7A318F60 ERROR           ffvideoscale .:0:: get_buffer() failed (-1 0 0 00000000)

0:00:01.434082000 3568   7A318F60 ERROR           ffvideoscale .:0:: decode_slice_header error

0:00:01.434082000 3568   7A318F60 ERROR           ffvideoscale .:0:: no frame!

my laptop has 4 gb of RAM (with Win 7 32 bit i can use up to 3 gb) but i can't increase the maximum available memory in the preferences beyond 1600 mb...

andres

Re: mixing multiple masked videos using GSVideo and GLGraphics

2 years ago

One problem I see in your code is in the movieEvent() method, where you call read() on clips[f]. This is incorrect, you have to do it on the GSMovie object that is passed as an argument to the method:

void movieEvent(GSMovie movie) {
movie.read();
}

This might be related to the crash you see in your code.

Another important thing to keep in mind (but probably not related to the crash) is that only the blue channel of the image mask is used for the masking, so you should paint your mask image using grayscale colors:

masks[f].beginDraw();
...
masks[f].fill(255/(fibonacci[rf]*0.25));
...
masks[f].fill(0);
masks[f].rect(x, y, l, height);
...

Top Reply