cannot store video feed to image array

Core Library Questions

nose_1.1

cannot store video feed to image array

in Core Library Questions • 1 year ago

This is inspired by openframeworks code shared by James George.

I am basically trying to save frames from a video feed into an image array, and then choose how that image array will be shown.

I can get it to work, almost, but it seems that the images which I am saving are all turning out the same. I can do the same thing with opencv, but it hiccups and does not work very well.

Here is the gist of the code:

import processing.video.*;
Capture video;
int frameNumber = 0; //start at frame 1
int fBM = 60; //fBM = frameBufferMaximum
PImage img;
PImage[] frameBuffer = new PImage[fBM+1]; //store 60 frames
void setup() {
size(800, 480, P2D);
// Uses the default video input, see the reference if this causes an error
video = new Capture(this, 640, 480);
video.start();
background(0);
img = createImage(video.width,video.height,RGB);
//prepare the image array
for (int i=0;i<fBM;i++){
frameBuffer[frameNumber]=createImage(video.width,video.height,RGB);
}
}
void draw() {
if (video.available()) {
video.read();
video.loadPixels();
//copy the video frame
img.copy(video,0,0,video.width,video.height,0,0,video.width,video.height);
//store the video frame
frameBuffer[frameNumber]=img;
frameBuffer[frameNumber].updatePixels();
frameNumber ++;
if (frameNumber > fBM) {
for (int i=0;i<fBM-1;i++){
//push frames down the array
frameBuffer[i]=frameBuffer[i+1];
frameNumber = fBM;
}
}
}
//draw the images on screen
for (int i=frameNumber;i>0-1;i--){
if((frameBuffer[i] != null)){
image(frameBuffer[i],width-2*i,0,100,height);
}
}
}

This should be fairly straight forward, but it seems all of the images are the same, current frame of video being shown. It should be generating a relative video delay with this process, but it doesn't work that way.

I've searched several forums, but cannot figure out how to get a video frame to be copied as an image for later use. It's very perplexing. Any help would be appreciated.

Replies(6)

PhiLho

Re: cannot store video feed to image array

1 year ago

//copy the video frame
img.copy(video,0,0,video.width,video.height,0,0,video.width,video.height);
//store the video frame
frameBuffer[frameNumber]=img;

You take an unnecessary step, and all you do is putting the reference of the img PImage in all the entries of the frameBuffer, so indeed they point to the same image. That's a common error, where people think that = does a copy of the data. Although you had a good idea with the img.copy(), but as I wrote, you took an extra step ruining the idea.

A possibly better way (I have no video capture device to test) can be to replace these lines with:
frameBuffer[frameNumber].copy(video,0,0,video.width,video.height,0,0,video.width,video.height);
Ie. do the copy of the frame directly in the right buffer.

cedrickie..

Re: cannot store video feed to image array

1 year ago

i actually dont know how the sketch should look like. maybe you can show a demo of the example that you are trying to replicate. but copying the video to an image seems to work, if you simply put image(img,0,0);

at the end of your draw loop, you see the video updating. so the problem might be somewhere else.

nose_1.1

Re: cannot store video feed to image array

1 year ago

Well, PhiLho, the reason I did the copy for img was that by just setting frameBuffer(n)=video meant I was creating another instance of the video feed... Or camera output. I got the same result, every instance of frameBuffer(0-n) became the current video frame. Doing frameBuffer.copy in the same format resulted in a long, complicated error that was avoided by using img.copy.

Unless I am misinterpreting what you are saying, and that changing a SINGLE entry of frameBuffer with ANY image will automatically change ALL instances of that ENTIRE array, even if you do not step through those individual instances? I'm fairly certain that isn't the way arrays should work, especially since it works properly with an opencv alternative (which has lower framerates)

But copying to an image buffer before storing in an array didn't work either. There is something messed up here. I need some way of just getting the pixel information out of the video feed to store in a single instance of frameBuffer(n).

Cedrickie, you are right, imaging it in that way will work. However, if I place an image of frameBuffer(0) and img beside each other, they will be the same. They should be out of sync. There is a step I am missing here, or the video library is not working the way it should for storing arrays of images.

If you search for James George SlitScan you will find a video or two, and possibly a link to the downloadable executable on his website. My goal is somewhat more modest, given the limitations of processing. I just want to rotate the images stored n frameBuffer in the y axis, and have them straighten out where the mouse is... It's part of a school,project... It looks really nice in the opencv version, but pokey with low framerates...

PhiLho

Re: cannot store video feed to image array

1 year ago

" Unless I am misinterpreting what you are saying, and that changing a SINGLE entry of frameBuffer with ANY image will automatically change ALL instances of that ENTIRE array, even if you do not step through those individual instances? I'm fairly certain that isn't the way arrays should work"
That's exactly what happens, and it works as expected when you fill an array with always the same value, the reference of the image pointed by the img variable.
img only holds one value, the reference to the PImage created in setup(). You create an instance of this class, which lives somewhere in memory, and img points to this place.
When you write frameBuffer[frameNumber]=img; you only put in the entry of frameBuffer this reference. So all the array holds is the same reference.
That's why I recommend to do the copy directly in the entry, since you initialized the array with instances of Pimage anyway. These instances are lost anyway, since you overwrite the reference to these objects.

" Doing frameBuffer.copy in the same format resulted in a long, complicated error that was avoided by using img.copy."
Perhaps you can show this error message...

nose_1.1

Re: cannot store video feed to image array

1 year ago

okay. getting rid of the whole img.copy thing...

replaced with:

frameBuffer[frameNumber].copy(video,0,0,video.width,video.height,0,0,video.width,video.height);

I get a null pointer exception that is highlighting this particular line.

The rest of that error displayed in the console is:

Exception in thread "Animation Thread" java.lang.NullPointerException

at slitscan2.draw(slitscan2.java:73)

at processing.core.PApplet.handleDraw(PApplet.java:2120)

at processing.opengl.PGL$PGLListener.display(PGL.java:2563)

at jogamp.opengl.GLDrawableHelper.displayImpl(GLDrawableHelper.java:189)

at jogamp.opengl.GLDrawableHelper.display(GLDrawableHelper.java:177)

at javax.media.opengl.awt.GLCanvas$DisplayAction.run(GLCanvas.java:928)

at jogamp.opengl.GLDrawableHelper.invokeGLImpl(GLDrawableHelper.java:425)

at jogamp.opengl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:364)

at javax.media.opengl.awt.GLCanvas$DisplayOnEventDispatchThreadAction.run(GLCanvas.java:945)

at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:199)

at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:702)

at java.awt.EventQueue.access$400(EventQueue.java:82)

at java.awt.EventQueue$2.run(EventQueue.java:663)

at java.awt.EventQueue$2.run(EventQueue.java:661)

at java.security.AccessController.doPrivileged(Native Method)

at java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)

at java.awt.EventQueue.dispatchEvent(EventQueue.java:672)

at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:296)

at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:211)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:201)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:196)

at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:188)

at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

When you write frameBuffer[frameNumber]=img; you only put in the entry of frameBuffer this reference. So all the array holds is the same reference.

so how can I copy the actual image that is in img, not the reference that this is pointing to. I just want the image...

If I use the following it seems to work:

frameBuffer[frameNumber]=video.get(0,0,video.width,video.height);

PhiLho

Re: cannot store video feed to image array

1 year ago

It might be because you created fBM+1 entries, but only initialized fBM of them. When you set frameNumber to fBM, you point to the last entry which is empty.

Top Reply