Can I play a video in detected blobs??

mbvcloud

Can I play a video in detected blobs??

in Contributed Library Questions • 2 years ago

Can I have two layers of video, one that plays as the background image and one that plays in the OpenCV blob shapes? This is what I have so far.

Any suggestions is greatly appreciated! thank you in advance.

import hypermedia.video.*;

import processing.video.*;

OpenCV opencv;

Movie mov;

Movie movtwo;

PImage movie1;

PImage movie2;

void setup() {

size( 640, 480, P3D );

mov = new Movie(this, "homepage.mov");

mov.loop();

movtwo = new Movie(this, "changehomepage.mov");

movtwo.loop();

// open video stream

opencv = new OpenCV( this );

opencv.capture( 640, 480 );

}

void movieEvent(Movie m) {

m.read();

}

void draw() {

background (255);

image(mov, 0, 0);

opencv.read(); // grab frame from camera

opencv.threshold(40); // set black & white threshold

// find blobs

Blob[] blobs = opencv.blobs( 20, width*height/2, 200, true);

// draw blob results

for( int i=0; i<blobs.length; i++ ) {

//stroke(255,255,255,10); this works to fill the blob shapes with a color but I want to replace this with the second video

beginShape();

for( int j=0; j<blobs[i].points.length; j++ ) {

vertex( blobs[i].points[j].x, blobs[i].points[j].y );

vertex (320,160,0);

}

endShape(CLOSE);

}

Replies(7)

amnon.owed

Re: Can I play a video in detected blobs??

2 years ago

Try drawing the opencv blobs into a black and white pgraphic, then use that as a mask on the second movie.

mbvcloud

Re: Can I play a video in detected blobs??

2 years ago

Thanks for your suggestion.

I tried to draw the blobs into a graphic by using a buffer? I'm not sure if that was a good idea or not, then I tried to load the image as a mask but I still can't quite get it. Now the image of the blobs (black and white) kinda flashes randomly over the movie as it is playing. Any suggestions as to what I'm doing wrong? This is my first time learning about masks...

Thanks very much

import hypermedia.video.*;

import processing.video.*;

OpenCV opencv;

Movie mov;

Movie movtwo;

PImage movie1;

PImage movie2;

PGraphics buffer;//new

PImage blobcopy;//new

void setup() {

size( 640, 480, P3D );

buffer = createGraphics(640, 480, P3D);//new

mov = new Movie(this, "homepage.mov");

mov.loop();

movtwo = new Movie(this, "homepageinvert.mov");

movtwo.loop();

// open video stream

opencv = new OpenCV( this );

opencv.capture( 640, 480 );

}

void movieEvent(Movie m) {

m.read();

}

void draw() {

background (255);

image(mov, 640, 480);

opencv.read(); // grab frame from camera

opencv.threshold(40); // set black & white threshold

// find blobs

Blob[] blobs = opencv.blobs( 20, width*height/2, 200, true);

// draw blob results

for( int i=0; i<blobs.length; i++ ) {

//stroke(255,255,255,10); // this works to fill the blob shapes with a color but I want to replace this with the second video

for( int j=0; j<blobs[i].points.length; j++ ) {

buffer.beginDraw();

beginShape();

vertex( blobs[i].points[j].x, blobs[i].points[j].y );

vertex (320,160,0);

}

endShape(CLOSE);

buffer.endDraw();

blobcopy = buffer.get(0, 0,640, 480);

mov.mask(blobcopy);

image(mov, 0, 0);

}

amnon.owed

Re: Can I play a video in detected blobs??

2 years ago

Check your formatting and code, especially these:

First mov displayed at location 640, 480
No buffer in front of beginShape and vertex
Buffer should be outside the for loops
Beginshape-endShape should be outside the inner for loop
vertex (320,160,0) makes no sense
fill/stroke color within buffer not set
buffer has no background call so it will be an aggregate aka accumulative drawing
everything after that is still within the outer for loop (!)
no need to get the buffer
masking and displaying should be done on movtwo, not on mov

Anyway, I made an example of how you can play two movies and have a thresholded opencv shape be the mask between them. This code assumes both videos are 640x480. If they are not resize them both within draw (or outside Processing).

Code Example

import processing.video.*;
import hypermedia.video.*;
Movie mov, movtwo;
OpenCV opencv;
int threshold = 240;
void setup() {
size(640, 480, P3D);
mov = new Movie(this, "mov1.mov");
mov.loop();
movtwo = new Movie(this, "mov2.mov");
movtwo.loop();
opencv = new OpenCV( this );
opencv.capture( width, height );
}
void draw() {
opencv.read();
opencv.threshold( threshold, 255, OpenCV.THRESH_BINARY_INV + OpenCV.THRESH_OTSU );
image(mov, 0, 0);
movtwo.mask(opencv.image());
image(movtwo, 0, 0);
}
void movieEvent(Movie m) {
m.read();
}
public void stop() {
opencv.stop();
super.stop();
}

mbvcloud

Re: Can I play a video in detected blobs??

2 years ago

Wow, thanks a lot for your help! I guess I had a lot of mistakes.

Your feedback has been really helpful.

One thing I would like to ask about, the reason that I had the vertex (320,160,0) was because I need all the blogs to be a mask (just like the code you posted for me) but I also need the vertices of the blobs to draw a line to a common point on the screen (in this case at 320,160,0) so they look like transparent blobs with a lines pointing to that central point. the video underneath is an image of the ocean, and then the blobs (people) who are detected create a mask allowing the second layer of video which has the reflection of sunlight on the water so show through and this line of light points to where the sun is in the video. So they create the reflection of the light on the water that we see when we look at the sun over the ocean and the reflection follows them as they move. Is it possible to add this vertex into the code you provided? I looked up online about OpenCV.THRESH_BINARY_INV + OpenCV.THRESH_OTSU and found some information but it is a bit over my head. I also tried the suggestions you gave for altering and formatting my code but still can't quite get it.

I really appreciate your help very much.

amnon.owed

Re: Can I play a video in detected blobs??

2 years ago

All right, here's some more code. Note that it's all untested.

To bring back the lines from vertex-to-center you could add something like this at the end of draw...

Blob[] blobs = opencv.blobs( 20, width*height/2, 200, true);
stroke(255,transparencyValue); // where transparencyValue is between 0 and 255
for( int i=0; i<blobs.length; i++ ) {
beginShape(LINES);
for( int j=0; j<blobs[i].points.length; j++ ) {
vertex( blobs[i].points[j].x, blobs[i].points[j].y );
vertex (320,160,0);
}
endShape();
}

To make the movtwo layer transparent you could use something like...

tint(255,transparencyValue); // where transparencyValue is between 0 and 255
image(movtwo, 0, 0);

If you want to work with your original draw() then at least it should look more like this...

void draw() {
opencv.read();
opencv.threshold(40);
Blob[] blobs = opencv.blobs( 20, width*height/2, 200, true);
buffer.beginDraw();
buffer.background(0);
buffer.fill(255);
buffer.stroke(125);
for( int i=0; i<blobs.length; i++ ) {
buffer.beginShape();
for( int j=0; j<blobs[i].points.length; j++ ) {
buffer.vertex( blobs[i].points[j].x, blobs[i].points[j].y );
buffer.vertex (320,160,0);
}
buffer.endShape(CLOSE);
}
buffer.endDraw();
background (255);
image(mov, 0, 0);
movtwo.mask(buffer);
image(movtwo, 0, 0);
}

mbvcloud

Re: Can I play a video in detected blobs??

2 years ago

THANK YOU so very much amnon.owed,

the information you provided was extremely helpful.

I didn't know about the tint function or how to properly set up the buffer before. Thanks for explaining and taking the time to answer my post.

Best,

mbvcloud

Re: Re: Can I play a video in detected blobs??

2 years ago

Could I ask you two more questions please? When I run the application there is a lot of flashing between the two video layers, meaning that the second layer flashes on the screen (not only in the blobs but on the whole screen) quickly. Is there any way to address this?

Also, can I draw the blobs only on the bottom half of the screen? I found that if I add /2 after the y in the shape it draws the blobs only on the top half of the screen, but how to get them only on the bottom?

for( int j=0; j<blobs[i].points.length; j++ ) {

buffer.vertex( blobs[i].points[j].x, blobs[i].points[j].y /2 );

buffer.vertex (320,260,50);

}

buffer.endShape(CLOSE);

thanks so very much for all your help. I really appreciate it.

Best

this is the code I'm using now from your suggestions:

import hypermedia.video.*;
import processing.video.*;
OpenCV opencv;
Movie mov;
Movie movtwo;
PImage movie1;
PImage movie2;
PImage trailsImg; //new
float threshold = 80f;//new
PGraphics buffer;
void setup() {
size( 640, 480, P3D );
mov = new Movie(this, "watersun640_480.mov");
mov.loop();
movtwo = new Movie(this, "watersunreflect.mov");
movtwo.loop();
// open video stream
opencv = new OpenCV( this );
opencv.capture( 640, 480 );
trailsImg = new PImage (640,480);//new
buffer = createGraphics(640, 480, P3D);//new
}
void movieEvent(Movie m) {
m.read();
}
void draw() {
opencv.read();
opencv.threshold(40);
Blob[] blobs = opencv.blobs( 20, width*height/2, 200, true);
buffer.beginDraw();
buffer.background(0);
buffer.fill(255);
buffer.stroke(125);
for( int i=0; i<blobs.length; i++ ) {
buffer.beginShape();
for( int j=0; j<blobs[i].points.length; j++ ) {
buffer.vertex( blobs[i].points[j].x, blobs[i].points[j].y );
buffer.vertex (320,260,50);
}
buffer.endShape(CLOSE);
}
buffer.endDraw();
background (255);
image(mov, 0, 0);
movtwo.mask(buffer);
image(movtwo, 0, 0);
blend(0, 0, 640, 480, 0, 0, 640, 480, BLEND);
}

Top Reply