Using several 2D primitives as mask when masking one video out of another

hjesper

Hi there,

I am trying to use different moving 2D primitives to mask out a video and play it on top of another video.

Here is how far I have gotten until now:

import processing.video.*;

Movie myMovie;
float a;
float b;
float c;
float d;
float theta = 0.0;
int threshold = 240;

void setup() {
  size(640, 360);
  myMovie = new Movie(this, "station.MOV");
  myMovie.loop();
  myMovie.volume(0);
  a = height/2;
}

void draw() {
  background(55);
  image(myMovie, 0, 0, width, height);

  Mask();
}

void Mask() { 
  float x = (sin(theta) + 1) * width/8; 
  theta += 0.005;

  //horizontal lines
  stroke(255);
  strokeWeight(1);
  line(0, a, width, a);  
  a = a - 0.2;
  if (a < 0) { 
    a = height;
  }

  strokeWeight(5);
  line(0, b, width, b);  
  b = b + 0.1;
  if (b > height) { 
    b = 0;  
  }

  //Vertical lines
    noStroke();
  fill(155);
  rect(c, 0, 50+x, height);
  c = c -0.2;
  if (c < -50-x) { 
    c = width;  
  }

  noStroke();
  fill(255);
  rect(d, 0, (10+x), height);
  d = d +0.3;
  if (d > (width+100)) { 
    d = -x;  
  }

}
void movieEvent(Movie m) {
  m.read();
}

So as you can see I am running a video as a background. On top of this video there are several moving white and grey rectangles and lines.

My question is: How can I use the primitives all together to mask out a second video?

I would like to do it in a way that allows transparency in relation to the brightness of the mask (50% gray = 50% transparent).

Hope my question makes sense.

Best regards

_vk

Google is your friend

https://www.google.com/search?client=safari&rls=en&q=mask++site:processing.org&ie=UTF-8&oe=UTF-8#q=mask+video++site:processing.org

https://forum.processing.org/one/topic/using-rects-as-masks.html

https://forum.processing.org/one/topic/image-behind-image.html

hjesper

Thank you very much _vk!

I feel like I have been googling forever. Sometimes i find it really hard to know what to search for, but thanks for pointing me in the right direction.

I got a my masks and videos up and running, but there seems to be some issues. There is a lot of flickering back and forth between the 2 videos, and a I get a very long list of errors that starts with:

java.lang.reflect.InvocationTargetException

import processing.video.*;

Movie myMovie;
Movie myMovie2;
float a;
float b;
float c;
float d;
float theta = 0.0;
PGraphics p;

void setup () {
  frameRate(30);
  size (400, 300, P2D);

  myMovie = new Movie(this, "IMG_2600.MOV");
  myMovie.loop();
  myMovie.volume(0);
  myMovie.resize(400, 300);
  myMovie2 = new Movie(this, "IMG_3709.MOV");
  myMovie2.loop();
  myMovie2.volume(0);
  myMovie2.resize(400, 300);
  a = height/2;
  p =  createGraphics(400, 300, P2D); 
  background(255);
}

void draw() {
  float x = (sin(theta) + 1) * width/8; 
  theta += 0.005;

  background(255);
  image(myMovie, 0, 0, width, height);

  p.beginDraw();

  p.background(0);

  p.stroke(255);
  p.strokeWeight(1);
  p.rect(0, a, width, 10+x);  
  a = a + 0.2;
  if (a > height+10+x) { 
    a = 0-10-x;
  }

  p.fill(155);
  p.rect(c, 0, 50+x, height);
  c = c -0.2;
  if (c < -50-x) { 
    c = width;
  }

  p.fill(255);
  p.rect(d, 0, (10+x), height);
  d = d +0.3;
  if (d > (width+100)) { 
    d = -x;
  }

  p.endDraw();

  myMovie2.mask(p);
  image(myMovie2, 0, 0, width, height);

  //press mouse to see only the pgraphics draw displayed
  if (mousePressed)image(p, 0, 0);
}
void movieEvent(Movie m) {
  m.read();
}

I tried setting different a lower framerate but that didn't help. It looks like theres an issue with what is gets drawn on top of what and in what order.

Thanks again!

GoToLoop

• Unless you're gonna mix 3D drawings along w/ your videos, stick w/ renderer JAVA2D.
• Using resize() on a Movie or Capture object is much probably moot.
• Overloaded image() w/ 5 parameters is slower. smooth() affects its resize quality too.
• set() & background() are faster than image() when using renderer JAVA2D.
• It's a good plan to delay() in setup() till the Movie object gets its width & height:
  while (myMovie.width == 0 | myMovie2.width == 0) delay(100);
• Then use surface.setSize(myMovie.width, myMovie.height); for Processing 3.
• Or size(myMovie.width, myMovie.height); for Processing 1 & 2.
• Doing so removes the need for 5-parameter image(). And you can even use background()! *-:)

hjesper

Wow thanks, I am not sure I understand what you just wrote, but I will try to look it up in the reference.

Thanks GoToLoop!!

GoToLoop

There is a lot of flickering back and forth between the 2 videos, ...
It looks like there's an issue with what is gets drawn on top of what and in what order.

• As you may already know, each Movie runs under their own Thread.
• And sketch's draw() runs under the "Animation" Thread.
• So while we're displaying some Movie in draw(), its actual pixels[] may happen of being updated under Movie's Thread at that exact moment from time to time! @-)
• For most cases such concurrency execution doesn't cause so many glitches/bugs.
• As long as only 1 Thread has exclusivity over modifying Movie's content.
• Unfortunately for your particular case, you're also concurrently modifying myMovie2 via mask().
• In order to avoid that race condition between "Animation" & myMovie2's threads, we need to put in place some protection "barrier" over mask()'s execution.
• Java has a keyword called synchronized () which does just that. :>
• Take notice that read(), loadPixels() and other critical methods in class Movie are already synchronized. You only need to "protect" mask() for the whole "fix" to kick in. :ar!
• In order for code blocks to be actually protected by synchronized (), we have to choose 1 shared object reference to be their "gatekeeper".
• Since class Movie already relies on this reference for their own internal synchronized methods, we just need to follow suit.
• That is, if we also need to modify myMovie2 under the "Animation" Thread in such a way that read() and other critical methods are temporarily blocked while mask() is busy (and vice-versa), we've gotta use myMovie2 itself as the "gatekeeper" (a.K.a. mutex) for synchronized ().
• B/c myMovie2 is the current's this object reference inside class Movie. :-j
• And to make sure image() actually displays the mask()-modified myMovie2, it's not a bad idea to include it inside the same synchronized () block too! :P

// ...

p.endDraw();

synchronized (myMovie2) {
  myMovie2.mask(p);
  image(myMovie2, 0, 0);
}

// ...

hjesper

Thanks for your kind help GoToLoop!

I managed to get it working, but it is not as efficient as I would like it to be.

Have you got some recommendations on how to optimize performance?

I am sorry if I didn't manage to implement all of your previous suggestions.

I am using Processing 3.

import processing.video.*;

Movie myMovie;
Movie myMovie2;
float a;
float b;
float c;
float d;
float theta = 0.0;
PGraphics p;

void setup () {
  frameRate(24);
  size(1920, 1080, JAVA2D);

  myMovie = new Movie(this, "367852631.mp4");
  myMovie.loop();
  myMovie.volume(0);
  myMovie2 = new Movie(this, "343380914.mp4");
  myMovie2.loop();
  myMovie2.volume(0);
  a = height/2;
  p =  createGraphics(myMovie.width, myMovie.height, JAVA2D); 
  while (myMovie.width == 0 | myMovie2.width == 0)  delay(100);
  background(255);
}

void draw() {
  float x = (sin(theta) + 1) * width/8; 
  theta += 0.005;

  background(myMovie);
  //image(myMovie, 0, 0, width, height);

  p.beginDraw();

  p.background(0);

  p.stroke(255);
  p.strokeWeight(1);
  p.rect(0, a, width, 10+x);  
  a = a + 0.2;
  if (a > height+10+x) { 
    a = 0-10-x;
  }

  p.fill(155);
  p.rect(c, 0, 50+x, height);
  c = c -0.2;
  if (c < -50-x) { 
    c = width;
  }

  p.fill(255);
  p.rect(d, 0, (10+x), height);
  d = d +0.3;
  if (d > (width+100)) { 
    d = -x;
  }

  p.endDraw();

  synchronized (myMovie2) {
    myMovie2.mask(p);
    image(myMovie2, 0, 0);
  }
}
void movieEvent(Movie m) {
  m.read();
}

GoToLoop

Although there's always some tuning here & there, I don't see anything obvious which could actually increase performance further.
However, since you're using Processing 3, you definitely should try newest FX2D renderer out, for both size() & createGraphics().