Hello discourse,

I'm trying to use image() to draw a movie that contains transparency and it does not work when using opengl. I'm doing this:

Code:

movie = new Movie(this, "mymovie.mov");
(...)
image(movie, 0, 0, 100, 100);

 

And when the movie is drawn, its background is set to black, as if the alpha channel was always set to 1.

Here's a few clues:
1. Using JAVA2D works, and the movie IS transparent where it should
2. Using set() or copy() produce the same result, with the movie transparent areas being opaque instead
3. Using image() with an image with transparency (PNG) works as intended

Here's where it gets weird though: if I try to copy the movie (as if it was an image) to another image, and then draw this new image, it works. Similarly, using alpha(get(x,y)) properly reads the transparenty of transparent pixels as being 0. So it's not a problem with the movie but apparently something with how image() accesses the image of a movie under OpenGL; alpha is ignored. So I'm doing this now:

Code:

movie = new Movie(this, "mymovie.mov");
(...)
imageBuffer = createImage(movie.width, movie.height, ARGB);
imageBuffer.copy(movie, 0, 0, movie.width, movie.height, 0, 0, movie.width, movie.height);
image(imageBuffer, 0, 0, 100, 100);

 

And it works fine. The problem, however, is that the additional copy() puts some additional strain in my execution - I already have a good number of videos I'm processing this way, and it's starting to make it slower than I'd like it to be. It works if I just shrink the video then draw it at the same screen size as before, but it loses quality as a result. And it's not something *too* crazy, it just has a number of small videos with transparency.

*Other* solution I'm looking into is using OpenGL to directly draw the image content, using the solution proposed here. I've modified the javaToNativeRGB function to preserve alpha so it also works and keeps alpha intact, and is pretty fast:

Code:

      PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;
      GL gl = pgl.beginGL();
      IntBuffer buf = BufferUtil.newIntBuffer(movie.width * movie.height);
      javaToNativeRGB(movie);
      buf.put(movie.pixels);
      buf.rewind();
      
      //gl.glPixelZoom(1, 1);
      gl.glRasterPos2i(__x, __y);
      gl.glDrawPixels(movie.width, movie.height, GL.GL_RGBA, GL.GL_UNSIGNED_BYTE, buf);
      pgl.endGL();

 

However, it introduces new problems, as it uses glDrawPixels() and that doesn't take interpolation into account (if I actually use glPixelZoom), *and* won't let me rotate the content, something I need for a few specific movies.

So my question is, is there anyway to use image() with a movie on OpenGL preserving alpha channel, *or* to have some way to write it fast to the screen? This is just 2D, no depth or anything needed... I'm really at a loss here, because anything I try is either too slow or just doesn't work as intended.

Any help is appreciated. Thanks.

I haven't been able to investigate where's the problem. However, for the lack of a definitive answer, here's some small things I did to remedy the problem.

First, instead of using copy() to transfer the Movie data to a PImage instance, I just copied the pixels[] array. It's faster this way. Like this:

Code:

imageBuffer.pixels = movie.pixels;
imageBuffer.updatePixels();
 

Second, the movie compression makes a huge difference in how fast the (decoding?) rendering works. I found that using Quicktime, with NO compression (compressor set as "NONE"), works best. Files are huge, but once loaded, they do perform best than smaller movies. That's expected to some degree, but I would usually have assumed access to huge-ass memory data would have negated the advantage of 0 decompression time. That's not the case.

Third, do not use movieEvent on the root pde of your sketch -- check for available() instead. This probably causes the movie framerate to slow down (instead of skipping so the time is accurate), but on my case it's ok.

And fourth and finally, having file sizes with dimensions of powers of two (128, 256, 512, 1024, etc), help immensely. This is also expected considering textures have to be uploaded with that kind of dimensions, but apparently Processing's auto conversion is pretty slow for that. So pre-rendering the movie file already using those dimensions help.

All in all this combination of tweaks make the image buffer technique actually useful. I can't say speed is as fast as I'd like, but at least it works as long as I'm willing to use movies with a lower resolution.

There's a class I'm using as a sort of a proxy for movie loading and playing. It works like this:

Code:

// Create movie
ZMovieBuffered whatever = new ZMovieBuffered (this, "whatever.mov");

// Start movie
whatever.loop();

void draw() {
    // Setup draw mode, like rect/image
    whatever.drawMode(CORNER);

    // Draw movie
    whatever.draw(x, y, w, h);
}

void movieEvent(Movie __movie) {
    // do nothing - leave this empty
    // (or else it's auto-created it seems)
    // zmovie checks for movie.available(),
    // all other movie rendering should too
}
 

It's not perfect but it works for me.

Here's the actual class:

Code:

class ZMovieBuffered {
  
  // Draws movies correctly in OpenGL (with transparency)


  // Properties -------------------------------------------

  PImage imageBuffer;
  Movie movie;
  boolean inited;
  int movieWidth;
  int movieHeight;
  int currentDrawMode;


  // Constructor ------------------------------------------ 

  ZMovieBuffered(PApplet __root, String __url) {
    movie = new Movie(__root, __url);
    inited = false;
    movieWidth = 0;
    movieHeight = 0;
  }

 
  // Methods ---------------------------------------------- 

  void loop() {
    movie.loop();
  }
  
  void drawMode(int __drawMode) {
    currentDrawMode = __drawMode;
  }

  void draw(int __x, int __y, int __width, int __height) {
    if (!inited && movie.available()) movie.read();

    if (!inited && movie.width > 1 && movie.height > 1) {
	imageBuffer = createImage(movie.width, movie.height, ARGB);
	movieWidth = movie.width;
	movieHeight = movie.height;
	inited = true;
    }
    
    if (inited) {
	// Create buffer if not executed this frame yet

	if (movie.available()) {
	  movie.read();
	  // Update transparency buffers
	  imageBuffer.pixels = movie.pixels;
	  imageBuffer.updatePixels();
	}

	imageMode(currentDrawMode);
	image(imageBuffer, __x, __y, __width, __height);
    }
    
  }

} 

Hopefully this helps someone else while there's no better way to render movies. One may have to add to the class though, as it lacks some movie methods like stop(), play() and such (my stuff loops endlessly).