Hi everyone:

I'm having some serious performance troubles with my program, may be someone can help me a little bit with this.

What I'm doing:

I'm trying to do a kind of live video mixing based on chroma keying. I load 9 videos from files, and by mouse clicks I want these videos to be shown in different order (in layers)

Color of the chroma keying is for now, a constant for each video/layer.

btw, I've already tried GSmovie, but I found the native movie library better since I want to use some h264 videos.

So, what  my little program does, is to play 9 videos at same time. These 9 video objects are in an array. The movieEvent() gets the frames for all the videos and puts them in an array of 9 PImage objects.

In the draw method I'm making a mask for each frame, based on color constants already setted at the beginning of the program.

After that, what I do is to mix all the 9 frames based in the masks, in the order that the array ''order[]'' says. (this thing about order is not important for now, I use it because I want that the order of the videos can be changed at run time)

thanks in advice

viko
Code:

import processing.video.*;
public static final int VIDEOS = 8;
int colorsEnd;
PImage colorsBar;

Movie[] myMovie = new Movie[VIDEOS];
PImage[] img = new PImage[VIDEOS];
PImage[] imgMask = new PImage[VIDEOS];

int[] order = new int[VIDEOS];
int[] R = new int[VIDEOS];
int[] r = new int[VIDEOS];
int[] G = new int[VIDEOS];
int[] g = new int[VIDEOS];
int[] B = new int[VIDEOS];
int[] b = new int[VIDEOS];

void setup()
{
  colorsBar = loadImage("square_002.gif"); //btw, that image is not a sqare but a rectangle
  colorsEnd = colorsBar.width;
  //i know 145 = colorsEnd 
  size(576+145, 576, P3D);
  background(0);
  frameRate(200);
    
  myMovie[0] = new Movie(this, "zoo02_576x576_h264_peces.blanco.negros.mp4");  
  R[0] = 250;
  r[0] = 1;
  G[0] = 254;
  g[0] = 1;
  B[0] = 254;
  b[0] = 1;

  myMovie[1] = new Movie(this, "zoo02_576x576_h264_royalturkey.mp4");
  R[1] = 200;
  r[1] = 100;
  G[1] = 200;
  g[1] = 100;
  B[1] = 200;
  b[1] = 100;

  myMovie[2] = new Movie(this, "zoo01_576x576_h264_peces+patos.mp4");
  R[2] = 200;
  r[2] = 100;
  G[2] = 200;
  g[2] = 100;
  B[2] = 200;
  b[2] = 100;

  myMovie[3] = new Movie(this, "campo_576x576_h264_nicovacas1.3.mp4");
  R[3] = 200;
  r[3] = 100;
  G[3] = 200;
  g[3] = 100;
  B[3] = 200;
  b[3] = 100;
  
  myMovie[4] = new Movie(this, "campo_576x576_h264_nicocaballos1.1.mp4");
  R[4] = 200;
  r[4] = 100;
  G[4] = 200;
  g[4] = 100;
  B[4] = 200;
  b[4] = 100;

  myMovie[5] = new Movie(this, "campo_576x576_h264_corderos1.6.mp4");
  R[5] = 200;
  r[5] = 100;
  G[5] = 200;
  g[5] = 100;
  B[5] = 200;
  b[5] = 100;

  myMovie[6] = new Movie(this, "zoo01_576x576_h264_comadre1.0.mp4");
  R[6] = 200;
  r[6] = 100;
  G[6] = 200;
  g[6] = 100;
  B[6] = 200;
  b[6] = 100;

  myMovie[7] = new Movie(this, "zoo01_576x576_h264_flamingos2.mp4");
  R[7] = 200;
  r[7] = 100;
  G[7] = 200;
  g[7] = 100;
  B[7] = 200;
  b[7] = 100;  

  for ( int i = 0 ; i < VIDEOS ; i++ )
  {
    myMovie[i].loop();
  }
  for ( int i = 0 ; i < VIDEOS ; i++ )
  {
    img[i] = new PImage(width-colorsEnd, height);
    imgMask[i] = new PImage(width-colorsEnd, height);
    order[i] = i;
  }
  loop();
}


void movieEvent( Movie m ) 
{
  for ( int k = 0 ; k < VIDEOS ; k++ )
  {
    if ( m == myMovie[k] )
    {
      myMovie[k].read();
      img[k] = m;
    }
  }
}


void draw()
{
 /////////////start mask maker/////////////////////////////////////////// 
  for ( int k = 0 ; k < VIDEOS ; k++ )
  {
      for (int j = 0; j < height; j++)
      {
        for (int i = 0; i < width-colorsEnd; i++)
        {
          float current_r = (img[k].get(i,j)) >> 16 & 0xFF;
          float current_g = (img[k].get(i,j)) >> 8 & 0xFF;
          float current_b = (img[k].get(i,j)) & 0xFF;
          if (     ( current_r > r[k] && current_r < R[k] ) 
            && ( current_g > g[k] && current_g < G[k] )
            && ( current_b > b[k] && current_b < B[k] ))
          {
            imgMask[k].set(i,j,0xFFFFFF);
          }
          else
          {
            imgMask[k].set(i,j,0x000000);
          }
        }
    }
  }
/////////////end mask maker////////////////////////////////////////////

  image(img[(order[0])], colorsEnd, 0);
  image(colorsBar, 0, 0);
  loadPixels();
  for ( int v = 0 ; v < VIDEOS ; v++ )
  {
    for( int j = 0 ; j < height ; j++ )
    {
      for( int i = 0 ; i < width-colorsEnd ; i++ )
      {
        if( img[(order[v])].get(i,j) > imgMask[(order[v])].get(i,j) )
        {
          pixels[ ( ( j * (width) ) + i + colorsEnd ) ] = img[v].get(i,j);
        }
      }
    }
  }
  updatePixels();
}


void mousePressed()
{
  print("\nx="+mouseX+"  y="+mouseY+"\n");
  int division = 60, clicked = 0;
  for ( int j = 0 ; j < VIDEOS ; j++ )
  {
    print("order["+j+"]= "+order[j]+"\n");
  }
  if( mouseX < colorsEnd )
  {
    if( ( mouseY > 0 && mouseY < division ) )
    {
      clicked = 0;
    }
    else if( ( mouseY > division && mouseY < 2*division ) )
    {
      clicked = 1;
    }
    else if( ( mouseY > 2*division && mouseY < 3*division ) )
    {
      clicked = 2;
    }
    else if( ( mouseY > 3 * division && mouseY < 4 * division ) )
    {
      clicked = 3;
    }
    else if( ( mouseY > 4 * division && mouseY < 5 * division ) )
    {
      clicked = 4;
    }
    else if( ( mouseY > 5 * division && mouseY < 6 * division ) )
    {
      clicked = 5;
    }
    else if( ( mouseY > 6 * division && mouseY < 7 * division ) )
    {
      clicked = 6;
    }
    else if( ( mouseY > 7 * division && mouseY < 8 * division ) )
    {
      clicked = 7;
    }
    int aux = order[VIDEOS-1];
    order[VIDEOS-1] = clicked;
    for( int i = 0 ; i < VIDEOS ; i++ )
    {
      if ( order[i] == clicked )
      {
        order[i] = aux;
        break;
      }
    }
  }
  for ( int j = 0 ; j < 8 ; j++ )
  {
    print("order["+j+"]= "+order[j]+"\n");
  }
}
 

hi noah, thanks for reply. it was really usefull for me. i have made some changes:
i'm looping some shorter, since I applied your recomendations.

i've replaced

for videos (9)
   for width (576)
       for height (576)

with

for width (576)
   for height (576)
       for videos (9 or less)

since in this way, last 'for' (videos) could stop looping before it run entirely


also, i've tried accesing by pixels[] instead of get(). it was a little better, but then i thought in doing everything without PImage, just with primitive arrays.

i've done it. but it's still not working WELL.

i'm not sure if not using PImage is suposed to do it better or worse :S

moreover, i don't know either, with which value should i set framerate


i post the new code

again, thanks in advice

Without spending too much time on it, a couple ideas:
In movie event, Movie m is a reference to the movie that called the function.

In other words, you can use "m.read()" instead of "myMovie[k].read()".
This lets you remove a short loop, but then, that loop is run every frame of every video.

When using a bunch of loops, indirection can be costly.
i.e.
Code:

for ( int i = 0 ; i < SIDE*SIDE ; i++ ) {
  myColor = m.pixels[i];
  rojo[k][i] = (myColor >> 16 & 0xFF);
  verde[k][i] = (myColor >> 8 & 0xFF);
  azul[k][i] = (myColor & 0xFF);
} 

Might be rewritten as
Code:

//since k won't change during this loop
// and since int[][] is an array of arrays
// we can do this:
int[] rarr = rojo[k];
int[] grarr = verde[k];
int[] blarr = azul[k];
for ( int i = 0 ; i < SIDE*SIDE ; i++ ) {
  myColor = m.pixels[i];
  rarr[i] = (myColor >> 16 & 0xFF);
  grarr[i] = (myColor >> 8 & 0xFF);
  blarr[i] = (myColor & 0xFF);
} 

So, instead of jumping to offset k, then to index i, we get a reference to k, and jump from there each time.

I have used this to shave some clock cycles off of my own projects.
I'll try to look in more detail next time.

If I'm not mistaken (please explain if I am), this sequence results in k being equal to the index of m or -1, in which case, it falls through.
Code:

byte k=7;
  while ( m != myMovie[k] && k >= 0 )
  {
    k--;
  } 

In pseudo code:
Set k to 7.
Subtract 1 from k until myMovie[k] is the same as m
or
k is less than zero.