Scaling Kinect input up

MarcoMeter

Hi folks,

I'm working on a realtime projection based on a backgorund and foreground video. The foreground video is supposed to be projected on a person dancing in front of the kinect.

Currently, I'm struggling with the scaling of this setup. Obviously the output of the processing sketch needs to be fullscreen. And the videos' resolution is 1280x720.

This is the source code:

import processing.video.*;
import SimpleOpenNI.*;

// Member
SimpleOpenNI context;
int[] userMap;
PImage rgbImage;
PImage userImage;
color pixelColor;
Movie foreground;
Movie background;

void setup() {
  size(640, 480);
  //frame.setResizable(true);
  // initialzie kinect
  context = new SimpleOpenNI(this);
  context.enableRGB();
  context.enableDepth();
  context.enableUser();
  // initialize user image and videos
  userImage = createImage(width, height, RGB);
  foreground = new Movie(this, "foreground.mp4");
  foreground.loop();
  //foreground.play();
  background = new Movie (this, "background.mp4");
  background.loop();
  //background.play();

  //println("foreground length : " + foreground.pixels.length);
  // background length is 0 on the very first frame
  //println("background length : " + background.pixels.length);
}

void draw() {
  // draw foreground
  image(foreground, 0, 0);

  context.update();
  rgbImage=context.rgbImage();

  userMap=context.userMap();
  for(int y=0;y<context.depthHeight();y++){
    for(int x=0;x<context.depthWidth();x++){
      // use silhuette to cut out foreground movie
      int index=x+y*640;
      if(userMap[index]!=0){
          pixelColor=rgbImage.pixels[index];
        userImage.pixels[index]=color(0,255, 0 , 0);
      }else{
        //userImage.pixels[index] = color(255);
        // the first frame of the second movie is not read on the very first frame !!!
        if(frameCount > 1){
          // -> index out of bounds exception if frameCount == 0
          userImage.pixels[index] = background.pixels[index];
        }
      }
    }
  }
  // display the result
  userImage.updatePixels();
  image(userImage,0,0);
}

// read both movies
void movieEvent(Movie m){
  if(m == foreground){
   foreground.read(); 
  } 
  if (m == background){
    background.read();
  }
}

What are the most logical or best approaches to scale this up to HD FullScreen? The kinect's ratio is 4:3 which makes it a little uncomfortable.

Thanks in advance!

MarcoMeter

Hey folks,

I'm just scaling the userImage up. But after I resize the userImage the alpha value is lost and I can't return the alpha values:

the end part of draw()

      userImage.updatePixels();
      userImage.resize(width, height);
      userImage.updatePixels();

      for(int i = 0; i < userImage.pixels.length; i++){
       if(userImage.pixels[i] == 0xffffff){
         println("if query !!");
        userImage.pixels[i] =color(0, 0, 0, 0);
       } 
      }

      userImage.updatePixels();
      image(userImage,0,0);
    }

MarcoMeter

The problem was that the userImage was setup as an RGB Image. So setting this parameter to ALPHA solved this.

But there is a new problem.

I need to flip the image horizontally.

I did it like this:

pushMatrix(); scale(-1,1); image(userImage,0,0); popMatrix();

Well, after that the image is gone and only the foreground movie is being visible. What's going on with this problem?