Kinect Projection Masking - Array out of bounds error

t_cain

Hi,

I'm currently working on an end of year project for university called 'motion tracking projection mapping'.

I'm currently looking into 'projection masking' using the kinect depth image. Basically i'm wanting to project some imagery through a projector and onto a moving person. I'm using the resize function in order to enlarge my mask image without exceeding the limitations of the Kinect. I am then wanting to project this enlarged image to fit a person/area. I've managed to change the size of the window, however when i'm running my code I keep getting 'Array Index out of bounds exception 7500' whenever i'm trying to resize my image. The number on the error increases when increasing the values within my resize() function. The error occurs on line 41.

Any help anyone could give me would be greatly appreciated. The code i'm using is below:

`import SimpleOpenNI.*;
SimpleOpenNI kinect;

//distance in cm depth, adapt to room 
int distance = 1500;
int distance2 = 3000;

int depthMapWidth = 640;
int depthMapHeight = 480;

PImage liveMap;

void setup(){
  size(1024,768);
  kinect = new SimpleOpenNI(this);
  if (kinect.isInit() == false){
    println("Camera not connected!");
    exit();
    return;
  }

  kinect.setMirror(true);
  kinect.enableDepth(); //enables depth image 
  liveMap = createImage(640,480,RGB); //creates empty image that will be the mask 
}

void draw(){
  background(color(0,0,0)); //set background colour to black 
  kinect.update();
  int[] depthValues = kinect.depthMap(); //array, distances 

  //liveMap.width = width;
  //liveMap.height = height;
  liveMap.loadPixels(); //overwrites pixels 

for (int y=0; y<depthMapHeight; y++){
  for(int x=0; x<depthMapWidth; x++){
    int i= x+(y*depthMapWidth);
    int currentDepthValue = depthValues[i]; //calculates the numnber of pixels in the array and gets the distance value 
    if (currentDepthValue>distance&&currentDepthValue<distance2) {
        liveMap.pixels[i] = color(255,255,255);  //if the distance lies within limits
        //change mask image to white 
      } else {
        liveMap.pixels[i] = color(0,0,0);  //if no change to black (creating mask)          
      }
    }  
  }
//mask image updated for use 
liveMap.resize(100,0);
liveMap.updatePixels();
image(liveMap,100,0); //change position here
}`

Thank you!

kfrajer

Two points. First, I will call resize only after updatePixels is called. So swap lines 49 and 50.

Second, after you resize liveMap object, its dimensions have been adjusted. So your for loop in line 36 and 37 will definitely be out of bounds. One way to solve this issue (maybe there is a better way but for now this shows the concept), replace lines 49,50 and 51 with:

liveMap.updatePixels();
PImge auxImg= liveMap.get();
auxImg.resize(100,0);
image(auxImg,100,0);

Kf

koogs

please don't post duplicates. what was up with the old thread?

https://forum.processing.org/two/discussion/21351/kinect-projection-masking-how-to-enlarge-image-to-fit-a-person

t_cain

kfrajer i'll try this thanks! And koogs I thought I would make another post since i'm posting a different question about resize function, but i'll post any further questions on the previous

t_cain

Hi guys, wondered if you could give me a hand on here, my previous discussion has been closed.

I've been working on my code and i've managed to solve some errors. I'm now wanting to assign key controls in a keyPressed section. I've managed to assign my size values to UP, DOWN, LEFT AND RIGHT. I'm wanting to change the position of my copied image by pressed W, A, S and D to move the image. When trying to enter letters nothing happens, for example:

if (keyCode == 'd') { newPosX = newPosX + 10;

I know i'm probably going about this the wrong way but can't find anything online about how to assign a keyPressed function to a letter key.

Any help would be appreciated, thanks. My updated code is below:

`import SimpleOpenNI.*;
SimpleOpenNI kinect;

//distance in cm depth, adapt to room 
int distance = 1500;
int distance2 = 3000;

int depthMapWidth = 640;
int depthMapHeight = 480;

int newWidth=1470;
int newHeight=890;
int newPosX = 0;
int newPosY = 0;

PImage liveMap;
PImage liveMapCopy;

void setup(){
  size(1470,890);
  kinect = new SimpleOpenNI(this);
  if (kinect.isInit() == false){
    println("Camera not connected!");
    exit();
    return;
  }

  kinect.enableDepth(); //enables depth image 
  liveMap = createImage(640,480,RGB); //creates empty image that will be the mask 
  liveMapCopy = createImage(1470,890,RGB);
}

void draw(){
  //background(color(100,100,100)); //set background colour to black 
  fill(100,100,100);
  rect(0,0,width,height);
  kinect.update();
  int[] depthValues = kinect.depthMap(); //array, distances 

  //liveMap.width = width;
  //liveMap.height = height;
  liveMap.loadPixels(); //overwrites pixels 

for (int y=0; y<depthMapHeight; y++){
  for(int x=0; x<depthMapWidth; x++){
    int i= x+(y*depthMapWidth);
    int currentDepthValue = depthValues[i]; //calculates the numnber of pixels in the array and gets the distance value 
    if (currentDepthValue>distance&&currentDepthValue<distance2) {
        liveMap.pixels[i] = color(230,255,0);  //if the distance lies within limits
        //change mask image to white 
      } else {
        liveMap.pixels[i] = color(0,0,0);  //if no change to black (creating mask)          
      }
    }  
  }
//mask image updated for use 

liveMapCopy.copy(liveMap,0,0,640,480,newPosX,newPosY,newWidth,newHeight);
//liveMapCopy.resize(mouseX+50,0);
//liveMapCopy.resize(640,480);
//liveMapCopy.updatePixels();
image(liveMapCopy,100,0); //change position here
}

void keyPressed(){
      if (keyCode == RIGHT) {
      newWidth = newWidth + 10;
    } if(keyCode == LEFT)   {
      newWidth = newWidth - 10;
    } if (keyCode == DOWN)  {
      newHeight = newHeight + 10;
    } if (keyCode == UP)    {
      newHeight = newHeight - 10;
    }
}
`

kfrajer

Check this: http://studio.processingtogether.com/sp/pad/export/ro.9cfU11egvzT6G/latest

Implement the usage of ADWS keys in your code and give it a try using the provided example.

Kf

kfrajer

This is a possible way to do things. keep in mind the code is not tested. You need to call updatePixels() as you are changing the image pixels.

In your case, when changing dimensions and position, keep those operations as two different transactions. This is the reason why I remove your copy function and I used an alternative way to copy the image for sake of clarity.

Beside other minor changes, the big one was the init of newPosX/Y to the center of the sketch and I set the imageMode to CENTER as this will do the positioning of the image easier: https://processing.org/reference/imageMode_.html

Lastly, I ensure resizing o the image(and its position) was within certain boundaries. It is important to avoid having an image of zero size (in my opinion).

Notice because I am changing different parameters in your original sketch, and since I didn't have the chance to test it, it is very likely something will not work right away. If you cannot figure how to fix it, post below.

Kf

import SimpleOpenNI.*;
SimpleOpenNI kinect;

//distance in cm depth, adapt to room 
int distance = 1500;
int distance2 = 3000;

int depthMapWidth = 640;
int depthMapHeight = 480;

int newWidth=1470;
int newHeight=890;
int newPosX = 0;
int newPosY = 0;

PImage liveMap;
PImage liveMapCopy;

void setup() {
  size(1470, 890);
  imageMode(CENTER);
  newPosX=width/2;
  newPosY=height/2;

  kinect = new SimpleOpenNI(this);
  if (kinect.isInit() == false) {
    println("Camera not connected!");
    exit();
    return;
  }

  kinect.enableDepth(); //enables depth image 
  liveMap = createImage(depthMapWidth, depthMapHeight, RGB); //creates empty image that will be the mask 
  //liveMapCopy = createImage(1470, 890, RGB);
}

void draw() {
  //background(color(100,100,100)); //set background colour to black 
  fill(100, 100, 100);
  rect(0, 0, width, height);
  kinect.update();
  int[] depthValues = kinect.depthMap(); //array, distances 

  liveMap.loadPixels(); //overwrites pixels 

  for (int y=0; y<depthMapHeight; y++) {
    for (int x=0; x<depthMapWidth; x++) {
      int i= x+(y*depthMapWidth);
      int currentDepthValue = depthValues[i]; //calculates the numnber of pixels in the array and gets the distance value 
      if (currentDepthValue>distance&&currentDepthValue<distance2) {
        liveMap.pixels[i] = color(230, 255, 0);  //if the distance lies within limits
        //change mask image to white
      } else {
        liveMap.pixels[i] = color(0, 0, 0);  //if no change to black (creating mask)
      }
    }
  }
  liveMap.updatePixels();

  liveMapCopy=liveMap.get();
  liveMapCopy.resize(newWidth, newHeight);

  image(liveMapCopy, newPosX, newPosY);
}

void keyReleased() {

  updateImgDimensions();
  updateImgPosition();
}

void  updateImgDimensions() {

  if (newWidth>100 && newWidth<2*width) {
    if (keyCode == RIGHT) 
      newWidth = newWidth + 10;

    if (keyCode == LEFT) 
      newWidth = newWidth - 10;
  }

  if (newHeight>100 && newHeight<2*height) {

    if (keyCode == DOWN) 
      newHeight = newHeight + 10;

    if (keyCode == UP) 
      newHeight = newHeight - 10;
  }
}

void  updateImgPosition() {

  if (newPosX> -width/2 && newPosX<width/2) {
    if (keyCode == 'a')
      newPosX-=10;
    if (keyCode == 'd')
      newPosX+=10;
  }

  if (newPosY> -height/2 && newPosY<height/2) {
    if (keyCode == 's')
      newPosY-=10;
    if (keyCode == 'w')
      newPosY+=10;
  }
}

t_cain

Hi kfrajer, Thank you so much for your help. I see that the way you have set things out with updateImgPostion ect. is a much clearer way of doing things so thank you. The code runs fine without any errors.

The resizing of my image works great, in fact better than I had it before. However, when testing out the AWSD keys nothing happens. I tried to switch keyReleased with keyPressed but still nothing happens when pressing the AWSD keys, which I can't see a reason for.

Thanks again.

kfrajer

That is odd. According to the reference, one should use key instead of keyCode. Try and see if that works in your case. Replace only the ADWS section. For the arrow section, stick to the keyCode reserved word.

Kf

t_cain

Tried to replace keyCode with just key but still nothing happens. I can change the values manually so it's not a massive problem but of course would make it a lot easier by not having to stop and re run the code.

I'm trying to figure out why it's not working now but please let me know if you spot an error. Cheers for your time.

tcain

kfrajer

Ok, so it must be my logic when I wrote those lines...

if (newPosX> -width/2 && newPosX<width/2)

should probably be

if (newPosX> 0 && newPosX<width)

This applies to the other conditional in the same function. Notice you don't really need those conditionals. i just add them as a way to demonstrate handling theuser interaction in our sketch. It is more of a crude way and there are better ways to do that (or maybe you don't even need to do that). I will also add a println in the first line of updateImgPosition() to ensure the conditionals values are proper for this case. I could spot the problem more easily if I were able to run the code....

Kf

t_cain

That works great now. Thanks so much for your help!