getting text in user's silhouette

dragonvlame

Hi,

For an interactive installation for my art study, using the kinect, I want to make a projection of the user's silhouette filled with text, like in the video here:

I have a working code to get the user's silhouette, but how can I replace it with text?

I've tried several things but it won't work.

I am working with processing 2.1 on windows 8.

My code so far:

import SimpleOpenNI.*;

SimpleOpenNI context;

PImage cam;

int[] userMap; 

void setup() {

size(640, 480);

context = new SimpleOpenNI(this);

// mirror the image to be more intuitive

context.setMirror(true);

context.enableDepth();

context.enableUser();

}

void draw() {
background(0);

context.update();

if(context.getNumberOfUsers() > 0) {

userMap = context.userMap();

loadPixels();

for(int i = 0; i < userMap.length; i++) {  

  if (userMap[i] !=0) {
    pixels[i] =color(0, 0, 255);
  }
}
updatePixels();
}
}

blyk

Here is the sample code ....enjoy

import SimpleOpenNI.*;
import java.util.*;
SimpleOpenNI context;
//--------------------------------------
color[] userColors = { 
  color(255, 0, 0), color(0, 255, 0), color(0, 0, 255), color(#FFC400), color(#8B00FF), color(#FF0D00)
};
color usericon = color(0, 255, 0);
int blob_array[];
int userCurID;
int cont_length = 640*480;
String[] sampletext = { "a", "b" , "c", "d", "e", "f", "g", "h" , "i", "j", "k"
}; // sample random text
void setup()
{
  size(640, 480); //window size
  context = new SimpleOpenNI(this); // start and enable kinect object  
  context.setMirror(true); // set mirroring object
  context.enableDepth(); // enable depth  camera
  context.enableUser(SimpleOpenNI.SKEL_PROFILE_ALL); // enable skeleton generation for all joints
  context.enableScene(); // enable scene 
  blob_array=new int[cont_length]; // initialize blob_array size
}

void draw() {
  background(-1); //set background to white color
  context.update(); // update kinect in each frame
  int[] depthValues = context.depthMap(); // save all depth values in a array
  int[] userMap = null; // initalize array to null
  int userCount = context.getNumberOfUsers(); // get number of user in scene
  if (userCount > 0) { // check if number of user is more than zero
    userMap = context.getUsersPixels(SimpleOpenNI.USERS_ALL); // get user pixles of all the users
  }
  loadPixels();
  for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth();
      if (userMap != null && userMap[index] > 0) {
        int colorIndex = userMap[index] % userColors.length;
        userCurID = userMap[index];
        blob_array[index] = 255;
        //usericon=userColors[colorIndex];
        text(sampletext[int(random(0,10))],x,y); // put your sample random text
      }
      else {
        blob_array[index]=0;
      }
    }
  }
}
void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
  println("lost: " + userId);
}

I dont have kinect now so I couldn't check the code you need to check the code and may have to fix some bugs but it would work I am sure

dragonvlame

dear blyk,

Thank you for your code. I noticed that your code doesn't work on my version of processing 2.1 and SimpleOpenNi. Functions like (SimpleOpenNI.SKEL_PROFILE_ALL) don't exist anymore. I fixed most of it, but I get stuck on the next point:

Here is the code I have until now: (with new parts I have corrected)

import SimpleOpenNI.*;
import java.util.*;

SimpleOpenNI context;

color[] userColors = { 
  color(255, 0, 0),
  color(0, 255, 0),
  color(0, 0, 255),
  color(#FFC400),
  color(#8B00FF),
  color(#FF0D00)
};

color usericon = color(0, 255, 0);
int blob_array[];
int userCurID;
int cont_length = 640*480;
String[] sampletext = { "a", "b" , "c", "d", "e", "f", "g", "h" , "i", "j", "k"
}; // sample random text

void setup(){

  size(640, 480);
  context = new SimpleOpenNI(this);
  context.setMirror(true);
  context.enableDepth();
  context.enableUser();

  blob_array=new int[cont_length];
}


void draw() {
  background(-1);
  context.update();
  int[] depthValues = context.depthMap();
  int[] userMap =null;
  int userCount = context.getNumberOfUsers();
  if (userCount > 0) {
  userMap = context.getUsersPixels(SimpleOpenNI.USERS_ALL);
  }

  loadPixels();
  for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth();
      if (userMap != null && userMap[index] > 0) {
        int colorIndex = userMap[index] % userColors.length;
        userCurID = userMap[index];
        blob_array[index] = 255;
        //usericon=userColors[colorIndex];
        text(sampletext[int(random(0,10))],x,y); // put your sample random text
          }
          else {
            blob_array[index]=0;
          }
        }
      }
   }
   void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
println("lost: " + userId);
}

Genral

You can try to do this using only the Depth camera. If you want to use the userMap you must change "userMap = context.getUsersPixels(SimpleOpenNI.USERS_ALL);" to "userMap = context.userMap();". I'm using the same version than you.

dragonvlame

I now have a code that can project something that looke like a silhouette, but I don't recognize the text in the silhouette, it's just a blurry mess... How can I fix this?

import SimpleOpenNI.*;
import java.util.*;
SimpleOpenNI context;

color[] userColors = { 
  color(255, 0, 0),
  color(0, 255, 0),
  color(0, 0, 255),
  color(#FFC400),
  color(#8B00FF),
  color(#FF0D00)
};


int[] userMap;
int blob_array[];
int userCurID;
int cont_length = 640*480;
String[] sampletext = { "a", "b" , "c", "d", "e", "f", "g", "h" , "i", "j", "k"
}; // sample random text


void setup() {
  size(640, 480);
  background(0);

  context = new SimpleOpenNI(this);

    // mirror the image to be more intuitive
    context.setMirror(true);

  context.enableDepth();
  context.enableUser();

  blob_array=new int[cont_length];

}

void draw() {
  background(0);
  context.update();

  int[]depthValues= context.depthMap();
  int[] userMap = null;
  int userCount = context.getNumberOfUsers();

  if(userCount > 0) {
    userMap = context.userMap();}

    loadPixels();
    for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth();
      if (userMap != null && userMap[index] > 0) {
        int colorIndex = userMap[index] % userColors.length;
        userCurID = userMap[index];
        blob_array[index] = 255;
        //usericon=userColors[colorIndex];
        text(sampletext[int(random(0,10))],x,y); // put your sample random text
          }
          else {
            blob_array[index]=0;
          }
        }
      }
   }
   void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
println("lost: " + userId);
}

dragonvlame

Can this code be simpler? Can, for example, the color parts be removed? I don't see the use of them...

blyk

I have written the code using older verion of simpleopenni 0.27 which was compatible with the old drivers for kinect Xbox360. Once I tried running new SimpleOpenni 1.96 version with kinect 1.6v sdk but I couldn't make it work so I was not able to test my code.

And yes you can remove that color code part which is certainly unnecessary for you. Actually I have given you a peice of my old code which had something do with that.

and don't forget to put fill(200,0,200); (you can actullly use any color) before line 59 .why? Because you need to define the text color.

blyk

@Genral thanks for giving that insight. Can you also mention that in my old post because I have started that discussion for pointing out the changes in new version of SimpleopenNI becasue there is no proper documentation other than java docs.

Post

@dragonvlame you can aslo refer to this discussion for any further problem

dragonvlame

@blyk , So if I would simplify your code, what exactly would be removed? I'm trying all sorts of things but for example the code in line 51and 52, what does it do, and is it neccesary?

blyk

line 51 & 52 scan through all the pixles one by one and as soon as pixles matches with the userpixles " userMap[index] " it takes the pixels coordinate and put the text at that position.

dragonvlame

Oke so that would be neccesary.. Thanks. I will try something and I will keep this up to date. Do you know why I get a blurry white mess in this code?:

import SimpleOpenNI.*;
import java.util.*;
SimpleOpenNI context;

color[] userColors = { 
  color(255, 0, 0),
  color(0, 255, 0),
  color(0, 0, 255),
  color(#FFC400),
  color(#8B00FF),
  color(#FF0D00)
};


int[] userMap;
int blob_array[];
int userCurID;
int cont_length = 640*480;
String[] sampletext = { "a", "b" , "c", "d", "e", "f", "g", "h" , "i", "j", "k"
}; // sample random text


void setup() {
  size(640, 480);
  background(0);

  context = new SimpleOpenNI(this);

    // mirror the image to be more intuitive
    context.setMirror(true);

  context.enableDepth();
  context.enableUser();

  blob_array=new int[cont_length];

}

void draw() {
  background(0);
  context.update();

  int[]depthValues= context.depthMap();
  int[] userMap = null;
  int userCount = context.getNumberOfUsers();

  if(userCount > 0) {
    userMap = context.userMap();}

    loadPixels();
    for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth();
      if (userMap != null && userMap[index] > 0) {
        int colorIndex = userMap[index] % userColors.length;
        userCurID = userMap[index];
        blob_array[index] = 255;
        //usericon=userColors[colorIndex];
        text(sampletext[int(random(0,10))],x,y); // put your sample random text
          }
          else {
            blob_array[index]=0;
          }
        }
      }
   }
   void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
println("lost: " + userId);
}

how can I control the size of the font and maybe the internal space between the words?

blyk

Refer to processing website: http://www.processing.org/reference/text_.html there are some other realtive functions mentioned on the website

textSize();
textAlign();
text()
textMode();

dragonvlame

Thanks. So if I look at your complete code now, with // in front of all the things I can delete in your opinion, I would get this, correct?

 import SimpleOpenNI.*;
 import java.util.*;

 SimpleOpenNI context;

 // can be removed color[] userColors = { 
   //color(255, 0, 0),
  // color(0, 255, 0),
  // color(0, 0, 255),
   //color(#FFC400),
   //color(#8B00FF),
  // color(#FF0D00)
 };

 //color usericon = color(0, 255, 0);
 int blob_array[];
 int userCurID;
 int cont_length = 640*480;
 String[] sampletext = { "a", "b" , "c", "d", "e", "f", "g", "h" , "i", "j", "k"
 }; // sample random text

 void setup(){

   size(640, 480);
   context = new SimpleOpenNI(this);
   context.setMirror(true);
   context.enableDepth();
   context.enableUser();

   blob_array=new int[cont_length];
 }


 void draw() {
   background(-1);
   context.update();
   int[] depthValues = context.depthMap();
   int[] userMap =null;
   int userCount = context.getNumberOfUsers();
   if (userCount > 0) {
   userMap = context.userMap();
   }

  loadPixels();
   for (int y=0; y<context.depthHeight(); y++) {
     for (int x=0; x<context.depthWidth(); x++) {
       int index = x + y * context.depthWidth();
       if (userMap != null && userMap[index] > 0) {
 //      int colorIndex = userMap[index] % userColors.length;
        userCurID = userMap[index];
         blob_array[index] = 255;
  //     usericon=userColors[colorIndex];
          fill(200,0,200)
         text(sampletext[int(random(0,10))],x,y); // put your sample random text
           }
           else {
             blob_array[index]=0;
           }
         }
       }
    }

  //  void onNewUser(int userId) {
  // println("detected" + userId);
 //}
 //void onLostUser(int userId) {
 //println("lost: " + userId);
 //}

dragonvlame

What is the numer "255" for atline 51, blob_Array[index] ?

dragonvlame

When I try to change something in your code, I get the error: ArrayIndexOutOfBondsException with a number behind it, which changed every time I run the sketch... Why am I getting this error?

dragonvlame

dragonvlame

@blyk , When I try anything different with the text, like textLeading, textSize or textAlign, I keep getting the error like above. How can I get less text so that I can see the different letters?

Thanks!

blyk

import SimpleOpenNI.*;
import java.util.*;

SimpleOpenNI context;
String[] sampletext = { 
  "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k"
}; // sample random text

void setup() {
  size(640, 480); // if you don't have graphics card then use this 
  size(640, 480, P3D); // if you r system have any grpshics card other than intel Graphics then use this // it will simply improve the speed of the processing
  context = new SimpleOpenNI(this);
  context.setMirror(true);
  context.enableDepth();
  context.enableUser();
}


void draw() {
  background(-1);
  context.update();
  int[] userMap =null;

  //  this part is necesasry when I have used old simpleopenNI library but I dont have any idea what would ahappen if you modify the code according to new version
  //  if (userCount > 0) {
  //    userMap = context.getUsersPixels(SimpleOpenNI.USERS_ALL);
  //  }

  int userCount = context.getNumberOfUsers();
  if (userCount > 0) {
    userMap = context.userMap();
  }


  loadPixels(); // keep this part it is necessary because it loads the depth pixles and ready them for any further modification
  for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth(); // it call the pixels index // so there two methods for calling pixles, either you call them by index or pixel coordinate (x,y);
      if (userMap != null && userMap[index] > 0) {  // match with the user's pixle with the kinect pixel
        fill(#43E505);// text color set to green
        text(sampletext[int(random(0, 10))], x, y); // call sample random text from array
      }
    }
  }
  updatePixels(); // first try with this if doesn't work then comment it back // it will show the modified pixles which is not nessary for you
}


//---------------------------------------------------------------------------
// This part is important so don't comment it//
void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
  println("lost: " + userId);
}
/////////////////////////////////

---

import SimpleOpenNI.*;
import java.util.*;
SimpleOpenNI context;
//--------------------------------------
String[] sampletext = { 
  "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k"
}; // sample random text
void setup()
{
  size(640, 480); //window size
  context = new SimpleOpenNI(this); // start and enable kinect object  
  context.setMirror(true); // set mirroring object
  context.enableDepth(); // enable depth  camera
  context.enableUser(SimpleOpenNI.SKEL_PROFILE_ALL); // enable skeleton generation for all joints
  context.enableScene(); // enable scene 
}

void draw() {
  background(-1); //set background to white color
  context.update(); // update kinect in each frame
  int[] userMap = null; // initalize array to null
  int userCount = context.getNumberOfUsers(); // get number of user in scene
  if (userCount > 0) { // check if number of user is more than zero
    userMap = context.getUsersPixels(SimpleOpenNI.USERS_ALL); // get user pixles of all the users
  }
  loadPixels();
  for (int y=0; y<context.depthHeight(); y++) {
    for (int x=0; x<context.depthWidth(); x++) {
      int index = x + y * context.depthWidth();
      if (userMap != null && userMap[index] > 0) {
        fill(#FF0335);
        text(sampletext[int(random(0, 10))], x, y); // put your sample random text
      }
    }
  }
}
void onNewUser(int userId) {
  println("detected" + userId);
}
void onLostUser(int userId) {
  println("lost: " + userId);
}

blyk

As I told you I don't have kinect now so I can't try it so you need to fix your bugs on your own I am really sorry

dragonvlame

By using the code you send first, with 57 lines, I get the same kind of image. I understand that you can't help me fix the bugs :)I don't blame you for that at all, I am really grateful for your help! :D

I'm trying to figure out why it's all so close together, but the problem stays :)

koogs

aren't you drawing a letter at each pixel position, meaning you get a massive overlap? try using random colours for each letter so you can see the individual shapes.

i've also got to say that i hate this:

String[] sampletext = {
  "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k"
};

what's up with String sampletext = "abcdefghijk"; and using charAt() to pick the letters out?

or char c = (char)random('a', 'z' + 1);

(untested!)

dragonvlame

Instead of having the y++ and the x++ in line 7 and 8 I put a y+=30 and a x+=30 and now it works fine. This thread can be closed :) Thanks all!