Skeletal data to grid data

edited February 2017 in Kinect

Sorry if this has been asked before, but I couldn't find anything. I'm quite new to both Kinect and programming, so it would be great if someone can help me out here.

In my processing sketch I have created a grid of 4*4 tiles, each having an assigned value of "0". I want to change this value to "1" if a human (or limb) is detected in that part of the grid.

This would mean if a head is spotted in the leftupper corner, tile #1 would get a value of "1". If a foot is also spotted in the rightlower corner, tile#16 would get a value of "1'. This results in a string of "1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1". This data is send to Arduino through serial commucation (I'm planning to use it to create silhouettes in LEDS).

I'm currently using the KinectPV2 library. I think I have to use the rawData of this library, where each value that is NOT 255 tells me that a limb is found there (I assume rawData gives me the 'coordinates' of where the limb is found, but I cant test that, see my question further below. If I'm wrong please correct me!).

To check this, I write this:

int [] rawData = kinect.getRawBodyTrack();

  for (int i = 0; i < rawData.length; i++){
    if (rawData[i] != 255){
      humanPresent = true;
      println(rawData[i]);
    }
  }

However, the problem seems to be that rawData is so extremely big (217088 values at minimum), that I can't run a loop like that without crashing Processing.

This brings me to my question: how could I check rawData in an efficient way, or, what would be a better way to change grid tile values based on the position of limbs?

Thanks!

Full code:

import processing.serial.*;
import KinectPV2.*;

KinectPV2 kinect;

//Grid
Table table;
int cols = 4; //grid Width (amount of valves)
int rows = 2 ;//gridHeight, amount of "pixels" we want in the grid, vertically
int gridTiles; //amount of tiles
int tileValue[];

//Enddata
String allGridValues = "";

//Serial
Serial myPort;

//Misc
boolean humanPresent = false;


void setup(){
  size(1200, 800);
  kinect = new KinectPV2(this);
  kinect.enableBodyTrackImg(true); //enables tracking bodies
  kinect.enableColorImg(true); //enables visualising the full video in color for testing purposes
  kinect.enableDepthImg(true); //enables black white image for testing purposes
  kinect.init();

  gridTiles = (rows*cols);
  setTable();

  //Serial comm
  //printArray(Serial.list()); //list available ports
  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 9600);

}


void draw(){
  clear(); //clears background
  background(255); //sets background to full white

  image(kinect.getColorImage(), 0, 0, width, height); //Full Color
  //image(kinect.getDepthImage(), 0, 0, width, height); //Black White

  //PImage kinectImg = kinect.getBodyTrackImage();
  //image(kinectImg, 512, 0);
  int [] rawData = kinect.getRawBodyTrack();

  for (int i = 0; i < rawData.length; i++){
    if (rawData[i] != 255){
      humanPresent = true;
      println(rawData[i]);
      //break
    }
  }
}

void setTable(){  
  table = new Table();
  table.addColumn("tile");
  table.addColumn("value");

  tileValue = new int[gridTiles];

  for (int i = 0; i < gridTiles; i++){
    tileValue[i] = 0;
    TableRow newRow = table.addRow();
    newRow.setInt("tile", i);
  }    
}

Answers

  • Please format your code.
    Press the gear icon to edit your post.
    Select your code and press ctrl + o.
    Leave a line above and below the code.

  • I fixed it right after posting Lord_of_the_Galaxy, you beat me though :)

  • The serial port is currently not being used. Why?

  • edited February 2017

    Because I don't have data to send through serial comm yet, due to above described problem. As soon as I have data to send I'll add

        if (mousePressed == true){
        myPort.write(allGridValues);
        }
    

    Or something similiar. allGridValues here will be the "1,0,0,0,1...." string

  • Ok. I don't have much knowledge of Kinect so can't help you, but was just trying to find any common mistakes.

  • I can't run a loop like that without crashing Processing.

    remove the println. too much output makes processing choke.

  • getBodyTrackImage isn't skeleton data (but, i think, is used to calculate it)

    instead, imagine an image with up to 6 shapes on it, each of them a body. the silhouette of the first is filled with 1s, the second with 2s... where there is no body then it's 255.

    so those 217088 values are effectively 512x424 pixels. you are reducing the resolution from that down to 4x4 (code says 2 rows, i guess that's wrong)

    you could iterate through a 16th of the screen at a time and break the moment you spot a non-255 value in that square.

    this example suggests you can use getBodyTrackUser() to get a count of how many users there are, skip checking the pixels if there are 0 as a possible optimisation.

    https://github.com/ThomasLengeling/KinectPV2/blob/3bda24ba8b7c62155bf308c0d86c961ca89dbfa3/Build_libs/KinectPV2_Eclipse/src/test/BodyTrackTest.java

  • (i'm wondering if there's an easier way of doing that, like copying those pixels to an image and using image(img, 0, 0, 4, 4) to resize it. small clumps of body will get ignored, but will you get small clumps of body? maybe a hand)

  • before:

    stick_man

    after:

    stick_man_4x4

    you can make out that 8 of those squares are empty (the bottom left is just slightly not white because of the foot)

  • Thank you Koogs. Haven't had the time to continue with this yet but your information seems very helpful.

Sign In or Register to comment.