...java/bin/java: "weird ASCII characters here" not found
    /home/pi/Desktop/processing-2.2.1/java/bin/java: 2: /home/pi/Desktop/processing-2.2.1/java/bin/java: Syntax error: "(" unexpected

As for the SimpleOpenNi, any direction on building it would be greatly appreciated as I get errors:

    /Installing OpenNI
    *********************************

    copying shared libraries...OK
    copying executables...OK
    copying include files...OK
    creating database directory...OK
    registering module 'libnimMockNodes.so' .../usr/bin/niReg: 1: /usr/bin/niReg: Syntax error: word unexpected (expecting ")")

I would like to share with you our last piece "Non-Finito", an interactive installation creating a 3D marble sculpture from users who enter into a kinect field of view. We use the kinect v2 library and GLSL Shader for materials and generative marble texture.

We hope you wil like it.

Non-Finito : http://www.bonjour-lab.com/project/non-finito/

You wan also see some making of still here : https://www.flickr.com/photos/alexr4/sets/72157666351804994

check out more here http://simonsloan.com/logistic

check out more here http://simonsloan.com/logistic

It uses the custom shapes class to create different shapes. would it be possible to change the code in such a way that i can change te shapes into images, i.e leaves or snow flakes? and if so, would anyone be so kind to help me in the right direction?

This is the CustomShape class that creates a shape and interacts with the shape of my body.

import java.util.List;
import java.util.Arrays;

class CustomShape {
  // to hold the box2d body
  Body body;
  // to hold the Toxiclibs polygon shape
  Polygon2D toxiPoly;
  // custom color for each shape
  color col;
  // radius (also used to distinguish between circles and polygons in this combi-class
  float r;

  CustomShape(float x, float y, float r, BodyType type) {
    this.r = r;
    // create a body (polygon or circle based on the r)
    makeBody(x, y, type);
    // get a random color
    col = getRandomColor();
  }

  void makeBody(float x, float y, BodyType type) 
  {
    // define a dynamic body positioned at xy in box2d world coordinates,
    // create it and set the initial values for this box2d body's speed and angle
    BodyDef bd = new BodyDef();
    bd.type = type;
    bd.position.set(box2d.coordPixelsToWorld(new Vec2(x, y)));
    body = box2d.createBody(bd);
    body.setLinearVelocity(new Vec2(random(-8, 8), random(2, 8)));
    body.setAngularVelocity(random(-5, 5));

    // box2d polygon shape
    //PolygonShape sd = new PolygonShape();
    /* toxiPoly = new Polygon2D(Arrays.asList(new Vec2D(-r, r*1.5), 
     new Vec2D(r, r*1.5), 
     new Vec2D(r, -r*1.5), 
     new Vec2D(-r, -r*1.5)));*/

    if (r == -1) 
    {
      // box2d polygon shape
      PolygonShape sd = new PolygonShape();
      // toxiclibs polygon creator (triangle, square, etc)
      toxiPoly = new Circle(random(5, 20)).toPolygon2D(int(random(3, 6)));
      // place the toxiclibs polygon's vertices into a vec2d array
      Vec2[] vertices = new Vec2[toxiPoly.getNumPoints()];

      for (int i=0; i<vertices.length; i++) 
      {
        Vec2D v = toxiPoly.vertices.get(i);
        vertices[i] = box2d.vectorPixelsToWorld(new Vec2(v.x, v.y));
      }
      // put the vertices into the box2d shape
      sd.set(vertices, vertices.length);
      // create the fixture from the shape (deflect things based on the actual polygon shape)
      body.createFixture(sd, 1);
    }

    else 
    {
      // box2d circle shape of radius r
      CircleShape cs = new CircleShape();
      cs.m_radius = box2d.scalarPixelsToWorld(r);
      // tweak the circle's fixture def a little bit
      FixtureDef fd = new FixtureDef();
      fd.shape = cs;
      fd.density = 1;
      fd.friction = 0.01;
      fd.restitution = 0.3;
      // create the fixture from the shape's fixture def (deflect things based on the actual circle shape)
      body.createFixture(fd);
    }
  }




  // method to loosely move shapes outside a person's polygon
  // (alternatively you could allow or remove shapes inside a person's polygon)
  void update() 
  {
    // get the screen position from this shape (circle of polygon)
    Vec2 posScreen = box2d.getBodyPixelCoord(body);
    // turn it into a toxiclibs Vec2D
    Vec2D toxiScreen = new Vec2D(posScreen.x, posScreen.y);
    // check if this shape's position is inside the person's polygon
    boolean inBody = poly.containsPoint(toxiScreen);
    // if a shape is inside the person
    if (inBody) {
      // find the closest point on the polygon to the current position
      Vec2D closestPoint = toxiScreen;
      float closestDistance = 9999999;
      for (Vec2D v : poly.vertices) 
      {
        float distance = v.distanceTo(toxiScreen);
        if (distance < closestDistance) 
        {
          closestDistance = distance;
          closestPoint = v;
        }
      }
      // create a box2d position from the closest point on the polygon
      Vec2 contourPos = new Vec2(closestPoint.x, closestPoint.y);
      Vec2 posWorld = box2d.coordPixelsToWorld(contourPos);
      float angle = body.getAngle();
      // set the box2d body's position of this CustomShape to the new position (use the current angle)
      body.setTransform(posWorld, angle);
    }
  }

  // display the customShape
  void display() {
    // get the pixel coordinates of the body
    Vec2 pos = box2d.getBodyPixelCoord(body);
    pushMatrix();
    // translate to the position
    translate(pos.x, pos.y);
    noStroke();
    // use the shape's custom color
    fill(col);

    if (r == -1) {
      // rotate by the body's angle
      float a = body.getAngle();
      rotate(-a); // minus!
      gfx.polygon2D(toxiPoly);
    } 
    else {
      ellipse(0, 0, r*2, r*2);
    }

    popMatrix();
  }

  // if the shape moves off-screen, destroy the box2d body (important!)
  // and return true (which will lead to the removal of this CustomShape object)
  boolean done() {
    Vec2 posScreen = box2d.getBodyPixelCoord(body);
    boolean offscreen = posScreen.y > height;
    if (offscreen) {
      box2d.destroyBody(body);
      return true;
    }
    return false;
  }
}

EOF

Thanks in advance

I can't find a way. Thanx!!

import processing.serial.*; import java.util.Map; import java.util.Iterator; import SimpleOpenNI.*; import processing.serial.*; SimpleOpenNI context; Serial myPort;

int handVecListSize = 20; Map<Integer,ArrayList> handPathList = new HashMap<Integer,ArrayList>(); color[] userClr = new color[]{ color(255,0,0), color(0,255,0), color(0,0,255), color(255,255,0), color(255,0,255), color(0,255,255) };

void setup() { // frameRate(200); size(640,480); context = new SimpleOpenNI(this);

if(context.isInit() == false) { println("Can't init SimpleOpenNI, maybe the camera is not connected!"); exit(); return;
}

// enable depthMap generation context.enableDepth();

// disable mirror context.setMirror(true);

// enable hands + gesture generation //context.enableGesture(); context.enableHand(); context.startGesture(SimpleOpenNI.GESTURE_WAVE); String portName = Serial.list()[0]; // This gets the first port on your computer. myPort = new Serial(this, portName, 9600);

// set how smooth the hand capturing should be //context.setSmoothingHands(.5); }

void draw() { // update the cam context.update();

image(context.depthImage(),0,0);

// draw the tracked hands if(handPathList.size() > 0)
{
Iterator itr = handPathList.entrySet().iterator();
while(itr.hasNext()) { Map.Entry mapEntry = (Map.Entry)itr.next(); int handId = (Integer)mapEntry.getKey(); ArrayList vecList = (ArrayList)mapEntry.getValue(); PVector p; PVector p2d = new PVector();

    stroke(userClr[ (handId - 1) % userClr.length ]);
    noFill(); 
    strokeWeight(1);        
    Iterator itrVec = vecList.iterator(); 
    beginShape();
      while( itrVec.hasNext() ) 
      { 
        p = (PVector) itrVec.next(); 

        context.convertRealWorldToProjective(p,p2d);
        vertex(p2d.x,p2d.y);
      }
    endShape();   

    stroke(userClr[ (handId - 1) % userClr.length ]);
    strokeWeight(4);
    p = vecList.get(0);
    context.convertRealWorldToProjective(p,p2d);
    point(p2d.x,p2d.y);

      myPort.write('S');

// Send the value of the mouse's x-position myPort.write(int(255* p2d .x/width)); // Send the value of the mouse's y-position myPort.write(int(255* p2d.y/height));

}        

} }

// ----------------------------------------------------------------- // hand events

void onNewHand(SimpleOpenNI curContext,int handId,PVector pos) { println("onNewHand - handId: " + handId + ", pos: " + pos);

ArrayList vecList = new ArrayList(); vecList.add(pos);

handPathList.put(handId,vecList); }

void onTrackedHand(SimpleOpenNI curContext,int handId,PVector pos) { //println("onTrackedHand - handId: " + handId + ", pos: " + pos );

ArrayList vecList = handPathList.get(handId); if(vecList != null) { vecList.add(0,pos); if(vecList.size() >= handVecListSize) // remove the last point vecList.remove(vecList.size()-1); }
}

void onLostHand(SimpleOpenNI curContext,int handId) { println("onLostHand - handId: " + handId); handPathList.remove(handId); }

// ----------------------------------------------------------------- // gesture events

void onCompletedGesture(SimpleOpenNI curContext,int gestureType, PVector pos) { println("onCompletedGesture - gestureType: " + gestureType + ", pos: " + pos);

int handId = context.startTrackingHand(pos); println("hand stracked: " + handId); }

// ----------------------------------------------------------------- // Keyboard event void keyPressed() {

switch(key) { case ' ': context.setMirror(!context.mirror()); break; case '1': context.setMirror(true); break; case '2': context.setMirror(false); break; } }

void onRecognizeGesture(String strGesture, PVector idPosition, PVector endPosition) { // SimpleOpenNI.GESTURE_HAND_RAISE //endPosition context.endGesture(SimpleOpenNI.GESTURE_HAND_RAISE ); context.startTrackingHand(endPosition); // context.startGesture(SimpleOpenNI.GESTURE_WAVE); }

Here's a video I made to animate Stravinsky's Infernal dance, from "The Firebird".

I developed the project with the help of two dancers in a studio, here in Quebec city. I've been working with Processing for some time now, but this is the first time I work with the Kinect and with pro dancers.

Hope you enjoy how it came out. Comments are welcome.

I tried using other libraries, such as the default video library and a sketch I found for that, which didn't work. And then the gsvideo library, which gave me no errors, but only showed a black screen (the background). I'd prefer to use this library, because I know it at least works on Macs. Any help would be greatly appreciated as my knowledge of code is very limited.

import SimpleOpenNI.*;
SimpleOpenNI kinect;

import jmcvideo.*;
import processing.opengl.*;
JMCMovie myMovie;
double movieDuration;
float ratio =0.0;
float old_mouse_pos = 0;

void setup() {
  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  kinect.enableUser();

  size(640, 480, P2D);
  //size(640,480,P2D);
  //background(0);
  // stroke(255, 0, 0);
  //strokeWeight(5);
  //textSize(20);

  myMovie = movieFromDataPath("drop.mov");

  myMovie.pause();

  movieDuration = myMovie.getDuration();
}

JMCMovie movieFromDataPath(String filename)
{
  return new JMCMovie(this, filename, RGB);
}

void draw()
{
  kinect.update();

  PImage depthImage =kinect.depthImage();
 // image(depthImage, 640, 0);

  IntVector userList = new IntVector();
  kinect.getUsers(userList);

  if (userList.size()>0) {
    int userId = userList.get(0);

    if (kinect.isTrackingSkeleton(userId)) {
      PVector leftHand = new PVector();
      //  PVector rightHand = new PVector();

      kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_LEFT_HAND, leftHand);

      //  kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_RIGHT_HAND, rightHand);

      // println(leftHand.y, rightHand.y);

      image(myMovie, 0, 0, 640, 480);
      //  image(myMovie, 0, 0, myMovie.width, myMovie.height);
      if (old_mouse_pos !=leftHand.y) {
        ratio=(float)(height+leftHand.y)/(2*height);
        myMovie.setCurrentTime((1-ratio)*movieDuration);
      }
      old_mouse_pos=leftHand.y;
    }
  }
  if (userList.size()

Here I post the Processing code and after the Arduino code.

Thanks in advance for your help!

import SimpleOpenNI.*;
SimpleOpenNI  kinect;
  import processing.serial.*;
     Serial port;

void setup() { 
  size(640, 480);
  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  kinect.enableUser(SimpleOpenNI.SKEL_PROFILE_ALL);
  kinect.setMirror(true);
   println(Serial.list());
     String portName = Serial.list()[8];
     port = new Serial(this, portName, 9600);
}
void draw() {
  kinect.update();
  PImage depth = kinect.depthImage();
  image(depth, 0, 0);
  IntVector userList = new IntVector();
  kinect.getUsers(userList);
  if (userList.size() > 0) {
    int userId = userList.get(0);
    if ( kinect.isTrackingSkeleton(userId)) {
      // get the positions of the three joints of our arm
      PVector rightHand = new PVector();
      kinect.getJointPositionSkeleton(userId,
                                      SimpleOpenNI.SKEL_RIGHT_HAND,
                                      rightHand);
      PVector rightElbow = new PVector();
      kinect.getJointPositionSkeleton(userId,
                                      SimpleOpenNI.SKEL_RIGHT_ELBOW,
                                      rightElbow);
      PVector rightShoulder = new PVector();
      kinect.getJointPositionSkeleton(userId,
                                      SimpleOpenNI.SKEL_RIGHT_SHOULDER,
                                      rightShoulder);
      // we need right hip to orient the shoulder angle
      PVector rightHip = new PVector();
      kinect.getJointPositionSkeleton(userId,
                                      SimpleOpenNI.SKEL_RIGHT_HIP,
                                      rightHip);
// reduce our joint vectors to two dimensions
PVector rightHand2D = new PVector(rightHand.x, rightHand.y); 
PVector rightElbow2D = new PVector(rightElbow.x, rightElbow.y);
PVector rightShoulder2D = new PVector(rightShoulder.x,
                                            rightShoulder.y);
      PVector rightHip2D = new PVector(rightHip.x, rightHip.y);
      // calculate the axes against which we want to measure our angles
      PVector torsoOrientation =
PVector.sub(rightShoulder2D, rightHip2D); 
PVector upperArmOrientation =
        PVector.sub(rightElbow2D, rightShoulder2D);

        // calculate the angles between our joints
float shoulderAngle = angleOf(rightElbow2D, 
                                        rightShoulder2D,
                                        torsoOrientation);
          float elbowAngle    = angleOf(rightHand2D,
                                        rightElbow2D,
                                        upperArmOrientation);
          // show the angles on the screen for debugging
          fill(255,0,0);
          scale(3);
          text("shoulder: " + int(shoulderAngle) + "\n" +
                " elbow: " + int(elbowAngle), 20, 20);
     byte out[] = new byte[2];
     out[0] = byte(shoulderAngle);
     out[1] = byte(elbowAngle);
     port.write(out);
        }

} }
    float angleOf(PVector one, PVector two, PVector axis) {
      PVector limb = PVector.sub(two, one);
      return degrees(PVector.angleBetween(limb, axis));
}
    // user-tracking callbacks!
    void onNewUser(int userId) {
      println("start pose detection");
      kinect.startPoseDetection("Psi", userId);
    }
    void onEndCalibration(int userId, boolean successful) {
      if (successful) {
        println("  User calibrated !!!");
        kinect.startTrackingSkeleton(userId);
      }
      else {
        println("  Failed to calibrate user !!!");
        kinect.startPoseDetection("Psi", userId);
} }
    void onStartPose(String pose, int userId) {
      println("Started pose for user");
      kinect.stopPoseDetection(userId);
      kinect.requestCalibrationSkeleton(userId, true);
}

Arduino code

#include <Servo.h> 
// declare both servos
Servo shoulder;
Servo elbow;

// setup the array of servo positions 
int nextServo = 0;
int servoAngles[] = {0,0};

void setup() {
// attach servos to their pins 

shoulder.attach(9); 
elbow.attach(10); 
Serial.begin(9600);
}
void loop() { 
  if(Serial.available()){ 

    int servoAngle = Serial.read();
    servoAngles[nextServo] = servoAngle;
    nextServo++;
        if(nextServo > 1){
          nextServo = 0;
        }
shoulder.write(servoAngles[0]); 
elbow.write(servoAngles[1]);
} }

This program is much like the AR SandBox project done with Kinect, Linux and some kind of version of C, but for our project we are using processing so that we can add more onto the code (for example, have fish swim in the areas that are supposed to be blue).

Here is the most recent code: `

import SimpleOpenNI.*;


SimpleOpenNI context;
float        zoomF =0.5f;
float        rotX = radians(180);  // by default rotate the hole scene 180deg around the x-axis, 
                                   // the data from openni comes upside down
float        rotY = radians(0);
boolean      autoCalib=true;


ArrayList <PVector> darkGreenPoints = new ArrayList <PVector>();
ArrayList  <PVector> greenPoints = new ArrayList <PVector> ();
ArrayList <PVector> yellowPoints = new ArrayList <PVector>();
ArrayList <PVector> brightBluePoints = new ArrayList <PVector>(); 
ArrayList <PVector> bluePoints = new ArrayList <PVector>();
ArrayList <PVector> darkBluePoints = new ArrayList <PVector>();


///variables to modify the depth level colors
int dgreenStart=609;
int dgreenEnd=837;
int greenEnd=1065;
int yellowEnd=1293;
int bblueEnd=1521;
int blueEnd=1749;
int dblueEnd=1977;

void setup()
{

  size(1024,768,P3D); 
  println("start setup");
  context = new SimpleOpenNI(this);
  if(context.isInit() == false)
  {
     println("Can't init SimpleOpenNI, maybe the camera is not connected!"); 
     exit();
     return;  
  }
  if(context.isInit() == true)
  {
     println("done!"); 
  }

  // disable mirror
  context.setMirror(false);

  // enable depthMap generation 
  context.enableDepth();

  // enable skeleton generation for all joints

  stroke(255,255,255);
  smooth();  
  perspective(radians(45),float(width)/float(height),10,150000);     
  println("end setup");
 }

void draw()
{
  println("begin draw");
  // update the cam
  println("updating cam");
  /////Here is where I'm having problems, the program gets stuck at the update method.
  context.update();
  println("cam updated");
  background(0,0,0);

  // set the scene pos
  translate(width/2, height/2, 0);
  rotateX(rotX);
  rotateY(rotY);
  scale(zoomF);

  println("map vars");
  int[]   depthMap = context.depthMap();
  int     steps   = 3;  // to speed up the drawing, draw every third point
  int     index;
  PVector realWorldPoint;

  println("color dot code");
  depthColor();

  translate(0,0,-1000);  // set the rotation center of the scene 1000 infront of the camera
  println("original dot code");
  // draw the pointcloud
  beginShape(POINTS);
  for(int y=0;y < context.depthHeight();y+=steps)
  {
    for(int x=0;x < context.depthWidth();x+=steps)
    {
      index = x + y * context.depthWidth();
      if(depthMap[index] > 0)
      { 
        // draw the projected point
        realWorldPoint = context.depthMapRealWorld()[index];
        stroke(100);
        point(realWorldPoint.x,realWorldPoint.y,realWorldPoint.z);
      }
    } 
  } 
  println("new dot code");
  for (int i=0;i<darkGreenPoints.size();i++){
   PVector d = darkGreenPoints.get(i);
   stroke(0,100,0);
   point(d.x,d.y,d.z); 
  }
  for (int i=0;i<bluePoints.size();i++){
   PVector d = bluePoints.get(i);
   stroke(0,0,200);
   point(d.x,d.y,d.z); 
  }
  for (int i=0;i<greenPoints.size();i++){
   PVector d = greenPoints.get(i);
   stroke(0,255,0);
   point(d.x,d.y,d.z); 
  }
  for (int i=0;i<yellowPoints.size();i++){
   PVector d = yellowPoints.get(i);
   stroke(255,255,0);
   point(d.x,d.y,d.z); 
  }
  for (int i=0;i<brightBluePoints.size();i++){
   PVector d = brightBluePoints.get(i);
   stroke(100,100,255);
   point(d.x,d.y,d.z); 
  }
  for (int i=0;i<darkBluePoints.size();i++){
   PVector d = darkBluePoints.get(i);
   stroke(0,0,100);
   point(d.x,d.y,d.z); 
  }
  endShape();  

  println("clearinglists");
  ///clearing lists to reset them, so that the drawing is exactly
  ///what the kinect sees in each current frame, not previous ones
  println(darkGreenPoints.size(), "before");
  darkGreenPoints.clear();
  bluePoints.clear();
  greenPoints.clear();
  yellowPoints.clear();
  brightBluePoints.clear();
  darkBluePoints.clear();
  println(darkGreenPoints.size(), "after");


  println("end draw");
}

void depthColor(){
    PVector b;
    println("depth color method");
    for(int i=0;i<context.depthMapRealWorld().length;i++){
     b=context.depthMapRealWorld()[i];
     if (b.z>dgreenStart && b.z<dgreenEnd){
      darkGreenPoints.add(b); 
     }
      if (b.z>dgreenEnd && b.z<greenEnd){
      greenPoints.add(b); 
      }
      if (b.z>greenEnd && b.z<yellowEnd){
      yellowPoints.add(b); 
     }
      if (b.z>yellowEnd && b.z<bblueEnd){
      brightBluePoints.add(b); 
     }
     if (b.z>bblueEnd && b.z<blueEnd){
      bluePoints.add(b); 
     }
     if (b.z>blueEnd && b.z<dblueEnd){
      darkBluePoints.add(b); 
     }
    }
  }

` This code was working beautifully a couple of weeks ago, and I have not made any drastic changes to it in between then, just some clean up. After adding in println() statements, I see that Processing doesn't go farther than context.update(). I get no errors in the code, and when I remove context.update(), the rest of the code is able to run. All I get is a grey screen.

I have been on the ultimate and extremely frustrating goose chase to find a way to fix this problem. The kinect is functional, as proven when plugged into a 360.

Please help me. I don't know what to do anymore.

Sincerely,

The incredibly defeated student

I see a lot of users are reporting the same problem online, but I've not found any solutions that work for Windows.

Can anyone tell me what the 'fix' is?
thanks

Can someone please tell what I'm doing wrong here?

I'm copying the implicit canvas (this.g) to my VDMXCanvas (PGraphics) and something is causing the pixels to be flipped 180 degrees - check the screenshot!

I've overlaid the VDMXCanvas at the bottom half of the screen so I can see the two together. Here's the patch:

https://github.com/msp/MspSyphonSceneImageBasic/blob/master/MspSyphonSceneImageBasic.pde

Any thoughts welcome!

Cheers

Matt

I am tring to have an oc-tree from pointclouds of kinect. Problem I had, was convert the

kinect.depthMapRealWorld()

that is PVector array, to a Vec3D because I am usin Toxiclibs library. Finally I have a Vec3D array with all points inside. Now I have another big problem. If I use

tree.addAll(new ArrayList(toxVecs.length));

it draw me only one cube of the octree and not all points. Instead if i write

for (int i = 0; i < toxVecs.length-1; i++)
{

  tree.addPoint(toxVecs[i]);
}

i receive the "Java heap error"!! toxVecs [] is my array with all pointclouds inside.

• Any help?
• Do you have other approach to solve this problem?

Help me please, don't leave me alone!

Thank you, Dam

sorry for my english.

thanks :)

Here's some more information on my code, if it will help: I use the closest depth value from the one half of the screen to send information to my first servo motor using closestDepthValue1, and I send the other half to the second servo using closestDepthValue2. Because I want the two servos to have the same angles, I sent the same value to each depending on the depth, but just incremented by 200 to differentiate. So closestDepthValue2 would send 220 as a value, but the motor would hopefully only move to 20 degrees. (Is there a more efficient/practical way to do this?) I also print the depth image from the Kinect to see what the camera is actually sensing, and that seems to work pretty well, just that Processing is not communicating with the Arduino correctly.

Thank you again! Sorry for the inconvenience! :)

Arduino Code: #include <Servo.h> Servo one; Servo two; int nextServo = 0; int servoAngles = 0; void setup() { one.attach(8); two.attach(9); Serial.begin(9600); } void loop() { if(Serial.available()) { int servoAngle = Serial.read(); if(servoAngle <= 160) { servoAngles = servoAngle; one.write(servoAngles); } if(servoAngle > 160) { servoAngles = servoAngle - 200; two.write(servoAngles); } } }

Processing Code: `import processing.serial.*; Serial myPort; import SimpleOpenNI.*; SimpleOpenNI kinect; int closestValue1; int closestValue2; int closestX; int closestY; int closestP; int closestM; void setup() { size(640, 480); String portName = Serial.list()[4]; myPort = new Serial(this, portName, 9600); kinect = new SimpleOpenNI(this); kinect.enableDepth(); } void draw() { closestValue1 = 8000; closestValue2 = 8000; kinect.update(); int[] depthValues = kinect.depthMap(); for (int y = 0; y < 480; y++) { for (int x = 0; x < 320; x++) { int i = x + y * 320; int currentDepthValue1 = depthValues[i]; if (currentDepthValue1 > 0 && currentDepthValue1 < closestValue1) { closestValue1 = currentDepthValue1; closestX = x; closestY = y; if (currentDepthValue1 >= 890) { myPort.write(20); } if (currentDepthValue1 >= 803 && currentDepthValue1 < 890) { myPort.write(90); } if (currentDepthValue1 < 803) { myPort.write(160); } } } } for (int p = 0; p < 480; p++) { for (int m = 320; m < 640; m++) { int t = m + p * 320; int currentDepthValue2 = depthValues[t]; if (currentDepthValue2 > 0 && currentDepthValue2 < closestValue2) { closestValue2 = currentDepthValue2; closestM = m; closestP = p; if (currentDepthValue2 >= 890) { myPort.write(220); } if (currentDepthValue2 >= 803 && currentDepthValue2 < 890) { myPort.write(290); } if (currentDepthValue2 < 803) { myPort.write(360); } } } }

image(kinect.depthImage(), 0, 0); fill(255, 0, 0); ellipse(closestX, closestY, 25, 25); ellipse(closestM, closestP, 25, 25); }`

After a long first semester, my partner and I have finished our first code! Of course, with the help of many along the way, which we would like to thank. Without the Processing community, this code would not have been possible.

This is a kinect code, so be sure to plug it in before drawing!

P.S. There is a small glitch that causes the program to close.

//main code from SIMPLEOPENNI library examples

//imported libraries
import java.util.Map;
import java.util.Iterator;

import SimpleOpenNI.*;
import java.util.ArrayDeque;
import java.util.Queue;

//initizalizing data - global
static final int INTERVAL = 2 * 15000; 
static final int HUE = 1<<10, FPS = 200, TOLERANCE = 40, ALIASING = 2;
static final int DIM = 100, MAX = 03000, DETAIL = 1000, DEPTH = 2000;
final Queue<PVector> points = new ArrayDeque(MAX);
int resetAt;

SimpleOpenNI context;
float handVecListSize = 20;
Map<Integer,ArrayList<PVector>>  handPathList = new HashMap<Integer,ArrayList<PVector>>();





void setup(){
thread("timedShots");

 //regular setup
  smooth(8);
  frameRate(300);
  size(640,480);
  background(0);





  //controls color of lines
  colorMode(HSB, HUE, 1, 1);
  smooth(ALIASING);
  noFill();
 // println("x: " + x);

//if the camera isn't connected
  context = new SimpleOpenNI(this);
  if(context.isInit() == false)
  {
     println("Can't init SimpleOpenNI, maybe the camera is not connected?"); 
     exit();
     return; 

  }   

  // enable depthMap generation 
  context.enableDepth();

  // disable mirror
  context.setMirror(true);

  // enable hands + gesture generation
  //context.enableGesture();
  context.enableHand();
  context.startGesture(SimpleOpenNI.GESTURE_WAVE);


 }
void reset(){
  resetAt= millis() + 30000;
}
void draw(){
  // update the cam
  context.update();

  fill(0);
  textSize(12);
  int timeLeft = resetAt - millis();
  text(timeLeft, 20, 20);



  if(timeLeft<3000){
    fill(0);
    noStroke();
    rectMode(CENTER);
    rect(width/8, height/8, 40,40);
    textSize(30);
    fill(255);
    textAlign(CENTER, CENTER);
    int countDown = 1 + (timeLeft / 1000);
    text( countDown, width/8, height/8);
  }
  if (millis () >= resetAt) {
    reset();
    background(0);
  }







 // image(context.depthImage(),0,0);

  // draw the tracked hands
  if(handPathList.size() > 0)  
  {    
    Iterator itr = handPathList.entrySet().iterator();     
    while(itr.hasNext())
    {
      Map.Entry mapEntry = (Map.Entry)itr.next(); 
      int handId =  (Integer)mapEntry.getKey();
      ArrayList<PVector> vecList = (ArrayList<PVector>)mapEntry.getValue();
      //PVector p;
      PVector p2;
      PVector p2d = new PVector();
      PVector p2d2 = new PVector();
      lines(vecList, p2d, 0, 2); //makes the three lines
      lines(vecList, p2d, 5, 2);
      lines(vecList, p2d, 10, 2);

    }        
  }
}

// Adding multiple lines
void lines(ArrayList<PVector> vl, PVector p2d, int distance, int thickness) {
      int fc = frameCount;

      //makes stroke change color
  stroke (fc++ & HUE-1, 1, 10);
        noFill(); 
        strokeWeight(thickness);        
        Iterator itrVec = vl.iterator(); 
        beginShape();
          while( itrVec.hasNext() ) 
          { 
            PVector p = (PVector) itrVec.next(); 

            context.convertRealWorldToProjective(p,p2d);
            vertex(p2d.x-distance,p2d.y-distance);
          }
        endShape();  
}



// -----------------------------------------------------------------
// hand events
//if a hand starts moving
void onNewHand(SimpleOpenNI curContext,int handId,PVector pos){
  //let us know hand is found
  println("onNewHand - handId: " + handId + ", pos: " + pos);

   //startscreen goes away when new hand is sensed
 /*if (f_startscreen) {
    f_startscreen = false;
   background(0) ;
 */

  ArrayList<PVector> vecList = new ArrayList<PVector>();
  vecList.add(pos);

  handPathList.put(handId,vecList);
  }
//}

//if the hand is tracked
void onTrackedHand(SimpleOpenNI curContext,int handId,PVector pos)
{
  //println("onTrackedHand - handId: " + handId + ", pos: " + pos );

  ArrayList<PVector> vecList = handPathList.get(handId);
  if(vecList != null)
  {
    vecList.add(0,pos);
    if(vecList.size() >= handVecListSize)
      // remove the last point 
      vecList.remove(vecList.size()-1); 
  }  
}
//if the hand is lost
void onLostHand(SimpleOpenNI curContext,int handId)
{
  //let us know the hand is lost
  println("onLostHand - handId: " + handId);
  handPathList.remove(handId);

  //startscreen comes back if no hand
  //f_startscreen = true;
}

// -----------------------------------------------------------------
// gesture events

void onCompletedGesture(SimpleOpenNI curContext,int gestureType, PVector pos)
{
  println("onCompletedGesture - gestureType: " + gestureType + ", pos: " + pos);

  int handId = context.startTrackingHand(pos);
  println("hand stracked: " + handId);
}

// -----------------------------------------------------------------

//timer
void timedShots() {
  for (;; delay(INTERVAL))  saveFrame(dataPath("####.jpg"));
}

Thanks a lot!

` import processing.opengl.*; import SimpleOpenNI.*;

SimpleOpenNI  kinect;
import saito.objloader.*;
OBJModel model;
void setup() { 
  size(1028, 768, OPENGL);
  model = new OBJModel(this, "kinect.obj", "relative", TRIANGLES);
  model.translateToCenter();
  BoundingBox box = new BoundingBox(this, model); 
  model.translate(box.getMin()); 
  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  kinect.enableUser();
  kinect.setSmoothingSkeleton(0.6);

  kinect.setMirror(true);
}`

I've included the code below just incase it helps to see how I'm working so far. Its to satisfy my project's minimum objective of being a glorified range finder. I'm using v.1.5.1 as it supports the macbook I'm working on, so sorry if this is in the wrong place.

import SimpleOpenNI.*;
import unlekker.util.*;
import unlekker.modelbuilder.*;
import processing.opengl.*;
import processing.serial.*;

SimpleOpenNI kinect;
UGeometry model;
UVertexList vertexList;
Serial myPort;

boolean scanning = false;
boolean printval = false;

boolean turn1 = false;
boolean turn2 = false;
boolean turn3 = false;
boolean turn4 = false;
boolean firstcontact = false;

String val;
PrintWriter output;

void setup ()
{
  size (940, 480);
  background(0);

  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();

  model = new UGeometry();
  vertexList = new UVertexList();

  output = createWriter("depths.csv");

  String portName = Serial.list()[0];
  myPort = new Serial(this, portName, 57600);
  myPort.bufferUntil('\n');
}

void draw ()
{
  kinect.update();

PImage depthImage = kinect.depthImage();
image(depthImage, 0 ,0);

int furthestValue = 0;

if (turn1 && !scanning)
{
  myPort.write('1');
  myPort.readStringUntil('\n');
  scanning = true;
}

if (turn2 && !scanning)
{
  myPort.write('2');
  myPort.readStringUntil('\n');
  scanning = true;
}

if (turn3 && !scanning)
{
  myPort.write('3');
  myPort.readStringUntil('\n');
  scanning = true;
}  

if (turn4 && !scanning)
{
  myPort.write('4');
  myPort.readStringUntil('\n');
  scanning = true;
  output.close(); //finishes the csv file
}

if (scanning)
{
// get the depth array from the kinect  
int[] depthValues = kinect.depthMap();
// for each row in depth image
for (int y = 0; y < 480; y++){
//look at each pixel in the row
  for(int x = 0; x < 640; x++){
// pull out the corresponding value from the depth array
    int i = x + y * 640;
    int currentDepthValue = depthValues[i];    
//if that pixel is the furthest so far
    if(currentDepthValue > 0 && currentDepthValue > furthestValue){
//save the value
      furthestValue = currentDepthValue;
    }
  }
scanning = false;
}
printval = true;
}

if (printval){
  float centimeters = furthestValue / 10;
  float inches = furthestValue / 25.4;
  println("Furthest Depth: " + furthestValue + " mm / "  
  + centimeters + " cm / " + inches + " inches");
//  output. println(centimeters + " cm"); 
  printval = false;
}
//output.close(); //finishes the csv file
// exit(); //exits program
}

void keyPressed() {
  if (key == ' ') {
    turn1 = true;
  }
}

Thank you very much in advance.

So far my results are fantastic - except for 1 joint rotation I can not figure out. When reaching up to cover the models eyes, for example, there is a point where the elbow does an unnatural rotation. It seems like when the hand joint is higher than the elbow joint the hand twists around, in an impossible way, showing the palm. Continuing to the eyes, the rotation eventually comes back to a natural position.

The sketch, and the version of NIMate's Big Buck Blender Model I'm using can be found here 1drv.ms/1BstWXN

The sketch is using simpleopenni, and a few other libraries that I've included in the link.

Surprisingly the math up to this point has been easy to understand.
Right now the bulk of the math sits in the "OSC" file inside the "sendJointMsg" function. I am rotating the majority of joints based on its relation to the body's CoM.

The only 2 that I'm not doing that with is the shoulder and the elbow. When I use CoM the positions of the arms aren't matching me. Instead, I found using a new vector of (1,0,0) satisfies those joints, but the limbs seem more rigid compared to leg motions, head motions, leaning forward/backward, etc.

Finally, one more puzzle is that using NIMates plugin in blender, you can auto-generate shapes at joint points. Using a blank blend, and enabling this option, you see a human shape and the joints seem to be rotating like you would expect. Its hard to know that for sure since there is no bone between joints though.

Any help on getting the arms to rotate naturally would be appreciated thanks

I'm using Processing 2.2.1, Kinect v.1, and i have the appropriate libraries but i have some problems to understand how con i replicate this.

Is there anyone who can help me? Thank you.

A sketch of processing forms with generative and particle I would make it react with the movement of those who pass in front of the kinect (x, y, and even z) I enclose sketch

how do I? What do you recommend to add?

void setup(){

size(1300, 800, P3D);

background(0);

stroke(100, 150);

fill(255, 255, 255);

float xstart = random(150);

float ynoise = random(150);

translate(width/2, height/2, 0);

for(float y = -(height/8); y <=(height/8); y+=3){

ynoise += 0.01;

float xnoise = xstart;

for (float x =-(width/8); x <=(width/8); x+=3) {

xnoise +=0.02;

drawPoint(x, y, noise(xnoise, ynoise));

}

} }

void drawPoint(float x, float y, float noiseFactor) {

pushMatrix();

translate(x * noiseFactor * 5, y * noiseFactor * 5, -y);

float edgeSize = noiseFactor * 6;

ellipse(2, 2, edgeSize, edgeSize);

popMatrix(); }

I am currently working on a project that tracks motion, which then draws depending on where the motion is sensed. At first, I was working with Opengl, however I now want to import a Kinect instead of using the built in camera. Please note that I am still in high school and am fairly new to processing. For the Kinect, I am using the library SimpleOpenNI.

Should I replace Opengl with Kinect or can I run them side by side without the libraries interfering with each other?

Thank you!

I'm trying to get a Kinect V2 to run on a Mac. I followed the instruction here http://blogs.msdn.com/b/kinectforwindows/archive/2014/07/28/developing-with-kinect-v2-on-a-mac.aspx installed Window To Go into an USB on Windows8, and tried booting my Mac from there but it just went to a black screen. My USB port is 3.0 as it requires. Just wondering if anyone got Kinect V2(or even windows to go) working on a Mac?

Thanks!!