Kinect Colour Tracking Issue

dungnud11

I've been trying to get Kinect and colour tracking together, and recently managed to get some sort of breakthrough. The code is below, but there are a still a few more issues that I found. When I placed in the size in regards to display, the kinect only seemed to capture the top-left part of the screen. Also, another issue I noticed was when I clicked on a colour to track, it seemed that the tracking was following something else (tracking was all over the screen, did not land anything). I am not exactly sure if it is due impart with the size -- but when I increased, the tracker would go off-screen too. Any help would be appreciated, thanks!

    //import processing.video.*;
    import org.openkinect.freenect.*;
    import org.openkinect.freenect2.*;
    import org.openkinect.processing.*;
    import org.openkinect.tests.*;

    // Variable for capture device
    Kinect2 kinect2; 

    // A variable for the color we are searching for.
    color trackColor; 

    // shows what users see 
    PImage display;

    // location  
    PVector loc;

    void setup() {
      size(640, 480);
      // Start off tracking for red
      trackColor = color(0, 0, 255); // blue colour 
      kinect2 = new Kinect2(this);
      kinect2.initVideo();
      kinect2.initDevice();
      display = createImage(kinect2.depthWidth, kinect2.depthHeight, RGB);
      background(0);

      // setup the vectors 
      loc = new PVector(0, 0); 
    }

    void draw() {
      // display.updatePixels();

      display = kinect2.getVideoImage(); 
      image(display, 0, 0); 

      display.loadPixels(); 

      // worldRecord and 
      // XY coordinate of closest color 
      float worldRecord = 750;
      int closestX = 0;
      int closestY = 0;

      // Begin loop to walk through every pixel
      for (int x = 0; x < kinect2.depthWidth; x ++ ) {
        for (int y = 0; y < kinect2.depthHeight; y ++ ) {
          int loc = x + y*kinect2.depthWidth;
          // What is current color
          color currentColor = display.pixels[loc];
          float r1 = red(currentColor);
          float g1 = green(currentColor);
          float b1 = blue(currentColor);
          float r2 = red(trackColor);
          float g2 = green(trackColor);
          float b2 = blue(trackColor);

          // Using euclidean distance to compare colors
          float d = dist(r1, g1, b1, r2, g2, b2); // We are using the dist( ) function to compare the current color with the color we are tracking.

          // If current color is more similar to tracked color than
          // closest color, save current location and current difference
          if (d < worldRecord) {
            worldRecord = d;
            closestX = x;
            closestY = y;
            display.updatePixels();
            image(display, kinect2.depthWidth, kinect2.depthHeight, 0, 0); // draw image out 
          }
        }
      }

      // We only consider the color found if its color distance is less than 10. 
      // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
      if (worldRecord < 10) { 
        // Draw a circle at the tracked pixel
        fill(trackColor);
        strokeWeight(4.0);
        stroke(0);
        ellipse(closestX, closestY, 16, 16);
      }
    }

    void mousePressed() {
      // Save color where the mouse is clicked in trackColor variable
      int loc = mouseX + mouseY*display.width;
      println(loc);

      trackColor = display.pixels[loc];
    }