Track color within a certain distance (Mix color tracking and depth?)

agareau

Hi, I've been experimenting with Processing 3 for the past weeks (Checking video from Daniel Shiffman) And now, I'm trying to do something I'm not even sure it can be done easily.

I want to know if I can mix some color tracking and a distance threshold. My plan is to have 3 objects of differents color, and people can use them to "paint" on a glass with the result showing on a wall (Must absolutly use the physical object because it's an art project).

But the problem is, people could have clothes with the same color as the object, so we want to implement some kind of minimal distance for the color detection. I already try minimal threshold distance with Kinect, but I wonder if I can mix them both? It's quite specific I'll admit, but maybe someone as an idea? Or other solution?

Thanks!

(For clarification, the painting part is not done by me, I'll just sent the position of the object to a server)

cameyo

A method to calculate color distance:

float colorDist(color c1, color c2)
{
  float rmean =(red(c1) + red(c2)) / 2;
  float r = red(c1) - red(c2);
  float g = green(c1) - green(c2);
  float b = blue(c1) - blue(c2);
  return sqrt((int(((512+rmean)*r*r))>>8)+(4*g*g)+(int(((767-rmean)*b*b))>>8));
}

koogs

but he's talking physical distance not just colour simularity.

moving to kinect category, you might get better answers there.

agareau

Thanks, hoping others Kinect user can help!

Actually, I've come close to something working (more like a early prototype), but I'm stuck, trying to mix to things.

I've been adapting this example:

To display an crosshair for red color object (Eventually, there would be a crosshair for blue and green). But the example seem to only work with a webcam... I'm trying to use the color image from Kinect to do the same things, without any result.

Here the code so far:

import KinectPV2.*;
import gab.opencv.*;
import processing.video.*;
import processing.sound.*;


KinectPV2 kinect2;
PImage img;
PImage img2;
float minThresh = 500;
float maxThresh = 600;
int posXSquareEng = 100;
int posYSquareEng = 100;
int posXSquareFr = 400;
int posYSquareFr = 100;
int widthSquare = 100;
int heightSquare = 100;

int imgWidth = 512;
int imgHeight = 424;
int PADim = (imgWidth)*(imgHeight);


Capture macCam;

float[] rednessArray = new float[PADim];


void setup() {

  size(512, 424);
  kinect2 = new KinectPV2(this);
  kinect2.enableColorImg(true);
  kinect2.enableDepthImg(true);

  kinectColor.enableColorImg(true);
  //Start Windows
  kinect2.init();


  img = createImage(kinect2.WIDTHDepth, kinect2.HEIGHTDepth, RGB);


}

void draw() {
  background(0);

  img.loadPixels();

  countEng = 0;
  img.updatePixels();
  image(img, 0, 0);
  img2 = kinect2.getColorImage(); 
  image(img2,0,0); 

  noFill();
  stroke(255);
  rect(posXSquareEng, posYSquareEng, widthSquare, heightSquare);
  rect(posXSquareFr, posYSquareFr, widthSquare, heightSquare);

  //camSetup();

  int[] pixelArray = img2.pixels; // If I were using a normal webcam, it would be int[] pixelArray = macCam.pixels;

  rednessArray = populateRednessArray(pixelArray);
  rednessArray = clippers(rednessArray);

  int[] centroid = centroid(rednessArray);

  drawCrosshairs(centroid);

  rect(centroid[0]-(widthSquare/2), centroid[1]-(heightSquare/2), widthSquare, heightSquare);

  println(str(centroid[0]) + " " + str(centroid[1]));

  // windows
  int[] depth = kinect2.getRawDepthData();

  for(int x = 0 ; x < kinect2.WIDTHDepth; x++){
    for(int y = 0 ; y < kinect2.HEIGHTDepth; y++){
      int offset = x + y * kinect2.WIDTHDepth;
      float d = depth[offset];

      if(d > minThresh && d < maxThresh){

        if((x < centroid[0] + widthSquare/2) && y < (centroid[1] + heightSquare/2) && x > centroid[0]-(widthSquare/2) && y > centroid[1]-(heightSquare/2) ){
          println("Sould Paint");

          img.pixels[offset] = color(0,0, 255);
        }else{
          img.pixels[offset] = color(255, 0, 150);
        }

      } else {
        img.pixels[offset] = color(0);
      }

    }
  }

}


void camSetup(){
   if(macCam.available()){
    macCam.read(); 
   }

   macCam.loadPixels();

   //macCam.updatePixels();


   //image(macCam,0,0);

}


float[] populateRednessArray(int[] pixelArray) {
  float[] rednessArray = new float[PADim];

  for(int i=0;i<PADim;i++){
    rednessArray[i] = redness(pixelArray[i]);
  }
  return rednessArray;

}


float[] clippers(float[] rednessArray){
  float cutoff = 0.175;
  for(int i=0; i<PADim; i++){
    if(rednessArray[i] > cutoff){
      rednessArray[i] = 1;
    } else {
     rednessArray[i] = 0; 
    }
  }
  return rednessArray;
}

int[] centroid(float[] clippedArray){
  int[] centroid = new int[2];
  int count = 1;
  int xCenter = 0;
  int yCenter = 0;
  int countCutoff = 100;

  for(int i=0; i<PADim; i++){
   if(clippedArray[i] == 1){
    count +=1;
    xCenter +=i%imgWidth;
    yCenter +=(i - i%imgWidth)/imgWidth;
   }
  }

  if(count > countCutoff){
    centroid[0] = xCenter/count;
    centroid[1] = yCenter/count;
  } else {
   centroid[0] = 0;
   centroid[1] = 0;
  }
  return centroid;
}

void drawCrosshairs(int[] centroid){
 int xPos = centroid[0]; 
 int yPos = centroid[1];

 stroke(255);
 line(xPos, 0, xPos, imgHeight);
 line(0,yPos, imgWidth, yPos);
 ellipseMode(RADIUS);
 noFill();
 ellipse(xPos, yPos, 8,8);

}