Speeding up Open CV detect face

CharlesDesign

Hi All,

I'm trying to implement a real time face detection using openCV from the IR image of the kinect, unfortunately it takes my sketch from 60fps down to 6fps. I am aware kinectPV2 does face detection but it's nowhere near as good as openCV. Can someone suggest a solution? I've tried the multithreaded "your are einstein" sketch but I couldn't get it to run.

import KinectPV2.*;
import gab.opencv.*;
import java.awt.Rectangle;

KinectPV2 kinect;

FaceData [] faceData;

OpenCV opencv;
Rectangle[] faces;

PImage img;

void setup() {
  size(1000, 500, P2D);

  opencv = new OpenCV(this, 512, 424);
  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  

  kinect = new KinectPV2(this);

  //for face detection base on the infrared Img
  kinect.enableInfraredImg(true);

  //enable face detection
  kinect.enableFaceDetection(true);

  kinect.enableDepthImg(true);

  kinect.init();
}

void draw() {
  background(0);
  img = kinect.getInfraredImage(); //512 424


  opencv.loadImage(img);
  faces = opencv.detect();



  image(img, 0, 0);
  image(kinect.getDepthImage(), img.width, 0);

  fill(255);
  text("frameRate "+frameRate, 50, 50);

  noFill();
  for (int i = 0; i < faces.length; i++) {
    rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
  }
}

jeremydouglass

@CharlesDesign -- If the Kinect is 30fps you can start with frameRate(30) (or 27, or 24). That should save wasted effort trying to recalculate each unchanging camera frame twice at 60fps.

Then you could also drop the framerate on the rect() updating, independent of the higher video framerate -- perhaps texts updated at 15 or 10 fps, something like this (untested):

// at frameRate(27)....
void draw() {
  background(0);
  img = kinect.getInfraredImage();
  image(img, 0, 0);
  image(kinect.getDepthImage(), img.width, 0);
  fill(255);
  text("frameRate "+frameRate, 50, 50);

  if(frameCount%3==0){ // 9fps at frameRate(27)
    opencv.loadImage(img);
    faces = opencv.detect();
    noFill();
    for (int i = 0; i < faces.length; i++) {
      rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height);
    }
  }
}

That's not speeding up your processor-intensive openCV calls, but it is hopefully letting your video run at full speed under slower rects -- and possibly speeding up the rects too since the overall load has dropped as well.

CharlesDesign

Good suggestion, thanks! :)