We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › Control mouse cursor with face
Page Index Toggle Pages: 1
Control mouse cursor with face (Read 889 times)
Control mouse cursor with face
Jan 20th, 2009, 12:55am
 
Hello all!

I wrote the code below to control the mouse cursor with the position of the center of my face relatively to a grid.

I'm having some doubts/lack of imagination:
1 - is it possible to keep my sketch window always on front (visible, king of floating over the other windows?)

2 - how do I thick the point I overlay on the image on the position corresponding to the center of the detected face?
I already tried to use strokeWeight(10) before point(...) but the grid also got thick. I tried pushMatrix() strokeWeight() point() popMatrix() but the result was the same.

3 - Would you do this (control the mouse with your face) in the same way? If not, how would you do it?

You can access the .xml files for face and hand detection on the following links:
http://inside.webhs.org/aGest.xml

http://inside.webhs.org/haarcascade_frontalface_alt.xml

These files should be placed inside a folder named "data" inside the sketch folder.

Any help would be greatly appreciated.

Thank you very much in advance.

Best regards!

import JMyron.*;
import pFaceDetect.*;

JMyron m;
PFaceDetect face;
PFaceDetect hand;
PImage img;

Robot robby;

int search_width = 320;
int search_height = 240;
float actualX=search_width/2;
float actualY=search_height/2;
int x_rato = 0;
int y_rato = 0;

void setup() {  
 size(320,240,P3D);  
 m = new JMyron();  
 m.start(width,height);  
 face = new PFaceDetect(this,width,height, "haarcascade_frontalface_alt.xml");
 hand = new PFaceDetect(this,width,height, "aGest.xml");
 
 img = createImage(width,height,ARGB);  
 noFill();  
 stroke(255,0,0);  
 
 try
 {
   robby = new Robot();
 }
 catch (AWTException e)
 {
   println("Robot class not supported by your system!");
   exit();
 }
}  

void draw() {  
 m.update();  

 //code for flipping  
 int[] imgNormal = m.cameraImage();  
 this.loadPixels();  
 for(int i=1; i<width;i++){  
   for(int j=1;j<height;j++){  
    this.pixels[(m.width() - i - 1) + j * m.width()] = imgNormal[(i) + j * m.width()];  
   }  
 }  
 this.updatePixels();  

 arraycopy(m.cameraImage(),img.pixels);
 
 face.findFaces(img); //chama função que detecta caras
 drawFace(); //desenha info relativa à posição das caras detectadas
 hand.findFaces(img);
 drawHand();
 
  // code for drawing the lines    
  line((width/2-10),0,(width/2-10),height);
  line((width/2+10),0,(width/2+10),height);
  line(0,(height/2-10),width,(height/2-10));
  line(0,(height/2+10),width,(height/2+10));
 
  //verifica posição do ponto
  //diagonal subir para direita
  if (actualX < (width/2-10) && actualY < (height/2-10))
   {
     x_rato = x_rato + 4;
     y_rato = y_rato - 4;
   }
   //diagonal subir para esquerda
   else if (actualX > (width/2+10) && actualY < (height/2-10))
   {
     x_rato = x_rato - 4;
     y_rato = y_rato - 4;
   }
   //subir
   else if (actualX < (width/2+10) && actualX > (width/2-10) && actualY < (height/2-10))
   {
     y_rato = y_rato - 4;
   }
   //descer
   else if (actualX < (width/2+10) && actualX > (width/2-10) && actualY > (height/2+10))
   {
     y_rato = y_rato + 4;
   }
   //diagonal descer para direita
   else if (actualX < (width/2-10) && actualY > (height/2+10))
   {
     x_rato = x_rato + 4;
     y_rato = y_rato + 4;
   }
   //diagonal descer para esquerda
   else if (actualX > (width/2+10) && actualY > (height/2+10))
   {
     x_rato = x_rato - 4;
     y_rato = y_rato + 4;
   }
   //movimento horizontal da direita para a esquerda
   else if (actualX > (width/2+10) && actualY < (height/2+10) && actualY > (height/2-10))
   {
     x_rato = x_rato - 4;
   }
   //movimento horizontal da esquerda para a direita
   else if (actualX < (width/2-10) && actualY < (height/2+10) && actualY > (height/2-10))
   {
     x_rato = x_rato + 4;
   }
   print(actualY+"\n");
   robby.mouseMove(screen.width/2+x_rato, screen.height/2+y_rato);
}

void drawFace() {
 int [][] res = face.getFaces();
 if (res.length>0) {
   for (int i=0;i<res.length;i++) {
     int x = res[i][0];
     int y = res[i][1];
     int w = res[i][2];
     int h = res[i][3];
     x = x + w/2;
     actualX = round(actualX + 0.1*(x-actualX));
     y = y + h/2;
     actualY = round(actualY + 0.1*(y-actualY));
     /*pushMatrix();
     strokeWeight(10);*/
     point(width-actualX, actualY);
     //popMatrix();
   }
 }
}

void drawHand() {
 int [][] res = hand.getFaces();
 if (res.length>0) {
   print("ai"+"\n");
   robby.mousePress(InputEvent.BUTTON1_MASK);
   robby.delay(500);
   robby.mouseRelease(InputEvent.BUTTON1_MASK);
   }
}


void stop() {
 m.stop();
 super.stop();
}
Re: Control mouse cursor with face
Reply #1 - Jan 31st, 2009, 3:56am
 
hmm. i didnt download your code and test it... but if your only problem is to make the point thicker just use ellipse() insteadt and give it a size you want...

and just something else...
pushmatrix etc. is only for rotating transfering a matrix...

if you want strokeWeight affect only your line for example you have to do it like this :

strokeWeight(100); //make it thick
line(x,y,y2,y2);
strokeWeight(1); //set it back to normal

and for the other problem, maybe use a programm like : http://www.fadsoft.net/AlwaysOnTopMaker.htm

to make your applet stay over all other windows on your desktop...
Re: Control mouse cursor with face
Reply #2 - Jan 31st, 2009, 10:20am
 
frame.setAlwaysOnTop(true);
Re: Control mouse cursor with face
Reply #3 - Jan 31st, 2009, 7:33pm
 
ah thats good to know Smiley much better
Re: Control mouse cursor with face
Reply #4 - Feb 1st, 2009, 10:09am
 
Note this method has been introduced with Java 1.5. So with Processing 0135, I suppose one would use the library you point to.
Page Index Toggle Pages: 1