We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpPrograms › Control mouse coordinates
Pages: 1 2 
Control mouse coordinates (Read 6420 times)
Control mouse coordinates
Nov 8th, 2008, 6:59pm
 
Hello all!

I'm new into Processing but from what I was already able to see and test it seems excellent.

I'd like to know if is possible to control the position of my mouse cursor through Processing.

I've seen many examples that read the mouse coordinates and use them. What I'd like to do is to "write" the mouse coordinates, instead.

Maybe this is trivial in Processing but I still haven't figured it out.

Any help on this?

Thanks in advance!

Best regards!
Re: Control mouse coordinates
Reply #1 - Nov 8th, 2008, 7:24pm
 
I wonder what do you want to do. Do you need to move the real cursor or can it be fake

In the first case, maybe the Robot class might help you (never tried it yet).
Re: Control mouse coordinates
Reply #2 - Nov 8th, 2008, 7:34pm
 
I would like to try to move my cursor with face tracking...

I would like to move the real cursor in order to make further actions, such as clicking...

I'll try what you've suggested.

Are there any more suggestions?

Thank you very much!
Re: Control mouse coordinates
Reply #3 - Nov 9th, 2008, 5:32pm
 
Hello!

I've been trying to use the Robot class but I didn't succeed. Sad

Is there the need to install or have anything aditional, such as a Java development kit or libraries?

Thanks in advance!

Best regards!
Re: Control mouse coordinates
Reply #4 - Nov 10th, 2008, 9:02am
 
Base test:
Code:
int xx = 10, yy = 10;
Robot robby;

void setup()
{
size(500, 500);
try
{
robby = new Robot();
}
catch (AWTException e)
{
println("Robot class not supported by your system!");
exit();
}
}

void draw()
{
xx = (xx + 2) % width;
yy = (yy + 2) % height;

// Might need to confine to sketch's window...
robby.mouseMove(xx, yy);
}

Perhaps you were stuck by the unhandled exception?
Re: Control mouse coordinates
Reply #5 - Nov 10th, 2008, 11:01am
 
Hey! Cheesy

This works perfectly!

Really, thank you very much, great! Wink

Best regards!
Re: Control mouse coordinates
Reply #6 - Jun 2nd, 2009, 4:10am
 
hello
I have used this robot function to control the mouse with face detection (openCV)  but im having trouble getting the mouse to stop jumping around. I would love it to run smoothly like the one at processing.org/hacks/hacks:robot

any suggestions would be hugely appreciated.


heres ma code:

import hypermedia.video.*;

OpenCV opencv;
Robot follow;

// press esc to exit

void setup() {

   size( 640, 480 );
try
 {
   follow = new Robot();
 }
 catch (AWTException e)
 {
   println("Robot class not supported by your system!");
   exit();
 }
   opencv = new OpenCV(this);
   opencv.capture( width, height );
   opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT);    // load the FRONTALFACE description file


}

void draw() {
   
   opencv.read();
    opencv.flip( OpenCV.FLIP_HORIZONTAL );
   opencv.convert( OpenCV.GRAY );
   
   
   image( opencv.image(), 0, 0 );
   // detect anything ressembling a FRONTALFACE
 //  Rectangle[] faces = opencv.detect();
   
Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );
   
   // draw detected face area(s)
   noFill();
   stroke(25,0,0);
  ellipseMode(CORNER);
   for( int f=0; f<faces.length; f++ ) {
   
   
int     X = (faces[f].x);
       int Y = (faces[f].y);
     int A = ( faces[f].width);
       int B = (faces[f].height);
       
          ellipse( X, Y, A, B );
         
            follow.mouseMove(X*5, Y*5);
           
   }}
Re: Control mouse coordinates
Reply #7 - Jun 2nd, 2009, 4:23am
 
Maybe the Arduino tutorial on Smoothing will help...  Basically create an array for each value you want to smooth, and in your loop populate it as data is received from OpenCV and divide the total of all values by the array length to get an average.  The length of the array determines the number of readings being averaged and therefore how much smoothing is applied.

I was able to use this technique recently in a Python script that smoothed accelerometer input from a Wii Nunchuck - seems pretty effective.  Having said that I wonder whether OpenCV already has something built in to achieve the same result
Re: Control mouse coordinates
Reply #8 - Jun 16th, 2009, 10:41pm
 
This is really useful, thanks for working out the exception part.
Re: Control mouse coordinates
Reply #9 - Apr 6th, 2010, 12:39pm
 
I have been running into an error trying to run code in processing, I 'm getting the error message "Cannot find a class or type named Robot". I don't have too much experience with java so I'm guessing my problem may be from me overlooking something that should be obvious.

I'm attempting to create a FTIR setup using openCV and have had great results so far except for using the robot class. I appreciate any help or advice. Thanks!

[code]// FTIR Multitouch Server
// Kyle Doerksen. IDEO. 2008


/* This blobThreshold is overwritten by the calibration XML file, so this is only for initial use */
float blobThreshold=0.12;

/* Set this to true if you're using a fisheye lens */
boolean doFisheyeCorrect = false;

/* This is the threshold of how far a blob can move before it is considered a new blob.
This is actually the square of the number of pixels moved, so in this case 30^2 = 900. */
int distanceThreshold = 900;

import proxml.*;              // library for xml read write (used for configuration files
import processing.net.*;      // used for network connectivity -- sends finger co-ordinates over a socket
import blobDetection.*;       // used for interpreting webcam image
import JMyron.*;              // used for camera connectivity

JMyron m; //a camera
PFont font; // need a font if we're going to use text

Server touchServer;           // server object to handle communication of touch events

int MINIMUM_DRAW=1;           // allows multiple levels of graphical verbosity.
int NORMAL_DRAW=2;            // once everything is dialed in, minimum_draw level should be used.
int MAX_DRAW=3;
int drawLevel=MAX_DRAW;
// higher drawing levels provide more useful information but slow down the execution.

// Perspective correction
int HEIGHT=768;
int WIDTH=1024;
int edgeDistance=250;
// these are the points used for calibration
double calX[] = {
 edgeDistance,WIDTH/2,WIDTH-edgeDistance,WIDTH-edgeDistance,WIDTH-edgeDistance,WI
DTH/2,edgeDistance,edgeDistance};
double calY[] = {
 edgeDistance,edgeDistance,edgeDistance,HEIGHT/2,HEIGHT-edgeDistance,HEIGHT-edgeD
istance,HEIGHT-edgeDistance,HEIGHT/2};
double measuredX[] = {
 0,0,0,0,0,0,0,0};
double measuredY[] = {
 0,0,0,0,0,0,0,0};
int nCalPoints =8; // must be equal to the length of the above arrays
perspectiveCorrection cal = new perspectiveCorrection();
int calibrating=-1;

// cursors
boolean mouseEmulation=false; // generates mouse messgages and clicks if true.
TuiCursorList cursorList=new TuiCursorList();

Exception e;

// Video Capture/Blob Detection stuff:
BlobDetection theBlobDetection;     // this handles blob tracking
PImage img;                         // the current frame from the webcam
PImage bgimg;                      // the background image
boolean newFrame=false;
int oldnBlobs;

XMLInOut xmlInOut;

void setup()
{
 font = loadFont("CourierNew36.vlw");
 textFont(font);
 // strangely, because our xmlEvent function is part of the PerspectiveCorrection class
 // we pass cal as the second parameter to this constructor to indicate where to look for that xmlEvent function
 xmlInOut = new XMLInOut(this, cal);
 size(WIDTH, HEIGHT);  // Size of applet
 imageMode(CORNER);

 m = new JMyron();//make a new instance of the object
 m.start(320,240);//start a capture at 320x240
 m.findGlobs(0);//disable the intelligence to speed up frame rate

 // BlobDetection
 // img which will be sent to detection (a smaller copy of the cam frame);
 img = new PImage(320,240);
 bgimg = new PImage(img.width, img.height);

 theBlobDetection = new BlobDetection(img.width, img.height);
 theBlobDetection.setPosDiscrimination(true);
 cal.readCalibration(xmlInOut);
 // will detect bright areas whose luminosity > blobThreshold;
 theBlobDetection.setThreshold(blobThreshold);

 touchServer = new Server(this, 11000);
}

float startTime;

void draw()
{
 m.update();//update the camera view

 if(drawLevel>=NORMAL_DRAW){
   background(0,0,0);
 }

 m.imageCopy(img.pixels);

 if(frameCount==12)      // waits twelve frames for things to stabilize and then captures the background image.
 {
   bgimg.copy(img,0,0,img.width, img.height,0,0,img.width, img.height);
   fastblur(bgimg,2);
 }

 fastblur(img, 2);  //  fast blur filters out some spatial noise.
 // background subtraction
 Diff(img,bgimg);   // calculates the difference between the image and the original background image

 theBlobDetection.computeBlobs(img.pixels);

 if(drawLevel==MAX_DRAW)
 {
   img.updatePixels();
   image(img,0,0,width,height);
   drawBlobsAndEdges(true,true);
 }

 int nBlobs = theBlobDetection.getBlobNb();
 Blob myBlob;
 Point correctedPoint = new Point();

 float r, r_corrected;
 float theta;
 float x_cor, y_cor;
 int i;

 CursorObject curObject;

 for(i=0;i<nBlobs;i++) {    
   myBlob = th
Re: Control mouse coordinates
Reply #10 - Apr 6th, 2010, 1:20pm
 
i guess it is due to the changes made in the latest processing version.
and the fact that you have to import the java classes.

i had a similar problem. maybe i can add my question here. How do i know what to import ?

I am not sure, maybe in your case "import java.awt.Robot; " will help.

Re: Control mouse coordinates
Reply #11 - Apr 6th, 2010, 5:06pm
 
Code:

Robot robot;

 try {
   robot = new Robot();
 }
 catch (Exception e) {
   println("Can't Initialize the Robot");
 }

robot.mouseMove(xpos,ypos);


A common tactic is to track mouse delta, and constantly reset the mouse location (hidden) to the center (every 40 ticks or so works well for me).  That way, you can show a false cursor at a location determined by delta, while keeping the actual mouse centered.

Edit: Thanks PhiLo -- removed that line.
Re: Control mouse coordinates
Reply #12 - Apr 7th, 2010, 5:53am
 
I don't want to be rude, BenHem, but import java.lang.Object.*; makes no sense! Wink

- It would be import java.lang.*; or import java.lang.Object; but not both.
- You never need to import java.lang package, it is always done by the compiler itself.

Smiley
Re: Control mouse coordinates
Reply #13 - Apr 7th, 2010, 9:52pm
 
To get your sketch to work with Processing 1.1 you need to add this:

Code:
import java.awt.Robot;
import java.awt.AWTException;
import java.awt.event.InputEvent;
Re: Control mouse coordinates
Reply #14 - Apr 8th, 2010, 1:59pm
 
Thanks everyone for your quick responses. I added Code:
import java.awt.Robot;
import java.awt.AWTException;
import java.awt.event.InputEvent;
and it seems to get me past the the issue I had with the robot class.

Now the hang up seems to be with Code:
if(mouseEmulation==true) 

returning the error message " expecting EOF, found 'if' "

I'm not sure if it's a bug or a compatibility issue trying to run the sketch on Processing 1.1.

Thanks again!
Pages: 1 2