How to take control of the mouse outside the sketch window ?

SimJ

Hi, I wrote a code that selects a color from the camera input and tracks this color. I want to use this as a remote to control my computer with a little colored object (works fine with a pen). Now I would like to run the sketch in background and control the mouse with my pen. I have absolutely no idea how to do that, any suggestions ?

MenteCode

Would the Robot class be what you're looking for?

Be careful with the sketch below, though. If you accidentally click on another window, depending on what you know about your operating system, you may or may not be able to close the sketch window, anymore.

import java.awt.Robot;

Robot rbt;

void setup() {
  size(300, 300);

  try {
    rbt = new Robot();
  } catch(Exception e) {
    e.printStackTrace();
  }
}

void draw() {
  int x = displayWidth/2 + (int) (sin(millis()/300f) * 200);
  int y = displayHeight/2 + (int) (cos(millis()/300f) * 200);

  rbt.mouseMove(x, y);
}

Otherwise, I'm sure you can figure out what code performs what action.

SimJ

That's perfect, thank you !

TechWiz777

Could you also use something like this to click on the external program, like my computer for example? Can I use this code and what should I add to it that can aid in that?

MenteCode

@TechWiz777 With the Robot class, you can do essentially anything involving the keyboard and mouse, in any order. You can even shut your own computer down, if you can to do it accurately.

Anything listed in the "Method Summary" section of the following link can be used to help you with mouse/keyboard input needs: http://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html

If you need further help, feel free to PM me, or make a new post. Let's not litter the OP's notification box with comments from an outside conversation. ;)

TechWiz777

@MenteCode thank you!

Jai

@MenteCode is there a way that we can have this sketch you provided click on a specific point in the screen ? using just x&y coordinates ? thats be so much easy for me since im wayyyy to new to the whole trig functions/parameters "cos,sin...ect"

AverageJoe

Sure, this is what I do:

void draw() {
  int x = 1910; // Window Close button.
  int y = 5;
  rbt.mouseMove(x, y);
}

MenteCode

@Jai:

Exactly what @AverageJoe said.

Don't worry about the sin and cos functions. They're there just so the cursor would move in a circle, as a demonstration of what you can do with the Robot class. :)