Robot Class in processing

Aaron Jam..

in Contributed Library Questions • 1 year ago

Hi Everyone,

I've used the kinect to follow my hand and use the robot class to control the mouse, but its limitation is that once my hand reachs the top of the sketch it wont go further.

I know you can fullscreen your sketch, but is it possible to make it completly transparent/ or a way to lay it under the desktop so i can still inter act with everything?

i've missed out the kinect callback stuff and my other class's this is where the main magic happens,

I was thinking a way to increase the movement generated by the skech to the cursor how would i approch it ?

void drawHands(){
   int userId = calibratedUserID;
//make a vector to store the left hand
        PVector rightHand = new PVector();
        //put the position of the left hand into the vector
        float confidence = kinect.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_LEFT_HAND, rightHand);
        //convert the detected hand postion, to projective co-ords, that match the depth image
        PVector convertedRightHand = new PVector();

        kinect.convertRealWorldToProjective(rightHand, convertedRightHand);

      //display it
      fill(255,0,0);
      ellipse(convertedRightHand.x, convertedRightHand.y, 10, 10);



       mouseX = int(convertedRightHand.x) ;
       mouseY = int(convertedRightHand.y) ;
         robby.mouseMove(mouseX, mouseY);
}

Thank you for looking and any help would be great!

Mirror is set true

Replies(10)

calsign

Re: Robot Class in processing

1 year ago

In other words, you want to make a fullscreen application to control a mouse with a Kinect.
In order to be able to interact with everything else, you need to shove the application into the back and remove its focus.
You're going to need to meddle with the frame object...

Perhaps something like this:

void setup() {
size(screenWidth, screenHeight);
frame.removeNotify();
frame.setUndecorated(true);
frame.setFocusableWindowState(false);
frame.addNotify();
com.sun.awt.AWTUtilities.setWindowOpacity(frame, 0);
}
void draw() {
if(frameCount == 1) frame.setLocation(0, 0);
if(frameCount == 2) frame.toBack();
}

This required maximizing the frame, moving it to the top left corner, making it unfocusable, and making it fully transparent.

If you want to draw anything, you have to increase the number on line 9 to a visible amount.
If you want access to the mouse coordinates, you need to increase it to at least 0.01.

The value must be contained within 0 (fully transparent) and 1 (opaque).

Aaron Jam..

Re: Robot Class in processing

1 year ago

Thanks alot works brilliantly,

Its more ore less what i wanted it to be, but, i control the mouse with the left hand, it doesn't like going over the edge of either the sketch, which is invisble or past the left side of my body, i assume its the 2nd as im using skeleton data.

i do have a solution in mind but im not sure how to do it,

ive tried if "convertedRightHand.x == convertedLeftHand.x) Ihave a sketch that tracks both hands, ive put in an if statment similar to the one above to try to swap colours let alone control of the cursor, its not quite working

could you suggest anything?

calsign

Re: Robot Class in processing

1 year ago

I'm afraid that I don't understand your problem, now...
Kinect isn't exactly my area of expertise. If the problem is more Kinect-oriented than before, I don't think I can help you.

Aaron Jam..

Re: Robot Class in processing

1 year ago

Yes it looks more like a kinect based problem now,

but the idea i had is purley processsing,

i have varibles containing the x, y co ordinates, an eclipse on each hand. all i need is some sort of colistion from eclipse 1 and 2 to swap the mouse control to the other hand.

Cheers for the quick replys!

calsign

Re: Robot Class in processing

1 year ago

In other words, you need circle collision?
So... you have the coordinates for both hands, all you need is the diameter of the circles.
The actual collision is simple: If the distance between the two circles is less than their combined radii, then they are touching.

For example:

if(dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < leftHandRadius + rightHandRadius) //They are touching

I'm not sure how you will find the repspective radii, but I imagine it's simple...

Next, you need to change mouse control.
The problem that you may encounter is that the control will change hands constantly as long as the hands are touching... you need a way to fix that.

The best way is to create a global boolean variable...
When the hands touch, switch hand control and assign the variable to be true.
First, however, check to make sure that the variable is false... before doing that.
When the hands are not touching, assign the variable to be false.

Try that...

Aaron Jam..

Re: Robot Class in processing

1 year ago

thanks yet again it helps! alot, ok so atm, i've got both eclipes drawn on both hands

is this what you mean? i then call it llater to turn false / true
boolean leftHandSwap = false;

      fill(255,0,0);
      ellipse(convertedRightHand.x, convertedRightHand.y, 10, 10);
      ellipse(convertedLeftHand.x, convertedLeftHand.y, 10, 10);

but im not sure how to get teh dadius, i'll look it up now!

Thats all i need as the varibles are convertedLeftHand.x, convertedLeftHand.y,

could the ,10, 10 be its first static position on draw i mean its x and y co-ords?

calsign

Re: Robot Class in processing

1 year ago

The radius is half the diameter (10); in this case, 5.
That would look like:

if(dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < 10) //They are touching

Aaron Jam..

Re: Robot Class in processing

1 year ago

Worked great, but now im trying to right en if statment for it and it doesn't like it, so ive tried putting them into seperate functions, only to realise its a top down compiler and it now stays on the right hand!

excuse the poor names but these are teh two functions that dont like going together!

void mouseControl(){
if(dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < 15);{ //They are touching
       mouseX = int(convertedRightHand.x);
       mouseY = int(convertedRightHand.y);
       robby.mouseMove(mouseX, mouseY);
}

if (mouseX = int(convertedRightHand.x), mouseY = int(convertedRightHand.y)){
else {
    (dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < 15);{ //They are touching
       mouseX = int(convertedLeftHand.x);
       mouseY = int(convertedLeftHand.y);
       robby.mouseMove(mouseX, mouseY);
}
}

it jsut gives me a Rcurly error when i try the if else statment :/

calsign

Re: Robot Class in processing

1 year ago

First thing: Go to Edit > Auto Format to clean up your code; it also makes it easier to spot errors.

There were numerous things wrong with your code, here is a short list:

Use "==" for comparing values, not "="; if you do that, the compiler will think that you are trying to assign something!
Don't put semicolons directly after if statements with set braces, they aren't necessary and may cause errors
You can use the "!" (not) operator to find the opposite of a boolean value

There may have been more errors with your code, but those were the ones that stood out the most.
In addition, you were missing a closing set brace...

Here is the fixed code, hopefully it should work as expected:

void mouseControl() {
if(dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < 15) {
mouseX = int(convertedRightHand.x);
mouseY = int(convertedRightHand.y);
robby.mouseMove(mouseX, mouseY);
}
if(!(mouseX == int(convertedRightHand.x) && mouseY == int(convertedRightHand.y))) {
if (dist(convertedRightHand.x, convertedRightHand.y, convertedLeftHand.x, convertedLeftHand.y) < 15) {
mouseX = int(convertedLeftHand.x);
mouseY = int(convertedLeftHand.y);
robby.mouseMove(mouseX, mouseY);
}
}
}

Aaron Jam..

Re: Robot Class in processing

1 year ago

I wondered we're auto correct was in processsing, i've never used it for this purpose so thanks for the advice!

its not flagging any errors now though, but its not controllling the mouse at all sigh, sorry half the mistakes are because ive been programming for over 12 hours now, no excuse though!

which value is the boolean in your code? as mouseX is a float built in.

this is the simple function to follow one of my hands, * by 3.5 allows me to reach the whole of one side of the screen thats why im trying to get this ability to swap hands!

void swaphands(){
       mouseX = int(convertedLeftHand.x * 3.5);
       mouseY = int(convertedLeftHand.y * 3.5);
       robby.mouseMove(mouseX, mouseY);
}

Im stopping for tonight but when ive slept, i'll go over the code again in the morning, il probably have loads hanging off the screen but thanks for the help so far its been of great value!

Top Reply