Processing 1.0 - Processing Discourse - Translating (2D) mouse coordinates to 3D

Markavian

Translating (2D) mouse coordinates to 3D
May 16^th, 2005, 9:13pm

Ref:
http://processing.org/discourse/yabb/YaBB.cgi?board=Contribution_Responsive;action=display;num=1045181982

After following through the trials of earlier P5'ers, whats the current status of this?

http://processing.org/reference/modelX_.html
'Coming soon...?'

Is this functino operational or not, if so can you give a working example? I've included the following in my code and it only does as I expect as long as I don't move the camera.
Code:

float mouse3DX = modelX(width/2-mouseX, height/2-mouseY, mouseZ);
float mouse3DY = modelY(width/2-mouseX, height/2-mouseY, mouseZ); 
float mouse3DZ = modelZ(width/2-mouseX, height/2-mouseY, mouseZ);

If I do move the camera, I get very strange results. I'm trying to use the mouse3DX and mouse3DY values.

Please take a look at the example I uploaded:
http://mkv25.net/applets/camera_and_3d_mouse/

Originally this was a test for importing OBJ files (stripped out) and then a camera control system (works very well).
To operate the red box/ship use the arrow keys.
To operate the camera, right click and drag to rotate, click middle button and drag vertically to zoom.

To centre the camera on the red box, press 'c'. To centre on the grid press 'r'. You can rotate the camera to a custom angle while centred on the red box.

My goal is to map the mouse coordinates to the onscreen grid. i.e./e.g. have the green box snap to the grid squares as the mouse rolls over them.

fry

Re: Translating (2D) mouse coordinates to 3D
Reply #1 - May 16^th, 2005, 10:49pm

the docs haven't been updated yet, but this is what's in the javadoc version:

/**
* Returns the model space x value for an x, y, z coordinate.
* <P>
* This will give you a coordinate after it has been transformed
* by translate(), rotate(), and camera(), but not yet transformed
* by the projection matrix. For instance, his can be useful for
* figuring out how points in 3D space relate to the edge
* coordinates of a shape.
*/

one difference from alpha is that the camera used to not affect modelX/Y/Z, but now it does.

Markavian Full Member Offline Posts: 174	Re: Translating (2D) mouse coordinates to 3D Reply #2 - May 18^th, 2005, 2:25am Thanks fry but I still don't quite understand what those functions are meant to do. I'm still a bit stuck figuring out how to solve my problem.

myT

Re: Translating (2D) mouse coordinates to 3D
Reply #3 - Jun 1^st, 2005, 2:45am

Seems like you want to get the point on the XY-plane, that the mouse is over when you click it.

Here is a function, that needs the same values as the camera() function and returns a vector with the intersection on the XY-plane. Be sure to use correct up values. Also if you want another plane than the XY-plane or if you changed the fov, you will need to adjust the code.

Code:

Vector getMouseGroundPlaneIntersection(float eyeX, float eyeY, float eyeZ,
			float centerX, float centerY,
			float centerZ, float upX, float upY, float upZ) {
  //generate the required vectors
  Vector eye = new Vector(eyeX, eyeY, eyeZ);
  Vector center = new Vector(centerX, centerY, centerZ);
  Vector look = (center.subtract(eye)).normalize();
  Vector up = new Vector(upX, upY, upZ).normalize();
  Vector left = up.crossProduct(look.normalize());

  //calculate the distance between the mouseplane and the eye
  float distanceEyeMousePlane = (height / 2) / tan(PI / 6);

  //calculate the vector, that points from the eye
  //to the clicked point, the mouse is on
  Vector mousePoint = look.multiply(distanceEyeMousePlane);
  mousePoint=mousePoint.add(left.multiply((float)((mouseX-width/2)*-1)));
  mousePoint=mousePoint.add(up.multiply((float)(mouseY-height/2)) );

  Vector intersection = new Vector(3);
  if (mousePoint.getZ() != 0) { //avoid zero division
    //calculate the value, the vector that points to the mouse
    //must be multiplied with to reach the XY-plane
    float multiplier = -eye.getZ() / mousePoint.getZ();
    //do not calculate intersections behind the camera
    if (multiplier > 0) { 
	//add the multiplied mouse point vector
	intersection = eye.add(mousePoint.multiply(multiplier));  
    }
  }
  return intersection;
}

The Vector class that I have used, can be found here http://www.millbridge.de/processing/Vector.pde

There might be easier ways, but I really like this school math

TomC

Re: Translating (2D) mouse coordinates to 3D
Reply #4 - Jun 1^st, 2005, 12:14pm

That looks like a really useful function (I've not tried it yet, but it's a handy thing to have around). One thing though - it's probably best not to use Vector as a class name because it's also the name of Java's standard resizable array class. Vec, Vec3d and Vec3D seem to be a popular choice for naming vector maths classes.

{sketches,links,blogs,hacks}

Ricard Full Member Offline Posts: 193	Re: Translating (2D) mouse coordinates to 3D Reply #5 - Jun 1^st, 2005, 1:09pm wouldn't it be a good idea to have a vector 3d lib or something like that (with vector and matrix handling in 3D space)? I see many people implement their own.

myT YaBB Newbies Offline Posts: 19	Re: Translating (2D) mouse coordinates to 3D Reply #6 - Jun 1^st, 2005, 1:44pm Yea, that's right TomC. Using only Processing makes no problems at all I think. Also I have not limited the vector to a 3D vector, so I will rename stuff to vec.

Markavian Full Member Offline Posts: 174	Re: Translating (2D) mouse coordinates to 3D Reply #7 - Jun 5^th, 2005, 6:39am Thank you for the input myT, I'm currently trying to rewrite my mouse-ui using the camera and the vector class. I think its a very nicely written class, but some documentation would be very handy, just to explain some of the functions. I'll post when I have what I wanted working.

Markavian

Re: Translating (2D) mouse coordinates to 3D
Reply #8 - Jun 5^th, 2005, 10:30am

MyT, I've got as far as I can perfecting how I want the camera the move but I can't get your plane-intersection code to work properly, and I'm not sure which values to change in your getMouseGroundPlaneIntersection() function to correct the problem.

Please see the files in action at:
http://mkv25.net/applets/cameraVector3DPlane/

The code should be quite easy to understand, a small scene is set up then using the right hand mouse button you drag horizontally to spin the scene around the Z axis and drag vertically to adjust the vertical viewing angle. You can also use the middle button + drag to zoom in and out.

I want the blue line to follow the mouse wherever it is on the grid. (Its so close, but not quite there).

Note: The mouse doesn't respond at all if you rotate up to vertical looking straight down at the Z axis.

myT

Re: Translating (2D) mouse coordinates to 3D
Reply #9 - Jun 5^th, 2005, 7:31pm

Hi Markavian,
As you probably already have realized and as I wrote in my last post here, is that your up vector must be set properly. Here is the method, that I normally use for updating the up vector:
Code:

void updateUp() {
  Vector helper = new Vector(3);
  //if z > length of xy
  if ( camLook.x[2]  >  (sqrt ( camLook.x[0] * camLook.x[0]  +  camLook.x[1] * camLook.x[1] ) ) ) {
    camUp.x[0] = 0;
    camUp.x[1] = 1;
    camUp.x[2] = 0;
    helper = new Vector(camLook);
    helper.x[1] = 0;
  }
  else {
    camUp.x[0] = 0;
    camUp.x[1] = 0;
    camUp.x[2] = 1;
    helper = new Vector(camLook);
    helper.x[2] = 0;
  }
  helper.normalizeMe();
  helper = (helper.crossProduct(camUp)).normalize();
  camUp  = (camLook.crossProduct(helper)).normalize();

// Calculate the roll if there is one
//if (roll != 0.0) {
//camUp = camUp.multiply(cos(roll));
// camUp = camUp.add(helper.multiply(sin(roll))); 
//}
}

I also updated your cameraVector3DPlane.pde. You can find the new one here www.millbridge.de/processing/cameraVector3DPlane.pde
The mousePoint calculation works now.

Markavian

Re: Translating (2D) mouse coordinates to 3D
Reply #10 - Jun 9^th, 2005, 9:34am

Thanks myT, I've made really good progress with my camera controlled scene now with a fully functioning mouse. I started building up a sample applet and want to make it into a fully interactive project - so I'll show you once I'm done.

I added a feature to my camera where I could pan the camera using the arrow keys. I had to use the same logic as calculating the position of the mouse on the ground plane. I started by centering the camera to the same position as the mouse offset and then replacing this with values set by the arrow keys. Anyhow, I rewrote your other function slightly to accept an xOffset and yOffset instead of being internally hardwired to mouseX, mouseY:

(It uses the new FVector class as well)

Code:

//originally created by david huebner (aka myT) (2005|05|01)
//edited by markavian (2005|06|07) - Two new method variables offsetX and offsetY added (would normally be mouseX and mouseY)
FVector getScreenGroundPlaneIntersection(float eyeX, float eyeY, float eyeZ,
   float centerX, float centerY, 
   float centerZ, float upX, float upY, float upZ,
   float offsetX, float offsetY) { 
  //generate the required vectors 
  FVector eye = new FVector(eyeX, eyeY, eyeZ); 
  FVector center = new FVector(centerX, centerY, centerZ); 
  FVector look = (center.subtract(eye)).normalize(); 
  FVector up = new FVector(upX, upY, upZ).normalize(); 
  FVector left = up.crossProduct(look.normalize()); 
 
  //calculate the distance between the mouseplane and the eye 
  float distanceEyeMousePlane = (height / 2) / tan(PI / 6); 
 
  //calculate the vector, that points from the eye 
  //to the clicked point, the mouse is on 
  FVector mousePoint = look.multiply(distanceEyeMousePlane); 
  mousePoint=mousePoint.add(left.multiply((float)((offsetX)*-1))); 
  mousePoint=mousePoint.add(up.multiply((float)(offsetY)) ); 
 
  FVector intersection = new FVector(3); 
  if (mousePoint.getZ() != 0) { //avoid zero division 
    //calculate the value, the vector that points to the mouse 
    //must be multiplied with to reach the XY-plane 
    float multiplier = -eye.getZ() / mousePoint.getZ(); 
    //do not calculate intersections behind the camera 
    if (multiplier > 0) {  
      //add the multiplied mouse point vector 
      intersection = eye.add(mousePoint.multiply(multiplier));   
    } 
  } 
  return intersection; 
}

BTW: The vertical up method doesn't quite work properly, the camera still goes a bit funny if rotated below the vertical axis. I prevented this from happening by locking the camera when it reaches the same level as the ground plane.