How to project mouse coordinates onto a tilted 3D plane -- restricted solution.

bob97086

I posted this question in September: https://forum.processing.org/two/discussion/24140/how-to-project-mouse-coordinates-onto-tilted-3d-plane#latest. I've worked out an answer to the simplified case where a known 3D surface is rotated only about the x-axis.

The motivation here is to allow the user to interact with a simple 3D surface. The underlying structure for my purposes is a "heightfield" of floating point values on a discrete grid z[x][y]. This could describe one frame of a dynamic water wave or static terrain such as a golf putting green.

In the dynamic case, at each cycle of the draw() loop, effects on z[x][y] are calculated and made visible by "knitting" a surface in a beginShape()... endShape() block. The sequence is (1) the user clicks on the sketch noted by coordinates (mouseX, mouseY), (2) a transformation finds the corresponding point on heightfield z[x][y] which (3) triggers an algorithmic change to z followed by (4) an update to the visible surface. Here, I'm only concerned with steps 1 and 2.

The base of surface S is w units wide corresponding to the 'x' in z[x][y] and h units wide ('y'). It is, in effect, centered on the sketch coordinate system width-by-height by two translations: translate(width/2, height/2) and translate(-w/2, -h/2). These are inside a pushMatrix()...popMatrix() pair along with the surface rotation and the shape construction.

In the diagrams below, the X and Y axes are parallel to the sketch window and intersect at point O on the Z axis. (The line OZ corresponds to the 3D perspective viewpoint "camZ".) In this figure, the rectangular green surface is marked with yellow on the horizontal (X) axis and vertical (Y) axis and rotated +30 degrees. So the upper edge appears narrower than the bottom edge due to perspective.

Now imagine a line drawn from the user's viewpoint at Z to some point D on the surface S which corresponds to a point on the array z[x][y]. Here the view is an XYZ frame which we can relate to the sketch coordinates. The ray from Z to D (dashed red line) intersects the XY plane at red point B which is Ay units above the XZ plane and Ax units from the YZ plane (dashed green lines). Point C is the projection of surface point D onto the YZ plane and C', D' are downward projections onto the XZ plane. (Sorry, the surface is not explicitly drawn -- the figure is already crowded.)

In the side view below, the surface (in red) is rotated theta degrees around the x-axis. Projection of the ZD line in 3-space onto the YZ plane is the line ZC which crosses the y-axis at Ay. Angle CZC' is marked as phi at the viewpoint Z.

The line OC is the distance on the surface (projected on YZ) corresponding to the mouse click marked at Ay. So one can compute the distance OC using the known mouseY position, the value Ay, the OZ distance ("camZ") and the given angle theta. Also helpful is that triangles ZAyO and ZCC' are similar triangles. Also note that the angle at C is equal to theta. Distance OAy is half the sketch height minus the mouseY value.

Denoting the ratios OAy/OZ as tan(phi), OC'/CC' as tan(theta) and CC'/OC as cos(theta), here is the distance from O to C in the YZ plane:

OC = OAy/[cos(theta) * {1 - tan(theta)*tan(phi)}].

If there is no rotation, theta = 0, cos(theta) = 1 and tan(theta) = 0. Then, OC = OAy which is what you'd expect for zero rotation. (There could also be a scale factor between the computed z[x][y] and the displayed surface S(x,y).)

Knowing OC on the surface S, one can find the row index y in the heightfield -- again with the possible application of a scale factor. We also need the column index x to apply the user-driven action. From a top view, the ray ZD passes through B a distance Ax from the YZ plane. In the figure below, the positive theta x-rotation puts the lower part of the surface below the XZ plane so it is given a dashed outline. The point of contact D projects onto the YZ plane at C (as above) and the triangle ZDC makes an angle rho at Z. (Keep in mind that edge DC is elevated above the XZ plane.)

Note that DC/ZC = tan(rho). Also note that line ZC can be related to quantities derived from the triangle ZCC' shown in the side view. I'm going to do some hand-waving here to finish the relations. There is another pair of similar triangles with a common point at Z. One has the edge DC. Another edge has Ax as one point and its projection on the y-axis as the other point (not labeled). This point also relates to the side view with a projection down to the XZ plane at O. The edge from the Y-axis to Ay (call it OAy) is half the sketch width minus the mouseX value.

Skipping several steps, we have a relation for distance DC which is directly related to the column index in the heightfield array:

DC = OC * cos(theta)*OAx / OAy.

Again, if theta is zero, the cosine is 1 and from above OC = OAy so DC = OAy -- what you'd expect for no x-rotation. An example of a mouse click at the black cross and a slant view of a disturbance whose base, allowing for perspective view, is directly under the mark:

I hope this helps someone and I'd be grateful anyone who could extend this to combined rotation about the y-axis.