We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpOpenGL and 3D Libraries › a poor men guide to 3D mouse picking
Page Index Toggle Pages: 1
a poor men guide to 3D mouse picking? (Read 4441 times)
a poor men guide to 3D mouse picking?
May 12th, 2005, 4:05pm
 
Hello all

I'm shameless to write once again asking for some enlightment.

My frightened brain still thinks that "Matrix: Transformations" is a geek movie starring a bad actor wearing an overcoat; it is not getting the right way to work with screenX, screenY and screenZ - and neither with the modelX, Y,Z.

I've searched the alpha forum, but I could not find a really simple example that would help taking me off the ignorance ground.

let's write a simple sketch:

Code:

float rot; // rotator
void setup() {
size(500,500,P3D);
rot = 0;
}
void draw() {
background(255);
beginCamera();
resetMatrix();
camera (0,0,1000,0,0,0,0,1,0);
endCamera();
fill (200);
rotateY(rot);
box (200);
rot+=PI/512;
}



what would be the right way, in this very simple sketch, to add a mouse over action to the 3D model?


really grateful for any directions or hints regarding this...
Re: a poor men guide to 3D mouse picking?
Reply #1 - May 14th, 2005, 12:27am
 
here's a hint:
screenX(), screenY(), screenZ() in the complete reference.
Re: a poor men guide to 3D mouse picking?
Reply #2 - May 14th, 2005, 5:58am
 
Hi Reas

yep, I'm able to do a simple picking with screenX,Y,Z - by adding, for example, the lines
Code:

if (mouseX>screenX(-100,0,0)&&mouseX<screenX(100,0,0)&&mouseY>screenY(0,-100,0)&&mouseY<screenY(0,100,0)) { fill(255,0,0); }
else { fill (200); }


to replace the fill on the program above;

but this is not the good way - it evaluates a "center rectangular slice" of my geometry - when the geometry rotates, the slice remains the same. there's no Z info to it.

Besides, when I rotate the geometry around the screen, the screenX coordinates do not stick to the translated geometry very well - the mouse over action gets even worse aligned.

I guess my real question is: is there a general way to build an invisible "bounding box" around my geometry, and evaluate the bounding box for mouse picking?

if anyone do not have a code for this but can point me out to any other relevant example, I'll be already happy.


thanks!
Re: a poor men guide to 3D mouse picking?
Reply #3 - Oct 30th, 2005, 6:27pm
 
I've just been experimenting with a couple of implementations of 3D picking for simple scenes composed of triangles -

http://www.tom-carden.co.uk/p5/picking_with_buffer/applet/

This one is really simple, and can be pixel-accurate (but isn't in this case because I reduced the picking resolution for speed reasons).  It requires you to draw your scene twice... once as you normally would, and once to an offscreen buffer using the triangle index as the fill colour.  You then look at the colour of the object under mouseX/Y in the buffer to get the index of the picked object.  It's a bit slow, but it works accurately even in this random-triangle scene.

http://www.tom-carden.co.uk/p5/picking_with_projection/applet/

This one projects all the triangles to screen space, depth sorts them, and checks if any of the projected triangles contains the mouse point.  It's not 100% accurate, but for scenes without intersecting triangles it will be fine, and it's faster than the buffer-based method.

A "picking_with_rays" applet is planned, but I'm not going to get to it until next week.  The idea is to make an imaginary line between the mouseX/Y on the screen (near clipping plane) and the equivalent point on the far clipping plane.  You then test for intersections with this line and your scene.  That check involves first testing if the ray intersects the plane containing the triangle, and then projecting the triangle and intersection point to one of the x/y/z planes and testing if the point is inside the triangle - I think it will actually be slightly slower than the projected triangles method above, but it's more accurate and it could be speeded up by using a spatial partition or testing object bounding box intersection first.

Hope that helps.
Re: a poor men guide to 3D mouse picking?
Reply #4 - Oct 30th, 2005, 7:22pm
 
Very nice demos!

This tutorial may be useful to explain the various techniques for picking, line intersection, and even arbitrary "where what is the 3d coordinate of the mouse on a 3d polygon."  In fact, TomC, you might be able to improve your second demo with that method, since it could handle intersecting polygons.

Marcello
Re: a poor men guide to 3D mouse picking?
Reply #5 - Oct 30th, 2005, 8:20pm
 
Yep, that's a good tutorial, thanks!

A helpful reference for me finishing off the picking with rays example Smiley

Also, marquee selection is a nice touch, I'll have to try that one.

Re: a poor men guide to 3D mouse picking?
Reply #6 - Jan 6th, 2006, 1:20pm
 
Has anybody managed yet to do proper OpenGL-style object picking using GL_SELECT mode, a name stack and a pick matrix?
(see http://www.rush3d.com/reference/opengl-redbook-1.1/chapter12.html for what I mean)

I would like to implement a basic picking routine, but fear the problems of just grabbing the GL object as mentioned in the FAQ. It seems to make more sense to dive into PGraphics3 and do it right there.
I've seen that picking is coming up on the roadmap (http://dev.processing.org/source/index.cgi/trunk/processing/future.txt?view=markup). Has there been any success yet? Or some promising plans or API definitions that I could integrate with?
Re: a poor men guide to 3D mouse picking?
Reply #7 - Jan 6th, 2006, 10:11pm
 
I've heard that the select method isn't used much even professionally.  I personally find it much easier (and more intuitive) to simply render the whole scene in a false-color mode in a backbuffer, then use the color as a form of "id" into what was actually selected.

This can be optimized a couple of ways.  If the scene doesn't change significantly, you can cache the whole false-color image.  Otherwise, you can simply crop and render the scene so that it's clipped immediately (would use something like gluPickMatrix as with the other method) around the coordinate you are trying to check so hidden surface removal can be performed on 99% of the image.
Re: a poor men guide to 3D mouse picking?
Reply #8 - Jan 10th, 2006, 8:59pm
 
Interesting! But isn't that pretty much the same as the standard OpenGL way? In both cases, the scene is rendered into a buffer; in your case identification is via a special color, in the other it's via a special name. And then, doing it the OpenGL way you get all the objects under the picked coordinate vs. just the topmost in the color method.

However, your method would also be easily applicable to the standard 2D processing mode. I'll try to find out how to get access to a back buffer there.
Re: a poor men guide to 3D mouse picking?
Reply #9 - Jan 11th, 2006, 10:56pm
 
Alright, I implemented a simple picking method for OpenGL mode using the false color mode as described above.

public int pick(int _x, int _y) {
 gl = ((PGraphicsGL)g).gl;
 gl.glDrawBuffer(GL.GL_FRONT);
 draw();
 gl.glReadBuffer(GL.GL_FRONT);
 byte[] pickedPixel = new byte[4];
 gl.glReadPixels(_x, height-_y, 1, 1,
   GL.GL_RGBA, GL.GL_UNSIGNED_BYTE, pickedPixel);
 gl.glDrawBuffer(GL.GL_BACK);
 return new Color(pickedPixel[0],pickedPixel[1],pickedPixel[2]).getRGB();
}

I tried it with several buffers, it only worked well with the front buffer. You can call this method from mousePressed() etc., anywhere, where you're officially allowed to draw.
Page Index Toggle Pages: 1