How to pick an object in 3D?

tessa

Hi all,

I have a set of PShapes in a 3D environment, and I want to be able to select them. I constructed them within a class, by using a ArrayList<ArrayList>, which is based on an imported geometry file. This is what it looks like:

[ PICTURE1 - see my comment, beneath ]

I am using Peasycam, and I'm working in Processing 2.0.3.

I feel like I tried literally everything, so I think I just need someone to think with me for a sec. Let me try to summarise what I've tried:

1. Toxiclibs

I found this example (Example1: Creating, Picking and Dragging shapes), on: http://amnonp5.wordpress.com/2011/04/23/working-with-toxiclibs/ It defines the location of the mouse as a point, and then it uses the function polygon.containsPoint(mouse) to see if the mouse is at the same location as the polygon. I tried to transform my PShapes into toxiclibs' Polygon2Ds, but I found out that a Polygon2D cannot be build with 3D points or vectors.

2. ScreenX, Y (and Z?)

To solve the problem I had with option 1, I tried to continuously project my PVectors (on which the PShapes are based), on the 2D screen. This, to eventually try to make Polygon2Ds based on them, without any colour or stroke, so that I would be able to perform option 1. However, I don't think I understand ScreenX, Y and Z. (Especially the last one: why, if you project a 3D point on a 2D HUD display, would you need a Z-coordinate for this projected point?). Anyway, I thought it would work like this:

          ArrayList <Polygon2D> polygons = new ArrayList <Polygon2D> ();

          for (ArrayList<PVector> points : polylinequads) {
              ArrayList<Vec2D> points2D = new ArrayList<Vec2D>();
              for(int i=0; i<points.size();i++){
                  PVector point = points.get(i);
                  float x = screenX(point.x,point.y,point.z);
                  float y = screenY(point.x,point.y,point.z);
                  //this is commented because I don't get the point of it (that's a pun)
                  //float z = screenZ(point.x,point.y,point.z);
                  Vec2D point2D = new Vec2D(x,y);
                  points2D.add(point2D);
              }
              fill(0);
              //Check if it works by displaying the points
              for (Vec2D p : points2D){
                stroke(0);
                point(p.x,p.y,0);
              }
              polygons.add(new Polygon2D(points2D));
              points2D.clear();
          }    

But this is what it gives, the points are somewhere far in a corner and rotating along with the rest of the model:

[ PICTURE2 - see my comment, beneath ]

3. Picking library

I found a library on the Processing website, called Picking. And it sounds perfect! However, it doesn't work / it crashes if I try an example. On its website it says it is checked for Processing version 2.0.2, and I'm using 2.0.3.

4. Tips anyone?

tessa

I'm so sorry, tried my best on the layout, but pictures don't work.

Here again:

PICTURE 1:

PICTURE2:

amnon

There are two main picking techniques:

1. Color picking draws the geometry in an offscreen buffer, giving each object a unique color. The mouse coordinates are then tested against the color buffer to return the unique color aka object that is under the mouse.

2. Ray picking sends a ray from the mouse coordinates (projected into world space) into the world and tests the intersections with the scene's geometry.

The first solution is probably easiest to implement and is used by the picking library and Hemesh's HET_Selector. However it has some downsides, such as the cost of an additional render pass, making picking depend on rendering, anti-aliasing inaccuracy and the limitation of only being able to return the closest object.

The second solution is probably the more common method. It is probably a bit harder to implement and is computationally more costly. This can be countered with lowering the amount of intersection checks for example by culling and first checking against bounding boxes instead of actual geometry. This method is also more accurate, does not require a (second) render pass and is able to find all the objects that intersect with the ray (not only the closest).

For a quick fix you can use Hemesh for color picking. See this example for a use of the HET_Selector class: http://amnonp5.wordpress.com/2012/03/14/hemesh-tutorial-color-dissolve/

Libraries such as Hemesh and Toxiclibs have useful classes (for 3d rays and intersection checks) that can help you implement ray picking. See this link for more information: http://schabby.de/picking-opengl-ray-tracing/

tessa

Thank you so much! The tutorial on ray picking has been a lot of help this morning :)

Have you ever used it before yourself? Cause in the end of the tutorial I am not sure what to do with the intersectionWithXyPlane() - fuction.

Do you have a code for which you used this tutorial? Can't find working examples anywhere..

amnon

I haven't. But please feel free to post the code once you get it working.

Just focus on implementing the idea, the code snippets aren't that solid it seems.

Mondonerd

Hello friends,

I was able to use the get(x,y) method in order to retrive the color of an object rendered on offscreen PGraphics and comparing it to known color indexes as documented on the old wiki... this seems to be impossible in current implementation (Processing 0233) of PGraphicsOpenGL...

Is there a different solution now?

Thank you in advance for any clue! :ar!

PS: If I try to use get() or loadPixels() something odd happens and frames are dropped... :(|)