Quite simply, no. The problem here is that when you compress a 3D scene to a 2D image, and then pick a point on that image, what you are actually selecting is a LINE in the 3D scene, not a point!

thanks a lot!

FYI:
http://wiki.processing.org/w/Picking_with_a_color_buffer

Since you are just dealing with points there is a simpler way to solve your problem by using screenX and screenY. These calculate the screen position of a 3D point as it will appear in the display so for your first point you have

float x, y, d;

x = screenX(pts[0].x, pts[0].y, pts[0].z);
y = screenY(pts[0].x, pts[0].y, pts[0].z);

The square of the distance of the point from the current mouse position would be

d = (mouseX - x) * (mouseX - x) + (mouseY - y)  * (mouseY - y) ;

The actual distance is sqrt(d) but when comparing distances it is usual to compare the square of the distances to avoid a costly square root operation.

now
if(d < 4){
      // point is within 2 pixels of the mouse position
}

Personally I have never used screenX and screenY I am going off the Processing reference but it should even with PeasyCam.

cool! thanks!

will explore ur suggestion later ...

I was emulating the method shown on: http://wiki.processing.org/w/Picking_with_a_color_buffer , but using points this time.

However, I can't get the point object ID correctly. When i pick a point, I always get -1 as object ID which is the background's ID... Can anybody kindly help to point out what I missed here? thanks!

BTW, may I ask how to export the image using different gray colors to represent different objects in a given frame?

1. import processing.opengl.*;
   import peasy.*;
   PeasyCam cam;
   Pt[] pts;
   PGraphics buffer;
   //____________________________________________________
   void setup(){
     size(500,500,OPENGL);
     smooth(8);
     cam = new PeasyCam(this,500);
    
     // buffer is created using applet dimensions
     buffer = createGraphics(width, height, P3D);
    
     pts = new Pt[3];
     for (int i=0; i<pts.length; i++){
       pts[i] = new Pt(i, random(-100,100), random(-100,100), random(-100,100), random(20,30));
     }
   } 
   //________________________________________________________________________
   void draw() {
     background(255);
   
     for (int i=0; i<pts.length; i++){
       pts[i].displayPt(this.g);
     }
   }
   //_________________________________________________________________
   void mouseClicked(){
     // draw the scene in the buffer
     buffer.beginDraw();
     buffer.background(getColor(-1)); // since background is not an object, its id is -1
     buffer.noStroke();
     for (int i=0; i<pts.length; i++) {
       pts[i].drawBuffer(buffer);
     } 
     buffer.endDraw();
    
     // get the pixel color under the mouse
     color pick = buffer.get(mouseX, mouseY);
     int id = getId(pick); // get object id
     println("object ID : " + id);
     if (id >= 0) { // if id > 0 (background id = -1)
       pts[id].changeColor(); // change pt color
     }
   }
   //___________________________________________________________________
   // id 0 gives color -2, etc.
   color getColor(int id){
     return -(id + 2);
   }
   //_______________________________________________________________________
   // color -2 gives 0, etc.
   int getId(color pt_color){
     return -(pt_color + 2);
   }
2. 
   
3. class Pt{
     //______________________________________________________________ 
     // variables
     int id; // id
     float x, y, z;  // position
     float pt_size;  // pt size
     color pt_color;  // pt color (in scene, not buffer)
     //______________________________________________________________ 
     // constructor
     public Pt(int id_, float x_, float y_, float z_, float pt_s_)
     {
       this.id = id_;
       this.x = x_; this.y = y_; this.z = z_;
       this.pt_size = pt_s_;
       pt_color = color(random(255), random(255), random(255));
   //    println("r: " + red(pt_color) + " g: " + green(pt_color) + " b: " + blue(pt_color));
     }
     //______________________________________________________________
     public void changeColor() {
       int r = (int)red(pt_color);
       int g = (int)green(pt_color);
       int b = (int)blue(pt_color);
       pt_color = color(r, 255-g, b);
     }
     // display the pt on the screen_________________________________________
     public void displayPt(PGraphics ecran) {
       ecran.stroke(pt_color);
       drawPt(ecran);
     }
     // draw the pt in the buffer______________________________________________ 
     public void drawBuffer(PGraphics buffer) {
       color idColor = getColor(id);
       buffer.stroke(idColor);
       drawPt(buffer);
     }
     //______________________________________________________________ 
     private void drawPt(PGraphics g) {
   //    g.pushMatrix();
       g.strokeWeight(pt_size);
       g.point(x,y,z);
   //    g.popMatrix();
     }
   }
   
   

The problem is caused because the display and the buffer have different transformation matrices, also PeasyCam changes the displays transformation matrix!

I have made 3 changes to your code which solves the problem.

1. In line 5 - I declare the buffer to be of type PGraphicsOpenGL
2. In line 12 - I cast the graphics object to type PGraphicsOpenGL
3. I have added line 30 [buffer.setMatrix(g.getMatrix());] which copies the current transformation matrix to the buffer

I have tested this in 2.0b6 if you are using 1.5.1 then use PGraphics3D instead of PGraphicsOpenGL in steps 1 & 2 and delete line 9 since smooth() is not supported by P3D.

HTH

1. import processing.opengl.*;
2. import peasy.*;
3. PeasyCam cam;
4. Pt[] pts;
5. PGraphicsOpenGL buffer;
6. //____________________________________________________
7. void setup() {
8.   size(500, 500, OPENGL);
9.   smooth(8);
10.   cam = new PeasyCam(this, 500);
11.   // buffer is created using applet dimensions
12.   buffer = (PGraphicsOpenGL)createGraphics(width, height, P3D);
13.   pts = new Pt[3];
14.   for (int i=0; i<pts.length; i++) {
15.     pts[i] = new Pt(i, random(-100, 100), random(-100, 100), random(-100, 100), random(20, 30));
16.   }
17. }
18. //________________________________________________________________________
19. void draw() {
20.   background(255);
21.   for (int i=0; i<pts.length; i++) {
22.     pts[i].displayPt(this.g);
23.   }
24. }
25. //_________________________________________________________________
26. void mouseClicked() {
27.   // draw the scene in the buffer
28.   buffer.beginDraw();
29.   buffer.background(getColor(-1)); // since background is not an object, its id is -1
30.   buffer.setMatrix(g.getMatrix());
31.   buffer.noStroke();
32.   for (int i=0; i<pts.length; i++) {
33.     pts[i].drawBuffer(buffer);
34.   }
35.   buffer.endDraw();
36.   // get the pixel color under the mouse
37.   color pick = buffer.get(mouseX, mouseY);
38.   int id = getId(pick); // get object id
39.   println("object ID : " + id);
40.   if (id >= 0) { // if id > 0 (background id = -1)
41.     pts[id].changeColor(); // change pt color
42.   }
43. }
44. //___________________________________________________________________
45. // id 0 gives color -2, etc.
46. color getColor(int id) {
47.   return -(id + 2);
48. }
49. //_______________________________________________________________________
50. // color -2 gives 0, etc.
51. int getId(color pt_color) {
52.   return -(pt_color + 2);
53. }
54. class Pt {
55.   //______________________________________________________________
56.   // variables
57.   int id; // id
58.   float x, y, z;  // position
59.   float pt_size;  // pt size
60.   color pt_color;  // pt color (in scene, not buffer)
61.   //______________________________________________________________
62.   // constructor
63.   public Pt(int id_, float x_, float y_, float z_, float pt_s_)
64.   {
65.     this.id = id_;
66.     this.x = x_;
67.     this.y = y_;
68.     this.z = z_;
69.     this.pt_size = pt_s_;
70.     pt_color = color(random(255), random(255), random(255));
71.     //    println("r: " + red(pt_color) + " g: " + green(pt_color) + " b: " + blue(pt_color));
72.   }
73.   //______________________________________________________________
74.   public void changeColor() {
75.     int r = (int)red(pt_color);
76.     int g = (int)green(pt_color);
77.     int b = (int)blue(pt_color);
78.     pt_color = color(r, 255-g, b);
79.   }
80.   // display the pt on the screen_________________________________________
81.   public void displayPt(PGraphics ecran) {
82.     ecran.stroke(pt_color);
83.     drawPt(ecran);
84.   }
85.   // draw the pt in the buffer______________________________________________
86.   public void drawBuffer(PGraphics buffer) {
87.     color idColor = getColor(id);
88.     buffer.stroke(idColor);
89.     drawPt(buffer);
90.   }
91.   //______________________________________________________________
92.   private void drawPt(PGraphics g) {
93.     //    g.pushMatrix();
94.     g.strokeWeight(pt_size);
95.     g.point(x, y, z);
96.     //    g.popMatrix();
97.   }
98. }

thank you very much, quarks!

I really appreciate your detailed advices! 

will try to read more on PGraphics and matrics related reference.

Hi, everybody, I further modified the code above and added GUI elements.

What I want to do are:
1. show the ID number of each point at the centre of each point
2. show the ID number in the "text area" GUI when clicking a point

The problems of the code below are:
1. the text for the ID number was shown somewhere away from the points in the 3D space
2. when a point is clicked, the ID of the point is no longer returned (or a "-1" ID num for background is always returned)
3. only when the gui() function inside the draw() block is disabled, can the ID of point be returned correctly

I assume it might be related to transform matrix due to the use of PeasyCam and ControlP5 library. Or the wrong use of screenX function...  But I haven't figured out the reason yet.

So, I'd appreciate if anybody can kindly advise on the source of problem.

Thank you in advance!

1. /*----------------------------------------------------------------------------------------
   orginial codes:
   http://wiki.processing.org/w/Picking_with_a_color_buffer
   
   Thanks for quarks' additional advices!
   https://forum.processing.org/topic/how-to-display-xyz-coordinates-of-a-point-in-3d-scene
   
   -----------------------------------------------------------------------------------------*/
   
   import processing.opengl.*;
   import controlP5.*; // GUI library
   ControlP5 cp5;
   Textarea myTextarea;
   import peasy.*; // camera library
   PeasyCam cam;
   PGraphicsOpenGL buffer;
   PMatrix3D currCameraMatrix;
   PGraphics3D g3;
   PFont myFont;
   Pt[] pts;
   
   //_____________________________________________________________________________________
   void setup(){
     size(1000,500,OPENGL);
     smooth(8);
     cam = new PeasyCam(this,500);
     g3 = (PGraphics3D)g;
    
     myFont = createFont("Arial",10);
     textFont(myFont);
    
     // buffer is created using applet dimensions
     buffer = (PGraphicsOpenGL)createGraphics(width, height, P3D);
    
     // initiate random pts
     pts = new Pt[10];
     for (int i=0; i<pts.length; i++){
       pts[i] = new Pt(i, // pt id
                       random(-150,150), // pt x
                       random(-150,150), // pt y
                       random(-150,150), // pt z
                       random(20,50)     // pt size
                      );  
     }
   
     cp5 = new ControlP5(this); // initiate GUI objects
     cp5.setAutoDraw(false);  // fix controlP5 GUI
    
     // setup a text area GUI
     myTextarea = cp5.addTextarea("txt")
                     .setPosition(800,50)
                     .setSize(150,100)
                     .setFont(createFont("arial",12))
                     .setLineHeight(14)
                     .setColor(color(128))
                     .setColorBackground(color(255,100))
                     .setColorForeground(color(255,100));
                     ;
                    
     myTextarea.setText("ID : " + " ");
   
   } 
   
   //_____________________________________________________________________________________
   void draw() {
     background(0);
   
     for (int i=0; i<pts.length; i++){
       pts[i].displayPt(this.g);
     } 
    
     gui();
   }
   
   //_____________________________________________________________________________________
   void mouseClicked(){
     // draw the scene in the buffer
     buffer.beginDraw();
     buffer.background(getColor(-1)); // since background is not an object, its id is -1
     buffer.noStroke();
     buffer.setMatrix(g.getMatrix()); 
     for (int i=0; i<pts.length; i++) {
       pts[i].drawBuffer(buffer);
     } 
     buffer.endDraw();
    
     color pick = buffer.get(mouseX, mouseY); // get the pixel color under the mouse
     int id = getId(pick); // get object id
     println("object ID : " + id);
     myTextarea.setText("ID : " + id);
    
     if (id >= 0) { // if id > 0 (background id = -1)
       pts[id].changeColor(); // change pt color
     }
   }
   
   //_____________________________________________________________________________________
   // id 0 gives color -2, etc.
   color getColor(int id){
     return -(id + 2);
   }
   
   //_____________________________________________________________________________________
   // color -2 gives 0, etc.
   int getId(color pt_color){
     return -(pt_color + 2);
   }
   
   //_____________________________________________________________________________________
   // to make controlP5 GUI always on top of everything else
   void gui(){
     hint(DISABLE_DEPTH_TEST);
     currCameraMatrix = new PMatrix3D(g3.camera);
     camera();
     cp5.draw();
     g3.camera = currCameraMatrix; 
     hint(ENABLE_DEPTH_TEST);
   }
2. 
   
3. class Pt{
     //______________________________________________________________ 
     // variables
     int id; // id
     float x, y, z;  // position
     float pt_size;  // pt size
     color pt_color;  // pt color (in scene, not buffer)
    
     //______________________________________________________________ 
     // constructor
     public Pt(int id_, float x_, float y_, float z_, float pt_s_)
     {
       this.id = id_;
       this.x = x_; this.y = y_; this.z = z_;
       this.pt_size = pt_s_;
       pt_color = color(random(255), random(255), random(255));
     }
    
     //______________________________________________________________
     public void changeColor() {
       int r = (int)red(pt_color);
       int g = (int)green(pt_color);
       int b = (int)blue(pt_color);
       pt_color = color(255-r, 255-g, 255-b);
     }
    
     // display the pt on the screen_________________________________
     public void displayPt(PGraphics ecran) {
       ecran.stroke(pt_color);
       drawPt(ecran);
     }
    
     // draw pt on buffer_____________________________________
     public void drawBuffer(PGraphics buffer) {
       color idColor = getColor(id);
       buffer.stroke(idColor);
       drawPt(buffer);
     }
    
     //_______________________________________________________________ 
     private void drawPt(PGraphics g) {
       g.strokeWeight(pt_size);
       g.point(x,y,z);
       g.text(id, screenX(x,y,z), screenY(x,y,z));
     }
   }
   
   

another thing I want to ask is how to save the image of the buffer showing objects in different gray colors ?

Thanks!

... ok, the text position problem is solved: text can be shown in 3d space:  text("hello", x, y, z)

but I don't know why the text is shown as a square surface with black background color ...

is there a way to make text always displayed as facing the user?

I assume it might be related to transform matrix due to the use of PeasyCam and ControlP5 library. Or the wrong use of screenX function...  But I haven't figured out the reason yet.

Your assumption is correct so you have figured it out (ControlP5 cahnges the matrix) but not the solution. Try modifying the gui to match the one below. The pushMatrix and popMatrix are away of saving and restoring the current transformation matrix and are always used in pairs. It worked for me.

1. //___________________________________________________
2. // to make controlP5 GUI always on top of everything else
3. void gui(){
4.       pushMatrix(); // Save the current matrix
   
5.    hint(DISABLE_DEPTH_TEST);
6.    camera();
7.    cp5.draw();
8.    // Other 2D code can go here
   
9.    hint(ENABLE_DEPTH_TEST);
10.       popMatrix(); // restore the old matrix
    
11. }

The changes made to gui will work in all cases you want to mix 2D/3D with ControlP5 but since you are using PeasyCam there is a way to avoid the gui method altogether - change the draw method to

1. //__________________________________________________________
2. void draw() {
3.   background(0);
4.   for (int i=0; i<pts.length; i++) {
5.     pts[i].displayPt(this.g);
6.   }
7.   cam.beginHUD();
8.      // put all your 2D stuff here
   
9.   cp5.draw();
10.   cam.endHUD();
11. }

Hi, quarks, thank you very much for your invaluable advices!

pls see the updated codes below.

One new problem showed up:
when I added the perspective related code in the setup() block, the point very close to the viewer will not get their id and color picked correctly (e.g. the point showing at the top left corner of the canvas, see image below). Only when the chunk of code related to defining near and far clip, will the point be picked correctly ...

So I don't know why view clip variable messed up the point picking method used here ...

1. /*----------------------------------------------------------------------------------------
   orginial codes:
   http://wiki.processing.org/w/Picking_with_a_color_buffer
   
   Many thanks to quarks' detailed advices!
   https://forum.processing.org/topic/how-to-display-xyz-coordinates-of-a-point-in-3d-scene
   -----------------------------------------------------------------------------------------*/
   
   import processing.opengl.*;
   import controlP5.*; // GUI library
   ControlP5 cp5; // declare ControlP5 GUI object
   Textarea myTextarea; // declare "Textarea" GUI object
   
   import peasy.*; // camera library
   PeasyCam cam; // declare camera object
   
   PGraphicsOpenGL buffer; // declare image buffer object
   
   PMatrix3D currCameraMatrix; // related to fixing GUI
   PGraphics3D g3; // declare PGraphics3D object
   
   PFont myFont; // declare font object
   
   Pt[] pts; // declare pt array
   
   //_____________________________________________________________________________________
   void setup(){
     size(1000,500,OPENGL);
     smooth(8);
   
     float fov = PI/3.0;  // field of view
     float nearClip = 10;
     float farClip = 1000;
     float aspect = float(width)/float(height);
     perspective(fov, aspect, nearClip, farClip);
    
     cam = new PeasyCam(this,500);
     g3 = (PGraphics3D)g;
    
     myFont = createFont("Arial",10);
     textFont(myFont);
    
     // buffer is created using applet dimensions
     // cast the graphics object "buffer" to type PGraphicsOpenGL
     buffer = (PGraphicsOpenGL)createGraphics(width, height, P3D);
    
     // initiate random pts
     pts = new Pt[10];
     for (int i=0; i<pts.length; i++){
       pts[i] = new Pt(i, // pt id
                       random(-150,150), // pt x
                       random(-150,150), // pt y
                       random(-150,150), // pt z
                       random(20,50)     // pt size
                      );  
     }
   
     cp5 = new ControlP5(this); // initiate GUI object
     cp5.setAutoDraw(false);  // related to fixing controlP5 GUI
    
     // setup a text area GUI
     myTextarea = cp5.addTextarea("txt")
                     .setPosition(780,10)
                     .setSize(200,300)
                     .setFont(createFont("arial",12))
                     .setLineHeight(14)
                     .setColor(color(200))
                     .setColorBackground(color(255,100))
                     .setColorForeground(color(255,100));
                     ;             
     myTextarea.setText("WELCOME TO PT VISUALIZER !!!");
   
   } 
   
   //_____________________________________________________________________________________
   void draw() {
     background(0);
   
     for (int i=0; i<pts.length; i++){
       pts[i].displayPt(this.g);
       /* All Processing applications contain a PGraphics variable called g,
       and that variable can be cast to the PGraphicsOpenGL variable type.
       ---"Programming Interactivity", pp.491 */
     } 
    
     gui();
   
     // if gui() is disabled, use the following
   //  cam.beginHUD();
   //  // put all your 2D stuff here
   //  cp5.draw();
   //  cam.endHUD();
   }
   
   //_____________________________________________________________________________________
   void mouseClicked(){
     // draw the scene in the buffer
     buffer.beginDraw();
     buffer.background(getColor(-1)); // since background is not an object, its id is -1
     buffer.noStroke();
     buffer.setMatrix(g.getMatrix()); 
     for (int i=0; i<pts.length; i++) {
       pts[i].drawBuffer(buffer);
     } 
     buffer.endDraw();
    
     color pick = buffer.get(mouseX, mouseY); // get the pixel color under the mouse
     int id = getId(pick); // get object id
     println("object ID : " + id);
    
     if (id==-1) {
       myTextarea.setText("background");    
     } else {
       myTextarea.setText("Point ID : " + id + "\n\n"
                          + "pt x : " + pts[id].x + "\n"
                          + "pt y : " + pts[id].y + "\n"
                          + "pt z : " + pts[id].z + "\n\n"
                          + "pt size : " + pts[id].pt_size + "\n"
                          + "pt color R : " + (int)red(pts[id].pt_color) + "\n"
                          + "pt color G : " + (int)green(pts[id].pt_color) + "\n"
                          + "pt color B : " + (int)blue(pts[id].pt_color) + "\n\n"
                         );
     }
    
     if (id >= 0) { // if id > 0 (background id = -1)
       pts[id].changeColor(); // change pt color
     }
   }
   
   //_____________________________________________________________________________________
   // id 0 gives color -2, etc.
   color getColor(int id){
     return -(id + 2);
   }
   
   //_____________________________________________________________________________________
   // color -2 gives 0, etc.
   int getId(color pt_color){
     return -(pt_color + 2);
   }
   
   //_____________________________________________________________________________________
   // to make controlP5 GUI always on top of everything else
   void gui(){
     pushMatrix(); // save the current matrix
     hint(DISABLE_DEPTH_TEST);
     currCameraMatrix = new PMatrix3D(g3.camera);
     camera();
     cp5.draw();
     g3.camera = currCameraMatrix; 
     hint(ENABLE_DEPTH_TEST);
     popMatrix(); // restore the old matrix
   }
2. 
   
3. class Pt{
     //______________________________________________________________ 
     // variables
     int id; // id
     float x, y, z;  // position
     float pt_size;  // pt size
     color pt_color;  // pt color (in scene, not buffer)
    
     //______________________________________________________________ 
     // constructor
     public Pt(int id_, float x_, float y_, float z_, float pt_s_)
     {
       this.id = id_;
       this.x = x_; this.y = y_; this.z = z_;
       this.pt_size = pt_s_;
       pt_color = color(random(255), random(255), random(255));
     }
    
     //______________________________________________________________
     public void changeColor() {
       int r = (int)red(pt_color);
       int g = (int)green(pt_color);
       int b = (int)blue(pt_color);
       pt_color = color(255-r, 255-g, 255-b);
     }
   
     // display the pt on the screen_________________________________
     public void displayPt(PGraphics ecran) {
       ecran.stroke(pt_color);
       drawPt(ecran);
     }
    
     // draw pt on buffer_____________________________________
     public void drawBuffer(PGraphics buffer) {
       color idColor = getColor(id);
       buffer.stroke(idColor);
       drawPt(buffer);
     }
    
     //_______________________________________________________________ 
     private void drawPt(PGraphics g) {
       g.strokeWeight(pt_size);
       g.point(x,y,z);
   //    g.text(id, x+10, y+10, z+10); // display pt id num
     }
   
   }
   
   

In OpenGL there are a number of transformations before an image is displayed on the screen, two of them are:
1) Viewing and modelling transformation.
The viewing transformation is controlled by the camera() method and defines the position of the eye and the point in space you are looking at. The modeling transformations include translate(), rotate and scale().
2) The perspective transformation
This is controlled by the perspective() method and is like changing the focal length on a camera lens i.e. wide angle <> telephoto. It effectively describes the volume of space that will be seen on the screen.

It is important that all these transformations are kept the same for the display and the buffer.

Up to now you have accepted the default perspective for the display and just kept the others in sync.

Try adding this line immediately after creating the buffer (I don't have time right now to try it myself)

1. // buffer is created using applet dimensions
2. // cast the graphics object "buffer" to type PGraphicsOpenGL
3. buffer = (PGraphicsOpenGL)createGraphics(width, height, P3D);
4. buffer.perspective(fov, aspect, nearClip, farClip); // new code
   

Thank you, quarks!

However, adding the buffer.perspective() part of code doesn't solve the problem.

Anyway, I'll keep exploring the solution...

Thanks again!