using modelX and screenX for 3D picking - but how?

edited January 2016 in Questions about Code

hello all,

I am trying to be able to click on a sphere in a 3D sketch with the mouse to select a sphere.

Since I use rotate and translate when placing the sphere, I wanted to use modelX,modelY to get the model position. But to have a match with the mouse I need ScreenX,ScreenY. Or so I am guessing.

Not sure how to handle this though:

this is part of class that in turn is stored in ArrayList to hold all the spheres:

    pushMatrix(); 
      translate(x, y, z); 

      fill(col); 
      noStroke(); 

      if (showBox||true) {     
        //box(55);
        //ellipse(0, 0, 7, 7);
        sphere(7);

        // the sphere / box was drawn at (0, 0, 0), store that location
        float x1 = modelX(0, 0, 0); // this takes care of translate and rotate 
        float y1 = modelY(0, 0, 0);
        float z1 = modelZ(0, 0, 0);

        scX = screenX(x1, y1, z1); // this takes care of the projection into 2D space for the mouse
        scY = screenX(x1, y1, z1);
      }
      popMatrix(); 

I now want to match scX and scY with mouseX and mouseY:

void mousePressed() {

  PVectorInt result = new PVectorInt(0, 0, 0);

  for (int i = 0; i < boxes.length; i++) {
    for (int j = 0; j < boxes[i].length; j++) {
      for (int k = 0; k < boxes[i][j].length; k++) {

        if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 40) {
          result = new PVectorInt( i, j, k); 
          listShape.add ( new  PVectorInt( i, j, k)); 
          println(listShape.size());
          return;
        }
      }
    }
  }
}

but it doesn't work....

please tell me how I handle modelX and screenX

thank you...!

Chrisir

Answers

  • edited March 2017
    // States
    final int HELP = 0; // consts 
    final int GAME = 1;
    int state = GAME;   // current 
    
    
    final int viewsTotal = 0;
    //final int viewsNeighbours = 1;  // 26 Neighbours 
    //final int viewsPlaneXY = 2;
    //final int viewsNeighboursInThePlaneXZ = 3;  // 8 Neighbours 
    //final int viewsNeighboursInThePlaneXY = 4;
    //final int viewsNeighboursInThePlaneYZ = 5;
    //final int viewsNeighboursDiagonally = 6;   // 8 Neighbours 
    
    int view = viewsTotal;
    
    // boxes 
    BoxClass[][][] boxes = new BoxClass[6][6][6];
    
    // lines
    ArrayList<PVectorInt> listShape = new ArrayList(); 
    
    // cam  // not in use 
    PVector camPos;     // its vectors 
    PVector camLookAt;
    PVector camUp;
    
    // cam rotation 
    float camCurrentAngle;      
    float camRadius=1.0;         
    
    PFont font1;
    
    
    // String currEntry = ""; 
    
    boolean showGrid = false;  
    
    
    // --------------------------------------------------------
    // main funcs 
    
    void setup() {
      size(1400, 800, OPENGL);
    
      // set cams vectors // not in use 
      camPos    = new PVector(width/2.0, height/2.0, 600);
      camLookAt = new PVector(width/2.0, height/2.0, -300);
      camUp     = new PVector( 0, 1, 0 );
    
      defineBoxes();
      background(111);
      font1 = createFont("Arial", 32);
      textFont(font1); 
      // selected = new PVectorInt (0, 0, 0);
    
      camCurrentAngle=90;
      //lookAtAngle();
    } // func 
    
    void draw() {
      switch (state) {
    
      case HELP:
        // to do  
        background(111);
        text("Help\n\nThis is the help..."
          +"\n\npress any key.", 
          32, 344);
        break;
    
      case GAME:
        playTheGame();   
        break;
      } // switch
    } // func draw()
    
    // ----------------------------------------------------
    
    void playTheGame() {
    
      background(111);
    
      // reset some stuff that was set differently by the HUD
      hint(ENABLE_DEPTH_TEST);
      textSize(32);
    
      //  camera (camPos.x, camPos.y, camPos.z, 
      //    camLookAt.x, camLookAt.y, camLookAt.z, 
      //    camUp.x, camUp.y, camUp.z);
    
      lights();
    
      scale (camRadius);
    
      translate(width/2, 0);
      rotateY(radians(camCurrentAngle));
    
      showTheGridDependingOnCurrentView();
    
      // show the selected cell with thick outline 
      // strokeWeight(5);          
      // boxes[ selected.x][ selected.y][ selected.z].show(true);
    
      strokeWeight(1);
    
      // camera 
      if (keyPressed&&key=='r')
        camCurrentAngle++;
      if (keyPressed&&key=='l')
        camCurrentAngle--;
      // lookAtAngle();
    
      // 2D part / HUD  ---------------------------------------------
      camera();
      //  hint(DISABLE_DEPTH_MASK);
      hint(DISABLE_DEPTH_TEST);
      noLights();
      textMode(MODEL);
      //  textMode(SHAPE);
      textSize(22);
      text("Hello. Use r and l.", 
        10, 20 );
      // text(mouseX + "," + mouseY, mouseX+7, mouseY-7);
    }
    
    // ------------------------------------------------------------------
    
    void showTheGridDependingOnCurrentView() {
    
      switch (view) {
    
      case viewsTotal:
    
        //   println("here");
        // show list 
        // for (PVectorInt pvInt : listShape) {
        stroke(255, 2, 2);
        if (listShape.size()>1) 
          for (int i = 0; i < listShape.size()-1; i+=2) {
            PVectorInt pvInt1=listShape.get(i);
            PVectorInt pvInt2=listShape.get(i+1);
    
            BoxClass b1=boxes[pvInt1.x][ pvInt1.y][ pvInt1.z];
            BoxClass b2=boxes[pvInt2.x][ pvInt2.y][ pvInt2.z];
    
            line (b1.x, b1.y, b1.z, 
              b2.x, b2.y, b2.z);
          }// for 
    
        // show boxes : all
        for (int i = 0; i < boxes.length; i++) {
          for (int j = 0; j < boxes[i].length; j++) {
            for (int k = 0; k < boxes[i][j].length; k++) {
              //          if (selected.x==i&&selected.y==j&&selected.z==k)
              //            strokeWeight(5); 
              //          else
              //            strokeWeight(1); 
              boxes[i][ j][k].show(false);
            }
          }
        }// for
        break; 
    
      default : 
        // error 
        break;
      } // switch
    } // func 
    
    // ----------------------------------------------------
    // input funcs
    
    void keyPressed () {
    
      switch (state) {
    
      case HELP : 
        state = GAME; 
        break; 
    
      case GAME : 
        if (key!=CODED) {
          // not CODED -----------------------------------
          if (key=='X') {
            // reset  
            defineBoxes();
          } else if (key == 's') {
            showGrid=!showGrid;
          } else {
            // do nothing
          }
        } else {
          // if (key==CODED) { --------------------------------------------
          //
          switch (keyCode) {
    
          case java.awt.event.KeyEvent.VK_F1 : 
            // F1 
            state = HELP; 
            break; 
    
          case java.awt.event.KeyEvent.VK_PAGE_UP : 
            camRadius-=.01; 
            break; 
    
          case java.awt.event.KeyEvent.VK_PAGE_DOWN : 
            camRadius+=.01; 
            break; 
    
          default : 
            // do nothing 
            break;
          } // switch
        } // else
    
        break;
      } // switch 
      //
    } // func 
    
    // ----------------------------------------------------
    // misc funcs
    
    void defineBoxes() {
      // define boxes
      int k2=0; 
      char letter=' '; 
      for (int i = 0; i < boxes.length; i++) {
        for (int j = 0; j < boxes[i].length; j++) {
          for (int k = 0; k < boxes[i][j].length; k++) {
            // prepare values 
            color currentCol = color (255); //  color (random(255), random(255), random(255));
            boolean exist = true; 
            //        // the percentage 
            //        if (random(100) > 50) { 
            //          exist = true;
            //        } else
            //          exist = false;
            //        if (k2<initText.length())
            //          letter = initText.charAt(k2);
            //        k2++;  
            // create a box   
            boxes[i][ j][k] = new BoxClass( -180 + i*(height/10), 
              158 + j*(height/10), 
              130 - k*(height/10), 
              currentCol, 
              exist, 
              ' ' );
          }
        }
      } // for
    } // func 
    
    void lookAtAngle222222222222222() {
      // rotate in the plane : cam 
      //camRadius = camLookAt.dist (camPos); 
      // camRadius = 100;
      camPos.x = camRadius * cos (radians(camCurrentAngle)) + camLookAt.x; 
      camPos.z = camRadius * sin (radians(camCurrentAngle)) + camLookAt.z;
    } // func 
    
    // ----------------------------------------
    
    void mousePressed() {
    
      PVectorInt result = new PVectorInt(0, 0, 0);
    
      for (int i = 0; i < boxes.length; i++) {
        for (int j = 0; j < boxes[i].length; j++) {
          for (int k = 0; k < boxes[i][j].length; k++) {
    
            if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 40) {
              result = new PVectorInt( i, j, k); 
              listShape.add ( new  PVectorInt( i, j, k)); 
              println(listShape.size());
              return;
            }
          }
        }
      }
    }
    
    void sphereParam222222(float x1, float y1, float z1) {
      // the red spheres 
      pushMatrix(); 
      translate(x1, y1, z1); 
      noStroke(); 
      fill(255, 2, 0); 
      sphere(16); 
      popMatrix();
    }
    
    // =====================================================
    // classes
    
    class BoxClass {
    
      // this class represents one Box / Cell
    
      float x; 
      float y; 
      float z; 
    
      float scX, scY; 
    
      // char letter = ' '; //  char ( int ( random (65, 65+24) )) ;  
    
      color col; 
      boolean exist = true; 
    
      // constr
      BoxClass(float x_, float y_, float z_, 
        color col_, 
        boolean exist_, 
        char letter_) {
        x = x_; 
        y = y_; 
        z = z_; 
        col = col_; 
        exist = exist_; 
        //letter = letter_;
      } // constr
    
      void show(boolean showBox) {
        if (exist||showGrid) {
          pushMatrix(); 
          translate(x, y, z); 
    
          fill(col); 
          noStroke(); 
    
          if (showBox||true) {     
            //box(55);
            //ellipse(0, 0, 7, 7);
            sphere(7);
    
            // the sphere / box was drawn at (0, 0, 0), store that location
            float x1 = modelX(0, 0, 0);
            float y1 = modelY(0, 0, 0);
            float z1 = modelZ(0, 0, 0);
    
            scX = screenX(x1, y1, z1); 
            scY = screenY(x1, y1, z1);
          }
          popMatrix(); 
    
    
    
          // scX = screenX(x, y, z); 
          // scY = screenY(x, y, z);
        }
      } // method
    } // class 
    
    // ==============================================================
    
    class PVectorInt {
      // it's like a PVector but int (not float)
      int x, y, z; 
      PVectorInt(int x_, int y_, int z_) {
        x=x_; 
        y=y_; 
        z=z_;
      } // constr
    }
    // =============================================================
    
  • edited January 2016

    the first post describes the problem.

    the 2nd is mcve if anyone needs a running version.

    lines 338-352 are relevant

    (and line 273)

    please help.

    thank you

    Chrisir ;-)

  • Answer ✓

    solved.

    ;-)

  • for anyone who is interested

    I had a typo above

    screenX and screenY themself do take into account the matrix (translate / rotate) so no need to use modelX... first

  • edited March 2017

    this is working version with screenX,screenY in case anyone is interested

        // States
        final int HELP = 0; // consts 
        final int GAME = 1;
        int state = GAME;   // current 
    
        final int viewsTotal = 0;
        //final int viewsNeighbours = 1;  // 26 Neighbours 
        //final int viewsPlaneXY = 2;
        //final int viewsNeighboursInThePlaneXZ = 3;  // 8 Neighbours 
        //final int viewsNeighboursInThePlaneXY = 4;
        //final int viewsNeighboursInThePlaneYZ = 5;
        //final int viewsNeighboursDiagonally = 6;   // 8 Neighbours 
    
        int view = viewsTotal;
    
        // boxes 
        BoxClass[][][] boxes = new BoxClass[6][6][6];
    
        // cam rotation 
        float camCurrentAngle;      
        float camRadius=1.0;         
    
        PFont font1;
    
        boolean showGrid = false;  
    
        PVectorInt selected = new PVectorInt(-1, 0, 0);
    
        // --------------------------------------------------------
        // main funcs 
    
        void setup() {
          size(1400, 800, P3D);
    
          defineBoxes();
          background(111);
          font1 = createFont("Arial", 32);
          textFont(font1); 
          // selected = new PVectorInt (0, 0, 0);
    
          camCurrentAngle=90;
          //lookAtAngle();
        } // func 
    
        void draw() {
          switch (state) {
    
          case HELP:
            // to do  
            background(111);
            text("Help\n\nThis is the help..."
              +"\n\npress any key.", 
              32, 344);
            break;
    
          case GAME:
            playTheGame();   
            break;
    
          default:
            //exit
            break;
          } // switch
        } // func draw()
    
        // ----------------------------------------------------
    
        void playTheGame() {
    
          background(111);
    
          // reset some stuff that was set differently by the HUD
          hint(ENABLE_DEPTH_TEST);
          textSize(32);
          lights();
    
          scale (camRadius);
    
          translate(width/2, 0);
          rotateY(radians(camCurrentAngle));
    
          showTheGridDependingOnCurrentView();
          strokeWeight(1);
    
          // camera 
          if (keyPressed&&key=='r')
            camCurrentAngle++;
          if (keyPressed&&key=='l')
            camCurrentAngle--;
    
          // 2D part / HUD  ---------------------------------------------
          camera();
          //  hint(DISABLE_DEPTH_MASK);
          hint(DISABLE_DEPTH_TEST);
          noLights();
          textMode(MODEL);
          //  textMode(SHAPE);
          textSize(22);
          text("Hello. Use r and l to rotate. Select a sphere with nouse click. Demonstrates the usage of screenX,screenY.", 
            10, 20 );
          // text(mouseX + "," + mouseY, mouseX+7, mouseY-7);
        }
    
        // ------------------------------------------------------------------
    
        void showTheGridDependingOnCurrentView() {
    
          switch (view) {
    
          case viewsTotal:
    
            //   println("here");
            // show list 
            // for (PVectorInt pvInt : listShape) {
            stroke(255, 2, 2);
    
            // show boxes : all
            boolean showInRed=false;
            for (int i = 0; i < boxes.length; i++) {
              for (int j = 0; j < boxes[i].length; j++) {
                for (int k = 0; k < boxes[i][j].length; k++) {
    
                  if (selected.x==i&&selected.y==j&&selected.z==k)
                    showInRed=true;
                  else
                    showInRed=false; 
    
                  boxes[i][ j][k].show(showInRed);
                }
              }
            }// for
            break; 
    
          default : 
            // error 
            break;
          } // switch
        } // func 
    
        // ----------------------------------------------------
        // input funcs
    
        void keyPressed () {
    
          switch (state) {
    
          case HELP : 
            state = GAME; 
            break; 
    
          case GAME : 
            if (key!=CODED) {
              // not CODED -----------------------------------
              if (key=='X') {
                // reset  
                defineBoxes();
              } else if (key == 's') {
                showGrid=!showGrid;
              } else {
                // do nothing
              }
            } else {
              // if (key==CODED) { --------------------------------------------
              //
              switch (keyCode) {
    
              case java.awt.event.KeyEvent.VK_F1 : 
                // F1 
                state = HELP; 
                break; 
    
              case java.awt.event.KeyEvent.VK_PAGE_UP : 
                camRadius-=.01; 
                break; 
    
              case java.awt.event.KeyEvent.VK_PAGE_DOWN : 
                camRadius+=.01; 
                break; 
    
              default : 
                // do nothing 
                break;
              } // switch
            } // else
    
            break;
          } // switch 
          //
        } // func 
    
        // ----------------------------------------------------
        // misc funcs
    
        void defineBoxes() {
          // define boxes
          int k2=0; 
          char letter=' '; 
          for (int i = 0; i < boxes.length; i++) {
            for (int j = 0; j < boxes[i].length; j++) {
              for (int k = 0; k < boxes[i][j].length; k++) {
                // prepare values 
                color currentCol = color (255); //  color (random(255), random(255), random(255));
                boolean exist = true; 
    
                // create a box   
                boxes[i][ j][k] = new BoxClass( -180 + i*(height/10), 
                  158 + j*(height/10), 
                  130 - k*(height/10), 
                  currentCol, 
                  exist, 
                  ' ' );
              }
            }
          } // for
        } // func 
    
        // ----------------------------------------
    
        void mousePressed() {
    
          for (int i = 0; i < boxes.length; i++) {
            for (int j = 0; j < boxes[i].length; j++) {
              for (int k = 0; k < boxes[i][j].length; k++) {
    
                if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 20) {
                  selected= new PVectorInt( i, j, k); 
                  return;
                }
              }
            }
          }
        }
    
        // =====================================================
        // classes
    
        class BoxClass {
    
          // this class represents one Box / Cell
    
          float x; 
          float y; 
          float z; 
    
          float scX, scY; 
    
          color col; 
          boolean exist = true; 
    
          // constr
          BoxClass(float x_, float y_, float z_, 
            color col_, 
            boolean exist_, 
            char letter_) {
            x = x_; 
            y = y_; 
            z = z_; 
            col = col_; 
            exist = exist_;
          } // constr
    
          void show(boolean showBoxAsRed) {
            if (exist||showGrid) {
              pushMatrix(); 
              translate(x, y, z); 
    
              fill(col); 
              if (showBoxAsRed)
                fill(255, 2, 2); //RED 
              noStroke(); 
    
              sphere(7);
              scX = screenX(0, 0, 0); 
              scY = screenY(0, 0, 0);
    
              popMatrix();
            }
          } // method
        } // class 
    
        // ==============================================================
    
        class PVectorInt {
          // it's like a PVector but int (not float)
          int x, y, z; 
          PVectorInt(int x_, int y_, int z_) {
            x=x_; 
            y=y_; 
            z=z_;
          } // constr
    
          PVectorInt copy() {
            return new PVectorInt(x, y, z);
          }
        }
        // ===========================================================
    
  • Answer ✓

    Sometimes when the corner of the cube is in the middle position (no facing the face of the cube but between two faces) the ball picking action doesn't seem to work in certain spheres in that rotation. However if you rotate 5 or 10 degrees, you will notice that the algorithm was indeed working but it was picking a sphere behind it.

    You think the algorithm is bias to picking objects at certain depth? @Chrisir, could you explain how do you project from a 2D into a 3D?

    Kf

  • edited March 2017

    Sometimes when the corner of the cube is in the middle position (no facing the face of the cube but between two faces) the ball picking action doesn't seem to work in certain spheres in that rotation. However if you rotate 5 or 10 degrees, you will notice that the algorithm was indeed working but it was picking a sphere behind it.

    True. I use dist() to figure out which ball has been clicked. And it stops at the first one. Which is sometimes wrong. Alternatively, we could store all distances and then keep the smallest distance. Might be slower but more precise.

    You think the algorithm is bias to picking objects at certain depth?

    This is referring to the same issue, right? It is biased in that it uses a triple for-loop in mousePressed, and the order of that determines which depth is checked first. The above idea should solve this. We could try to run the depth from behind to the front so that we have the smallest balls (more depth) first.

    could you explain how do you project from a 2D into a 3D?

    You can't but there are workarounds.

    See my sketch here (and also good discussion):

    With my sketch you can drag a ball in 3D space with the mouse:

    • when you just drag it, you change x,y

    • when you hold a key and drag, you change Z with mouse up and down (y)

    https://forum.processing.org/two/discussion/20755/how-to-map-a-2d-coordinate-into-3d#latest

Sign In or Register to comment.