#### Howdy, Stranger!

We are about to switch to a new forum software. Until then we have removed the registration on this forum.

# using modelX and screenX for 3D picking - but how?

edited January 2016

hello all,

I am trying to be able to click on a sphere in a 3D sketch with the mouse to select a sphere.

Since I use rotate and translate when placing the sphere, I wanted to use `modelX,modelY` to get the model position. But to have a match with the mouse I need `ScreenX,ScreenY`. Or so I am guessing.

Not sure how to handle this though:

this is part of class that in turn is stored in ArrayList to hold all the spheres:

``````    pushMatrix();
translate(x, y, z);

fill(col);
noStroke();

if (showBox||true) {
//box(55);
//ellipse(0, 0, 7, 7);
sphere(7);

// the sphere / box was drawn at (0, 0, 0), store that location
float x1 = modelX(0, 0, 0); // this takes care of translate and rotate
float y1 = modelY(0, 0, 0);
float z1 = modelZ(0, 0, 0);

scX = screenX(x1, y1, z1); // this takes care of the projection into 2D space for the mouse
scY = screenX(x1, y1, z1);
}
popMatrix();
``````

I now want to match scX and scY with mouseX and mouseY:

``````void mousePressed() {

PVectorInt result = new PVectorInt(0, 0, 0);

for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {

if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 40) {
result = new PVectorInt( i, j, k);
listShape.add ( new  PVectorInt( i, j, k));
println(listShape.size());
return;
}
}
}
}
}
``````

but it doesn't work....

please tell me how I handle modelX and screenX

thank you...!

Chrisir

Tagged:

• ``````// States
final int HELP = 0; // consts
final int GAME = 1;
int state = GAME;   // current

final int viewsTotal = 0;
//final int viewsNeighbours = 1;  // 26 Neighbours
//final int viewsPlaneXY = 2;
//final int viewsNeighboursInThePlaneXZ = 3;  // 8 Neighbours
//final int viewsNeighboursInThePlaneXY = 4;
//final int viewsNeighboursInThePlaneYZ = 5;
//final int viewsNeighboursDiagonally = 6;   // 8 Neighbours

int view = viewsTotal;

// boxes
BoxClass[][][] boxes = new BoxClass;

// lines
ArrayList<PVectorInt> listShape = new ArrayList();

// cam  // not in use
PVector camPos;     // its vectors
PVector camLookAt;
PVector camUp;

// cam rotation
float camCurrentAngle;

PFont font1;

// String currEntry = "";

boolean showGrid = false;

// --------------------------------------------------------
// main funcs

void setup() {
size(1400, 800, OPENGL);

// set cams vectors // not in use
camPos    = new PVector(width/2.0, height/2.0, 600);
camLookAt = new PVector(width/2.0, height/2.0, -300);
camUp     = new PVector( 0, 1, 0 );

defineBoxes();
background(111);
font1 = createFont("Arial", 32);
textFont(font1);
// selected = new PVectorInt (0, 0, 0);

camCurrentAngle=90;
//lookAtAngle();
} // func

void draw() {
switch (state) {

case HELP:
// to do
background(111);
text("Help\n\nThis is the help..."
+"\n\npress any key.",
32, 344);
break;

case GAME:
playTheGame();
break;
} // switch
} // func draw()

// ----------------------------------------------------

void playTheGame() {

background(111);

// reset some stuff that was set differently by the HUD
hint(ENABLE_DEPTH_TEST);
textSize(32);

//  camera (camPos.x, camPos.y, camPos.z,
//    camLookAt.x, camLookAt.y, camLookAt.z,
//    camUp.x, camUp.y, camUp.z);

lights();

translate(width/2, 0);

showTheGridDependingOnCurrentView();

// show the selected cell with thick outline
// strokeWeight(5);
// boxes[ selected.x][ selected.y][ selected.z].show(true);

strokeWeight(1);

// camera
if (keyPressed&&key=='r')
camCurrentAngle++;
if (keyPressed&&key=='l')
camCurrentAngle--;
// lookAtAngle();

// 2D part / HUD  ---------------------------------------------
camera();
hint(DISABLE_DEPTH_TEST);
noLights();
textMode(MODEL);
//  textMode(SHAPE);
textSize(22);
text("Hello. Use r and l.",
10, 20 );
// text(mouseX + "," + mouseY, mouseX+7, mouseY-7);
}

// ------------------------------------------------------------------

void showTheGridDependingOnCurrentView() {

switch (view) {

case viewsTotal:

//   println("here");
// show list
// for (PVectorInt pvInt : listShape) {
stroke(255, 2, 2);
if (listShape.size()>1)
for (int i = 0; i < listShape.size()-1; i+=2) {
PVectorInt pvInt1=listShape.get(i);
PVectorInt pvInt2=listShape.get(i+1);

BoxClass b1=boxes[pvInt1.x][ pvInt1.y][ pvInt1.z];
BoxClass b2=boxes[pvInt2.x][ pvInt2.y][ pvInt2.z];

line (b1.x, b1.y, b1.z,
b2.x, b2.y, b2.z);
}// for

// show boxes : all
for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {
//          if (selected.x==i&&selected.y==j&&selected.z==k)
//            strokeWeight(5);
//          else
//            strokeWeight(1);
boxes[i][ j][k].show(false);
}
}
}// for
break;

default :
// error
break;
} // switch
} // func

// ----------------------------------------------------
// input funcs

void keyPressed () {

switch (state) {

case HELP :
state = GAME;
break;

case GAME :
if (key!=CODED) {
// not CODED -----------------------------------
if (key=='X') {
// reset
defineBoxes();
} else if (key == 's') {
showGrid=!showGrid;
} else {
// do nothing
}
} else {
// if (key==CODED) { --------------------------------------------
//
switch (keyCode) {

case java.awt.event.KeyEvent.VK_F1 :
// F1
state = HELP;
break;

case java.awt.event.KeyEvent.VK_PAGE_UP :
break;

case java.awt.event.KeyEvent.VK_PAGE_DOWN :
break;

default :
// do nothing
break;
} // switch
} // else

break;
} // switch
//
} // func

// ----------------------------------------------------
// misc funcs

void defineBoxes() {
// define boxes
int k2=0;
char letter=' ';
for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {
// prepare values
color currentCol = color (255); //  color (random(255), random(255), random(255));
boolean exist = true;
//        // the percentage
//        if (random(100) > 50) {
//          exist = true;
//        } else
//          exist = false;
//        if (k2<initText.length())
//          letter = initText.charAt(k2);
//        k2++;
// create a box
boxes[i][ j][k] = new BoxClass( -180 + i*(height/10),
158 + j*(height/10),
130 - k*(height/10),
currentCol,
exist,
' ' );
}
}
} // for
} // func

void lookAtAngle222222222222222() {
// rotate in the plane : cam
} // func

// ----------------------------------------

void mousePressed() {

PVectorInt result = new PVectorInt(0, 0, 0);

for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {

if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 40) {
result = new PVectorInt( i, j, k);
listShape.add ( new  PVectorInt( i, j, k));
println(listShape.size());
return;
}
}
}
}
}

void sphereParam222222(float x1, float y1, float z1) {
// the red spheres
pushMatrix();
translate(x1, y1, z1);
noStroke();
fill(255, 2, 0);
sphere(16);
popMatrix();
}

// =====================================================
// classes

class BoxClass {

// this class represents one Box / Cell

float x;
float y;
float z;

float scX, scY;

// char letter = ' '; //  char ( int ( random (65, 65+24) )) ;

color col;
boolean exist = true;

// constr
BoxClass(float x_, float y_, float z_,
color col_,
boolean exist_,
char letter_) {
x = x_;
y = y_;
z = z_;
col = col_;
exist = exist_;
//letter = letter_;
} // constr

void show(boolean showBox) {
if (exist||showGrid) {
pushMatrix();
translate(x, y, z);

fill(col);
noStroke();

if (showBox||true) {
//box(55);
//ellipse(0, 0, 7, 7);
sphere(7);

// the sphere / box was drawn at (0, 0, 0), store that location
float x1 = modelX(0, 0, 0);
float y1 = modelY(0, 0, 0);
float z1 = modelZ(0, 0, 0);

scX = screenX(x1, y1, z1);
scY = screenY(x1, y1, z1);
}
popMatrix();

// scX = screenX(x, y, z);
// scY = screenY(x, y, z);
}
} // method
} // class

// ==============================================================

class PVectorInt {
// it's like a PVector but int (not float)
int x, y, z;
PVectorInt(int x_, int y_, int z_) {
x=x_;
y=y_;
z=z_;
} // constr
}
// =============================================================
``````
• the first post describes the problem.

the 2nd is mcve if anyone needs a running version.

lines 338-352 are relevant

(and line 273)

thank you

Chrisir ;-)

• solved.

;-)

• for anyone who is interested

screenX and screenY themself do take into account the matrix (translate / rotate) so no need to use modelX... first

• this is working version with screenX,screenY in case anyone is interested

``````    // States
final int HELP = 0; // consts
final int GAME = 1;
int state = GAME;   // current

final int viewsTotal = 0;
//final int viewsNeighbours = 1;  // 26 Neighbours
//final int viewsPlaneXY = 2;
//final int viewsNeighboursInThePlaneXZ = 3;  // 8 Neighbours
//final int viewsNeighboursInThePlaneXY = 4;
//final int viewsNeighboursInThePlaneYZ = 5;
//final int viewsNeighboursDiagonally = 6;   // 8 Neighbours

int view = viewsTotal;

// boxes
BoxClass[][][] boxes = new BoxClass;

// cam rotation
float camCurrentAngle;

PFont font1;

boolean showGrid = false;

PVectorInt selected = new PVectorInt(-1, 0, 0);

// --------------------------------------------------------
// main funcs

void setup() {
size(1400, 800, P3D);

defineBoxes();
background(111);
font1 = createFont("Arial", 32);
textFont(font1);
// selected = new PVectorInt (0, 0, 0);

camCurrentAngle=90;
//lookAtAngle();
} // func

void draw() {
switch (state) {

case HELP:
// to do
background(111);
text("Help\n\nThis is the help..."
+"\n\npress any key.",
32, 344);
break;

case GAME:
playTheGame();
break;

default:
//exit
break;
} // switch
} // func draw()

// ----------------------------------------------------

void playTheGame() {

background(111);

// reset some stuff that was set differently by the HUD
hint(ENABLE_DEPTH_TEST);
textSize(32);
lights();

translate(width/2, 0);

showTheGridDependingOnCurrentView();
strokeWeight(1);

// camera
if (keyPressed&&key=='r')
camCurrentAngle++;
if (keyPressed&&key=='l')
camCurrentAngle--;

// 2D part / HUD  ---------------------------------------------
camera();
hint(DISABLE_DEPTH_TEST);
noLights();
textMode(MODEL);
//  textMode(SHAPE);
textSize(22);
text("Hello. Use r and l to rotate. Select a sphere with nouse click. Demonstrates the usage of screenX,screenY.",
10, 20 );
// text(mouseX + "," + mouseY, mouseX+7, mouseY-7);
}

// ------------------------------------------------------------------

void showTheGridDependingOnCurrentView() {

switch (view) {

case viewsTotal:

//   println("here");
// show list
// for (PVectorInt pvInt : listShape) {
stroke(255, 2, 2);

// show boxes : all
boolean showInRed=false;
for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {

if (selected.x==i&&selected.y==j&&selected.z==k)
showInRed=true;
else
showInRed=false;

boxes[i][ j][k].show(showInRed);
}
}
}// for
break;

default :
// error
break;
} // switch
} // func

// ----------------------------------------------------
// input funcs

void keyPressed () {

switch (state) {

case HELP :
state = GAME;
break;

case GAME :
if (key!=CODED) {
// not CODED -----------------------------------
if (key=='X') {
// reset
defineBoxes();
} else if (key == 's') {
showGrid=!showGrid;
} else {
// do nothing
}
} else {
// if (key==CODED) { --------------------------------------------
//
switch (keyCode) {

case java.awt.event.KeyEvent.VK_F1 :
// F1
state = HELP;
break;

case java.awt.event.KeyEvent.VK_PAGE_UP :
break;

case java.awt.event.KeyEvent.VK_PAGE_DOWN :
break;

default :
// do nothing
break;
} // switch
} // else

break;
} // switch
//
} // func

// ----------------------------------------------------
// misc funcs

void defineBoxes() {
// define boxes
int k2=0;
char letter=' ';
for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {
// prepare values
color currentCol = color (255); //  color (random(255), random(255), random(255));
boolean exist = true;

// create a box
boxes[i][ j][k] = new BoxClass( -180 + i*(height/10),
158 + j*(height/10),
130 - k*(height/10),
currentCol,
exist,
' ' );
}
}
} // for
} // func

// ----------------------------------------

void mousePressed() {

for (int i = 0; i < boxes.length; i++) {
for (int j = 0; j < boxes[i].length; j++) {
for (int k = 0; k < boxes[i][j].length; k++) {

if (dist(mouseX, mouseY, boxes[i][j][k].scX, boxes[i][j][k].scY) < 20) {
selected= new PVectorInt( i, j, k);
return;
}
}
}
}
}

// =====================================================
// classes

class BoxClass {

// this class represents one Box / Cell

float x;
float y;
float z;

float scX, scY;

color col;
boolean exist = true;

// constr
BoxClass(float x_, float y_, float z_,
color col_,
boolean exist_,
char letter_) {
x = x_;
y = y_;
z = z_;
col = col_;
exist = exist_;
} // constr

void show(boolean showBoxAsRed) {
if (exist||showGrid) {
pushMatrix();
translate(x, y, z);

fill(col);
if (showBoxAsRed)
fill(255, 2, 2); //RED
noStroke();

sphere(7);
scX = screenX(0, 0, 0);
scY = screenY(0, 0, 0);

popMatrix();
}
} // method
} // class

// ==============================================================

class PVectorInt {
// it's like a PVector but int (not float)
int x, y, z;
PVectorInt(int x_, int y_, int z_) {
x=x_;
y=y_;
z=z_;
} // constr

PVectorInt copy() {
return new PVectorInt(x, y, z);
}
}
// ===========================================================
``````
• Sometimes when the corner of the cube is in the middle position (no facing the face of the cube but between two faces) the ball picking action doesn't seem to work in certain spheres in that rotation. However if you rotate 5 or 10 degrees, you will notice that the algorithm was indeed working but it was picking a sphere behind it.

You think the algorithm is bias to picking objects at certain depth? @Chrisir, could you explain how do you project from a 2D into a 3D?

Kf

• Sometimes when the corner of the cube is in the middle position (no facing the face of the cube but between two faces) the ball picking action doesn't seem to work in certain spheres in that rotation. However if you rotate 5 or 10 degrees, you will notice that the algorithm was indeed working but it was picking a sphere behind it.

True. I use dist() to figure out which ball has been clicked. And it stops at the first one. Which is sometimes wrong. Alternatively, we could store all distances and then keep the smallest distance. Might be slower but more precise.

You think the algorithm is bias to picking objects at certain depth?

This is referring to the same issue, right? It is biased in that it uses a triple for-loop in mousePressed, and the order of that determines which depth is checked first. The above idea should solve this. We could try to run the depth from behind to the front so that we have the smallest balls (more depth) first.

could you explain how do you project from a 2D into a 3D?

You can't but there are workarounds.

See my sketch here (and also good discussion):

With my sketch you can drag a ball in 3D space with the mouse:

• when you just drag it, you change x,y

• when you hold a key and drag, you change Z with mouse up and down (y)

https://forum.processing.org/two/discussion/20755/how-to-map-a-2d-coordinate-into-3d#latest