3D CAD-style navigation with mouse

ankhelos

Hello beautiful people,

I have been working with processing for years and still haven't managed to create a nice, simple 3d navigation piece of code. I have tried various methods, including translation/rotation of the scene, as well as use of the camera() function, but no method gives me exactly the type of navigation I want.

That is the ability to pan/rotate/zoom around the center of the scene as it is currently viewed. Here is an example sketch using both methods:

float rotX, rotY, camX, camY, camZ;

void setup()
{
  size(600, 600, P3D); 
//  camX = width /2.0;
//  camY = height/2.0;
}

void draw()
{ 
  background(255);
  lights();
  fill(192);
  stroke(0);

  //navigation style 1
  translate(camX, camY, camZ);
  translate(width/2.0-camX, height/2.0-camY);
  rotateY(rotY);
  rotateX(rotX);
  translate(-(width/2.0-camX), -(height/2.0-camY));

  //navigation style 2
//  float D = (camZ*sin(rotY));
//  camera( camX + camZ*sin(rotX), camY + camZ*sin(rotY), camZ*cos(rotY)*cos(rotX),     camX ,camY, 0,     0,1,0);  

  rect(100,100,400,400);
}

void mouseDragged()
{
  if (mouseButton == LEFT)
  {
    //navigation style 1
    rotY += (pmouseX - mouseX)*0.01;
    rotX += (pmouseY - mouseY)*0.01;
    //navigation style 2
//    rotX += (mouseX - pmouseX)*0.01;
//    rotY += (mouseY - pmouseY)*0.01;
  }
  if (mouseButton == RIGHT)
  {
    //navigation style 1
    camX -= (pmouseX - mouseX);
    camY -= (pmouseY - mouseY);
    //navigation style 2
//    camX -= (mouseX - pmouseX);
//    camY -= (mouseY - pmouseY);
  }
  if (mouseButton == CENTER)
  {
    //navigation style 1
    camZ += (pmouseY - mouseY);
    //navigation style 2
//    camZ += (mouseY - pmouseY);
  }
}

In the first case, I get the rotation roughly correct (it's not really rotating aligned to the view) but panning doesn't work properly. In the second one, I get panning and one-axis rotation working seemingly ok, but rotation along xgives funny results.

Any help with what seems to me a really trivial problem would be greatly appreciated.

PS: I am not interested in using a library, like PeaseyCam (both for compatibility reasons, as well as educational ones)

ankhelos

I was hoping for some more interaction on this but just in case, In thought I'd update this..

After all, I've decided to temporarily use the obsessive camera direction library (http://gdsstudios.com/processing/libraries/ocd/) which offers a few very nive implemented ways to control a camera. Most importantly you gan get the camera's position, which was essential in my case, as I wanted to use the camera on an -offscreen- PGraphics..

However, it's still not what I was hoping to get to.

Any ideas would be very welcome.

blyk

You can use mouseWheel scroll event for zooming in and out

Reference http://processing.org/reference/mouseWheel_.html

ankhelos

Thanks for the advice blyk. However I am looking more into the problem of navigation. I am surprised that no one has replied on this basic question yet. I suppose this is not a very interesting question.. :(

hydrodog

It's not uninteresting, but it's not really about processing, more about OpenGL. Projections are confusing. First, maybe frustum would give you a definite sense of what your viewpoint is? I'm not an expert in OpenGL, but I would suggest you ask on OpenGL.org where they specialize in this problem which is, after all, not really about Processing at all.

I like your basic concept, that you take deltaX and turn it into a rotation around Y, and deltaY and turn it into a rotation around X.

Chrisir

I find it very interesting and I remember another guy tried something like this

Alas, the maths is over my head.

What I don't like about your approach is the usage of pMouseX and pmouseY. With those you don't have a defined start position you can return to.

Instead I would prefer it if the center of the window would be 0 rotation

go 10 pixels right and he rotates 10 degree right or so

go 10 pixels left from the center and he rotates 10 degree right or so

and same for up and down

Thus a mouse screen pos always results in the same 2 angles...

you could use map or so

Best, Chrisir ;-)

koogs

Is the peasycam library any use?

ankhelos

Thanks for all the suggestions.

@hydrodog: Thanks for the OpenGl suggestion, I have indeed looked into the OpenGL projection and transformation matrix but still, I haven't been able to work out a simple version for processing.

Also, I kindly disagree with your suggestion that this is not relevant to processing, as in my experience a good mouse navigation has been needed numerous times.

@koogs of course, peasycam is a very good solution, although it has a couple of drawbacks. First all this lag in dragging which causes issues with sketches that run at lower framerates, but also the inability to rotate properly per axis.

@Chrisir Your suggestion for mapping the rotations to the frame is very interesting and I have also tried that. However my problem is that sometimes you need to be able to finely navigate - eg rotate a very small amount and zoom to a part of the scene - something that I believe would not be possible this way, although I would be very glad to be proven wrong.

Again thank you all for your answers!

ankhelos

PS: I forgot to attach these links for future reference:

http://www.songho.ca/opengl/gl_projectionmatrix.html

http://www.songho.ca/opengl/gl_transform.html

http://www.fastgraph.com/makegames/3drotation/

kosowski

For 3D navigation I always use ProScene, it gives you a lot of work done and also lets you customize the way you use it. Just take a look at the examples included in the library to see how many things you can do with it.

ankhelos

Thanks, that's also a very good suggestion, I've used ProScene myself occasionally and found it very rich in features, but as mentioned in the question, I am looking for a solution that does not use libraries, both for compatibility and educational reasons.

In terms of compatibility ProScene has been quite error-prone, which I assume is due to its use of an independent renderer from processsing. Although it is rich in features it is also very heavy on the graphic card.

Especially for teaching, I believe it is essential for someone to have complete understanding and control of their code.

Thanks again for the good suggestion though.

quark

Any help with what seems to me a really trivial problem would be greatly appreciated

Working with 3D is never a trivial exercise. At least I have never found it trivial. :)

PS: I am not interested in using a library, like PeaseyCam (both for compatibility reasons, as well as educational ones)

Not sure what you mean by compatibility here. When discussing ProScene you talk about being 'error-prone' but PeasyCam is extremely stable.

I am looking for a solution that does not use libraries, both for compatibility and educational reasons.

On the educational side, are you a teacher? Are you teaching 3D manipulation?

Especially for teaching, I believe it is essential for someone to have complete understanding and control of their code.

The last point is a good one but surely 'their' code could be making use of the PeasyCam, not knowing the inner workings of PeasyCam does not mean they do not fully understand the code they created to make use of PeasyCam. (Unless it is about learning 3D manipulation). After all Processing and Java is just another software layer between us and the CPU, it doesn't mean we have to know all their inner workings. :)

of course, peasycam is a very good solution, although it has a couple of drawbacks. First all this lag in dragging which causes issues with sketches that run at lower framerates,

This library uses damped actions to give smoother rotations so there is bound to be some delay with rapid mouse movements. Processing sets the framerate to ~60 fps irrespective of the computer hardware (must either be ancient hardware or very CPU intensive sketch for it to drop below)

but also the inability to rotate properly per axis.

depends on the coordinate system chosen, easy for the camera's local space (see below) but extremely difficult using world space.

---

So going back to your question regarding navigation style - I would go for style 2 i.e. the camera because it is easier to visualise using a camera to move though a 3D scene, rather than standing still and moving the scene round the camera.

In justification of my choice I am going to make some statements /assumptions about how it might work.

The origin of the 3D world is fixed and the position of any objects in the world are defined by their xyz coordinates relative to the world origin.

There would be a simple inner class to represent the camera and it would have a number of PVectors to represent the camera's position, heading, up and side vectors. The 'view' would only be changed through the class members (functions) which would change the PVectors mentioned.

Rotations would be in the camera's local coordinate space so you effectively have pitch, yaw and roll. (The rotations would NOT be about the world axis that would not give the visual effect of moving through the 3D scene)

Zooming would be movement along the heading vector

Panning would be movement along the side vector

Once the camera class has been built then you can experiment with mapping user input to the class methods to get the movement you want.

Chrisir

Actually I wrote such a class once

Maybe I can post it

The disadvantage is that lights() makes the scene dark (or lies in shadows) when the camera is behind the scene (seen from the lights)

quark

The camera rotations for pitch, yaw and roll rotations are relative to the current state of the camera. They are NOT absolute it is important to recalculate the heading, side and up vectors after every rotation. Because PVector uses floats rather than doubles rounding errors can be introduced so the heading, side and up vectors have to be normalised after 20 or so rotations.

I have a camera class that will zoom (move forward/back), pan (move left/right), pitch, yaw and roll. It uses the Rot and Rot order classes from the Shapes3D library but they could easy be copied from the library then there would be no library dependencies. :)

Chrisir

have a look at my cam class

// see also peasycam.
// see also http://forum.processing.org/two/discussion/comment/16112#Comment_16112 


final int stateKeyboard                    = 0; 
final int stateRotateAroundScene           = 1;
final int stateCameraRotatesAroundItself   = 2;
final int stateCameraBall                  = 3;
final int stateCameraBall2                 = 4;
final int stateCameraBall3                 = 5;
int state = stateRotateAroundScene;

// cubes 
float angleCubes=45; // angle for the cubes in the scene
CameraClass cam; 

PVector ballPos = new PVector (300, 300, -100);
PVector ballVel = new PVector (1, -1, -1);

// ---------------------------------------------

void setup() {
  size (600, 600, P3D);
  cam = new CameraClass ();
}

void draw() {
  background(111);
  lights();

  switch (state) {

  case stateCameraBall:
    // rotate 
    // cam.camCurrentAngle--;
    cam.moveToAngle(); 
    // set the values of the class to the real camera
    cam.set();
    ballPos.add(ballVel);
    fill(0, 255, 0);
    noStroke();
    mySphere(ballPos.x, ballPos.y, ballPos.z, 10);
    contain(ballPos, ballVel);
    // upper left corner 
    cam.HUD_text("The ball mode 1. ");
    break;

  case stateCameraBall2:
    // rotate 
    // cam.camCurrentAngle--;
    cam.lookAtPVector(ballPos); 
    // set the values of the class to the real camera
    cam.set();
    ballPos.add(ballVel);
    fill(0, 255, 0);
    noStroke();
    mySphere(ballPos.x, ballPos.y, ballPos.z, 10);
    contain(ballPos, ballVel);
    // make the scene with the boxes 
    scene();
    // upper left corner 
    cam.HUD_text("The ball mode 2. ");
    break;

  case stateCameraBall3:
    // rotate 
    cam.follow(ballPos); 
    // set the values of the class to the real camera
    cam.set();
    // 
    ballPos.add(ballVel);
    fill(0, 255, 0);
    noStroke();
    mySphere(ballPos.x, ballPos.y, ballPos.z, 10);
    contain(ballPos, ballVel);
    // make the scene with the boxes 
    scene();
    // upper left corner 
    cam.HUD_text("The ball mode 3. ");
    break;

  case stateKeyboard:
    // set the values of the class to the real camera
    cam.set();
    // cam data
    cam.printData();
    // make the scene with the boxes 
    scene();
    // upper left corner 
    cam.HUD_text("Use keyboard to move the camera. ");
    break;

  case stateRotateAroundScene:
    // rotate 
    cam.camCurrentAngle--;
    cam.moveToAngle(); 
    // set the values of the class to the real camera
    cam.set();
    // make the scene with the boxes 
    scene();
    // upper left corner 
    cam.HUD_text("Camera flies around a center. ");
    break;

  case stateCameraRotatesAroundItself:
    cam.camCurrentAngle--;
    cam.lookAtAngle();
    // set the values of the class to the real camera
    cam.set();
    // make the scene with the boxes 
    scene();
    // upper left corner 
    cam.HUD_text("Camera is fixed but rotates around itself ");
    break;

  default:
    println ("Fatal error, unknow state, 035 with state-value "
      +state+".");
    exit();
    break;
  } // switch
} // func 

// ---------------------------------------------------

void keyPressed () {

  if (key==CODED) 
  {
    // special keys like cursor keys 
    switch (keyCode) { 
      //
      // 4 cursors signify parallel movement (y and x)
    case UP:
      cam.moveParallelAbsolute ( new PVector ( 0, -1, 0 ) );
      break;
    case DOWN:
      cam.moveParallelAbsolute ( new PVector ( 0, 1, 0 ) );
      break;
    case LEFT:
      cam.moveParallelAbsolute ( new PVector ( -1, 0, 0 ) );
      break;
    case RIGHT:
      cam.moveParallelAbsolute ( new PVector ( 1, 0, 0 ) );
      break;
    } // switch
  }

  else 
  {
    // normal keys like letters etc.
    switch (key) {
      // movement back and forth (Z)
    case 'w':
      cam.moveParallelAbsolute ( new PVector ( 0, 0, -1 ) );
      break;
    case 's':
      cam.moveParallelAbsolute ( new PVector ( 0, 0, 1 ) );
      break;
      // turn / rotate on the spot Left/Right: change camLookAt (look left / right)    
    case 'a':
      // left
      cam.camCurrentAngle--;
      cam.lookAtAngle(); 
      break;
    case 'd':
      // right
      cam.camCurrentAngle++;
      cam.lookAtAngle();
      break; 
      // rot       
    case 'p':
      cam.camCurrentAngle--;
      cam.moveToAngle(); 
      break; 
    case 'o':
      cam.camCurrentAngle++;
      cam.moveToAngle();
      break; 

      // reset view completely
    case 'r':
      cam.reset();
      break; 

    default:
      // 
      if (key>='0' && key <= '9') {
        state = int(key)-48;
        println ("state is now "+state);
        // exit();
      } 
      break;
    } // switch
  } // else
} // func 

// -------------------------------------------

void scene () {
  //fill ( 200, 23, 23);
  stroke(2);
  //noStroke();
  // rect (400, 300, 20, 20);
  // line(22, 22, 33, 33);

  float z ; 

  // one wall of boxes
  for (int x = 10; x < 600; x+= 100)
  {
    fill(x/3, 2, 2);
    for (int y = 10; y < 600; y+= 100)
    {
      z = -600;
      myBox(x, y, z, 24);
      // println ( "Box: " + x + ", "+y+ " "+z);
    }
    fill(0, 0, 254);
    z=-800;
    myBox(x, 10, z, 24);
  }
  // 
  // a few additional boxes
  fill(0, 0, 254);
  z=-400;
  myBox(220, 10, z, 24);
  myBox(600, 10, z, 24);
  z=-400;
  myBox(220, 510, z, 24);
  myBox(600, 510, z, 24);
  z=399;
  myBox(220, 510, z, 24);
  myBox(600, 510, z, 24);
  z=900;
  myBox(220, 510, z, 24);
  myBox(600, 510, z, 24);
  angleCubes++;
  // save("test.jpg");
}

void myBox(float x, float y, float z, 
float size1) {
  // one nice wrapper for build in box-command
  pushMatrix();
  translate(x, y, z);
  rotateY(radians(angleCubes));
  rotateX(radians(45));
  box(size1);
  popMatrix();
}

void mySphere(float x, float y, float z, 
float size1) {
  // one nice wrapper for build in box-command
  pushMatrix();
  translate(x, y, z);
  rotateY(radians(angleCubes));
  rotateX(radians(45));
  sphere(size1);
  popMatrix();
}

void contain(PVector ballPos, PVector ballVel) {
  //
  if (ballPos.x>500) 
    ballVel.x=abs(ballVel.x)*-1;
  if (ballPos.y>500) 
    ballVel.y=abs(ballVel.y)*-1;
  if (ballPos.z>500) 
    ballVel.z=abs(ballVel.z)*-1;

  if (ballPos.x<100) 
    ballVel.x=abs(ballVel.x);
  if (ballPos.y<100) 
    ballVel.y=abs(ballVel.y);
  if (ballPos.z<100) 
    ballVel.z=abs(ballVel.z);
}

//
// =====================================

class CameraClass {
  // capsules the normal camera() command and its vectors 
  PVector camPos;     // its vectors 
  PVector camLookAt;
  PVector camUp;

  PVector camPosInitial;     // its vectors - the default (unchanged) 
  PVector camLookAtInitial;
  PVector camUpInitial; 

  // for follow
  PVector camWhereItShouldBe = new PVector(0, 0, 0);
  PVector camAdd = new PVector(0, -50, 0);
  float easing = .027; // .07; // how fast it changes

  float camCurrentAngle=-90;   // for cam rotation around itself (around Y-axis)
  float camRadius;             // same situation 

  // constructor without parameters
  CameraClass() {
    // constr
    // set vectors 
    camPos    = new PVector(width/2.0, height/2.0, 0);
    camLookAt = new PVector(width/2.0, height/2.0, -600);
    camUp     = new PVector( 0, 1, 0 );
    // save the initial values
    camPosInitial    = camPos.get();
    camLookAtInitial = camLookAt.get();
    camUpInitial     = camUp.get();
  }  // constr

  void set() {
    // apply vectors to actual camera
    camera (camPos.x, camPos.y, camPos.z, 
    camLookAt.x, camLookAt.y, camLookAt.z, 
    camUp.x, camUp.y, camUp.z);
  }

  void reset() {
    // restore the initial vectors
    camPos    = camPosInitial.get();
    camLookAt = camLookAtInitial.get();
    camUp     = camUpInitial.get();
  } 

  // -------------------------------------------------

  void setLookAt (float x1, float y1, float z1) {
    camLookAt = new PVector(x1, y1, z1);
  }

  // -------------------------------------------------

  void printData() {
    println ( "Cam at " + camPos 
      + " looking at " + camLookAt 
      + " (angle = "
      +camCurrentAngle
      +").");
  }

  // --- rotate: two ways of rotation
  void lookAtAngle() { 
    // cam rotates in the plane (around itself); the pos of the cam is fixed.
    camRadius = camLookAt.dist (camPos); 
    camLookAt.x = camRadius * cos (radians(camCurrentAngle)) + camPos.x;
    camLookAt.z = camRadius * sin (radians(camCurrentAngle)) + camPos.z;
  }

  void moveToAngle() { 
    // rotate in the plane (around center of the scene); the pos of the cam flies
    // on a circle looking to its center.
    camRadius = camLookAt.dist (camPos); 
    camPos.x = (1.0 * camRadius) * cos (radians(camCurrentAngle)) + camLookAt.x;
    camPos.z = (1.0 * camRadius) * sin (radians(camCurrentAngle)) + camLookAt.z;
  }

  // ---- move parallel

  void moveParallelAbsolute( PVector MovementParallel ) {
    // 4 cursors signify parallel movement (of lookAt and pos).
    // E.g. you fly parallel to a wall and the angle of your looking stays 90° to the wall
    // because camPos and LookAt changes the same amount.
    // Absolute here means that the movement is the same no matter whether you rotated 
    // around yourself or not (which is a difference for x and z but not for y).
    camPos.add   ( MovementParallel );
    camLookAt.add( MovementParallel );
  } // method

  void moveParallelRelative( PVector MovementParallel ) {
    // 4 cursors signify parallel movement (of lookAt and pos).
    // E.g. you fly parallel to a wall and the angle of your looking stays 90°.
    // Relative (or locally) here means that the movement is according your previous rotation 
    // around yourself (which is a difference for x and z but not for y).
    camPos.add   ( MovementParallel );
    camLookAt.add( MovementParallel );
  } // method

  // --------

  void lookAtPVector (PVector followMe) {
    // follows a player (e.g.)
    camLookAt = followMe.get();
  }

  void follow (PVector followMe) {
    // follows a player or ball (e.g.)
    camLookAt = followMe.get();

    // behind ball but with easing
    camWhereItShouldBe.set(followMe.x+camAdd.x, followMe.y+camAdd.y, followMe.z+camAdd.z);
    camPos.x+= ( camWhereItShouldBe.x - camPos.x ) * easing; 
    camPos.y+= ( camWhereItShouldBe.y - camPos.y ) * easing;   
    camPos.z+= ( camWhereItShouldBe.z - camPos.z ) * easing;
  }

  // ---------------------------------

  void HUD_text (String a1) {
    // HUD text upper left corner 
    // this must be called at the very end of draw()

    // this is a 2D HUD 
    camera();
    hint(DISABLE_DEPTH_TEST);
    noLights();
    // ------------------
    textSize(16);
    text (a1, 20, 20);
    // ------------------
    // reset all parameters to defaults
    textAlign(LEFT, BASELINE);
    rectMode(CORNER);
    textSize(32);
    hint(ENABLE_DEPTH_TEST); // no HUD anymore
    lights();
  } // method
  //
} // class

// =====================================

ankhelos

It's taken a while to respond, as I've been a bit snowed under..

So, again, thanks for great suggestions, it has certainly made me think through the problem more in depth than I expected. Specifically:

@Chrisir: thanks a lot for sharing your code! Your camera class is very helpful and if stripped down from the rest of the sketch, it's a nice implementation that uses the core camera function. I will try to find some time to simplify it and repost it soon. One of the issues I think I will still have is that sometimes rotations are not going to happen around the screen, like in a CAD software, but I might be mistaken.

@quark: Thank you a lot for your input! Very interesing points!

Yes, working in 3D is never trivial for me even more. I was suggesing that the navigation issue would be trivial for people with some computer science knowledge, but I am glad to know I am not the only one struggling. :)

PeasyCam is indeed very stable. I've never had a single compatibility problem with it - I was actually refering to ProScene and other camera libraries like the OCD that I have also used.

Yes I am teaching Processing and 3D manipulation, among other things and this is why I want to build a clear, super simple and robust camera class that students will be able to use and understand. But your point is very good indeed. My intention is that students have the ability to tweak the 3D navigation code, as they would be implementing it in different ways in their sketches, but also that they undertstand the inner workings of that camera, reaching as low as they can in the level of the process. Your example with Java and the machine is excellent though and has made me wonder about the validity of my argument, but to justify it I would say that similarly to my interest in understanding the inner works of the virtual machine (up to a point) I would hope that they would like to reach one level down in understanding 3D manipulation. But maybe you are right. Sometimes we just need to do things.

Final point, yes the damped actions are sometimes really hard to control, as the sketches produces are in many cases very CPU intensive (physcis simulation, optimization etc). Also I believe the rotation controls are not covering a CAD style navigation. I am insisting on the CAD style as sometimes we need to restrict rotations along one axis or plane and this is what I think the problem comes down to.

Finally, regarding your suggestion, I think you are right. That it the way to go about it. For all the reasons you mention, plus the fact that rotating/translating the whole scene affects the actual position of all geometry in the scene, which then would create more problems further, down, eg exporting a dxf, writing some vertices out etc..

I have some code that is based on the ocd library, which I will try to post soon. Again, thank you for your input, really appreciated!..

:D

Chrisir

hello,

you wrote

One of the issues I think I will still have is that sometimes rotations are not going to happen around the screen

not sure what you mean.

Do you refer to this method within the class ?

  void moveToAngle() { 
    // rotate in the plane (around center of the scene); the pos of the cam flies
    // on a circle looking to its center.
    camRadius = camLookAt.dist (camPos); 
    camPos.x = (1.0 * camRadius) * cos (radians(camCurrentAngle)) + camLookAt.x;
    camPos.z = (1.0 * camRadius) * sin (radians(camCurrentAngle)) + camLookAt.z;
  }

It rotates around the camLookAt.You could add

camPos.y =  camLookAt.y; 

When your scene (or its center) is somewhere else, you need to set camLookAt accordingly (e.g. average of your point cloud or whatever)

BTW

I think btw some functions in my class are not finished yet; e.g. moveParallelRelative and follow.

I enjoyed reading your post btw.

Best, Chrisir ;-)

ankhelos

Hi Chrisir,

Thanks for the reply and again apologies for taking long to reply. I have to fix my notification settings, really..

Yes I meant the rotation happening around the center of the screen. Adding this line sure helps combined also with relevant user input. Overall, I think your class is very good and thanks for sharing it. I am aiming to use it and hopefully come back with additions or more questions.

Thanks for taking the time to read the post and reply. After using processing for five years, it's so good to finally feel the community. :)