In urgent need of assistance to help get me started with this idea for my final project

edited June 2017 in Programming Questions

Although a little ambitious I want to create a interactive installation for my final piece in two weeks time. I have devised to procedurally generate a landscape that is displayed on a screen via a projector. To create some interactivity I have obtained kinect, with the use of this I want the kinect to track the speed of movement in the viewers body, forming sharp jagged terrain when the viewer moves quickly and smoothed and less harsh gradients when the viewer moves subtly. I am a complete novice.

Could someone please help me get the code started off and if you have any useful advice of how to approach the code in processing to enable all these functions that would be much appreciated

Thank you for your time,

Answers

  • Have you installed processing and the kinect?

    Did you run an example of kinect?

  • I installed processing a few weeks ago and have grasped some of the basics and my kinect is in the pos. I have run the majority of examples and experimented with creating some basic terrains but nothing as advanced

  • edited May 2017

    show your entire code

    break your project down into single steps:

    • track the speed of movement in the viewers body - can you do this?

    • forming sharp jagged terrain when the viewer moves quickly and smoothed and less harsh gradients when the viewer moves subtly

    • capture the direction of the head - can you do this?

    • to allow the viewer to virtually navigate through the landscape depending on where they face there head.

    where are you stuck exactly?

  • That was a nice example ! erm I don't really know whether kinect can track the speed of something, can it??

    I'm stuck at the point of transforming a procedrual landscape that is fixed to a landscape which is interactive enabling the user to explore using a mouse etc, btw thank you for your quick responses

  • ps sorry for duplicating i was just trying to increase its presence

  • post your code

  • This is my current stage, the code is based on shiffman's //Procdeurally generated sharp mapped terrain

    int cols, rows; int scl = 20; int w = 3000; int h = 1600;

    float flying = 0;

    float[][] terrain;

    void setup() { size(1700, 1000, P3D); cols = w / scl; rows = h/ scl; terrain = new float[cols][rows]; }

    void draw() {

    flying -= 0.035;

    float yoff = flying; for (int y = 0; y < rows; y++) { float xoff = 0; for (int x = 0; x < cols; x++) { terrain[x][y] = map(noise(xoff, yoff), 0, 1, -250, 300); xoff += 0.20; } yoff += 0.1; }

    background(100); stroke(200); noFill();

    translate(width/2, height/2+130); rotateX(PI/3); translate(-w/2, -h/2); for (int y = 0; y < rows-1; y++) { beginShape(TRIANGLE_STRIP); for (int x = 0; x < cols; x++) { vertex(xscl, yscl, terrain[x][y]);

      vertex(x*scl, (y+1)*scl, terrain[x][y+1]);
    
    }
    endShape();
    

    } }

    How would I atleast implement a mouse function to enable to user to navigate in the terrain or simply move in a particular direction?

    Thank you

  • edited May 2017

    i don't know how to post the code so it is in the proper format

  • edited May 2017
    //Procdeurally generated sharp mapped terrain
    
    int cols, rows;
    int scl = 20;
    int w = 3000;
    int h = 1600;
    
    float flying = 0;
    
    float[][] terrain;
    
    void setup() {
      size(1700, 1000, P3D);
      cols = w / scl;
      rows = h/ scl;
      terrain = new float[cols][rows];
    }
    
    
    void draw() {
    
      flying -= 0.035;
    
      float yoff = flying;
      for (int y = 0; y < rows; y++) {
        float xoff = 0;
        for (int x = 0; x < cols; x++) {
          terrain[x][y] = map(noise(xoff, yoff), 0, 1, -250, 300);
          xoff += 0.20;
        }
        yoff += 0.1;
      }
    
    
    
      background(100);
      stroke(200);
      noFill();
    
      translate(width/2, height/2+130);
      rotateX(PI/3);
      translate(-w/2, -h/2);
      for (int y = 0; y < rows-1; y++) {
        beginShape(TRIANGLE_STRIP);
        for (int x = 0; x < cols; x++) {
          vertex(x*scl, y*scl, terrain[x][y]);
    
          vertex(x*scl, (y+1)*scl, terrain[x][y+1]);
        }
        endShape();
      }
    }
    
  • and of course I will credit whoever helps

  • We have different views on what "is based on" entails. What you have there is basically a direct copy of https://github.com/CodingTrain/Rainbow-Code/blob/master/CodingChallenges/CC_11_PerlinNoiseTerrain/CC_11_PerlinNoiseTerrain.pde . What have you done to make it your own? Change up a few values here and there? Swap the colors? It's not enough. If you want help, the first step is to TRY DOING IT YOURSELF.

    Since your Kinect isn't physically available yet, don't fret too much about tracking head movement or movement speeds. Instead, get some keyboard input working for spoofing head movement, and maybe use the mouse pointer's speed as spoofed input for movement speed. Can you then have that input affect your terrain?

    You may want to move away from randomly generating heights with noise(), since you'll want to be saving the generated heights in a data structure anyway. Do you know enough OOP to be able to make an array (2D) of Cells?

  • edited May 2017

    Hell, I'll throw this in for nothing:

    float[] ys = new float[61];
    
    void setup(){
      size(600,400);
    
      stroke(255);
    }
    
    void draw(){
      background(0);
      translate(0,height/2);
      for(int i = 0; i < ys.length-1; i++){
        line(10*i,ys[i],10*(i+1),ys[i+1]);
        ys[i]*=.99;
      }
      ys[ys.length-1]*=.99;
    }
    
    void mouseMoved(){
      int x = mouseX/10;
      float dy = mouseY - pmouseY;
      ys[x]+=dy;
      if(x>1){ ys[x-1]+=dy/2; }
      if(x<ys.length-1){ ys[x+1]+=dy/2; }
    }
    

    Can you make that generate your terrain?

  • edited June 2017

    Thank you for the help, that sketch is impressive!

  • could you please add notes to the code to help me understand it

  • edited May 2017
    I want a list of 61 numbers.
    
    Do these things first, and only once:
      Realize that the sketch I want is 600 pixels wide and 400 pixels tall.
    
      When you draw any lines, they should be white.
    That's the end of those things.
    
    Then do this over and over:
      Paint the whole sketch black.
      Move the position of (0,0) so that it is centered vertically on the left side of the sketch.
      Do these things almost as many times as that list of numbers I had long is:
        Draw a line between a couple of points. The points depend on what time we've done this, and what numbers you're remembering in that list of numbers.
        Change one of the numbers (the next one in the list, yeah?) so it is only 99% of its current value.
      That's all of those things to do so many times.
      Change the last number in the list too, so it's 99% of what it was.
    That's all you have to do over and over.
    
    If the mouse pointer moves, do these things:
      Map the mouse's X position from it's normal range of 0 to width to the range 0 to 60. Remember what that number is.
      Work out how much the mouse moved up or down. Remember this too.
      Change a number in the list of numbers (the one based on the mouse's X position thing you just remembered) so it's changed by how much the mouse moved up or down.
      If there's a number before that one in the list, change it by half that amount too.
      If there's also a number after that one in the list, change it by half that amount too.
    That's all you have to do when the mouse moves.
    
  • Many thanks! The mouseMoved if statements confused me

  • I've gt the kinect up and running and I have played around with some of shiffman's examples, I watched and re watched his tutorials on OOP, arrays etc but i haven't grasped it enough to the extend that I can execute the original idea, if any1 could help get me started with some code it would be much appreciated only have a couple of days before the handin and i'm starting to fret

  • edited May 2017

    Ah yes, you've had five days since we last saw you.

    Did you managed to get the mouse input to control terrain spikes yet?

    How about keyboard input to control the terrain direction?

    Do you have the Kinect tracking silhouettes of people? Have you worked out how to transform that 3D data set into a 1D up/down change data? I would use the number of pixels that belong to silhouettes in a given column (and how that amount changes).

    Do you have a camera working yet? Or some other means of determining a left-right movement? If you've got an image from the Kinect too, great! Have you tried using the openCV library to do face detection (and thus determine which way the majority of faces are facing)?

    As an aside: I have smashed the code on this page together already. The mouse input makes terrain spikes (got that working five days ago). The keyboard input controls terrain direction changes (got this working in about ten minutes, just now). I'm not just posting teh codez because I want to see where you've gotten over the last five days.

  • edited June 2017

    Yes I have successfully set up kinect and got depth images, no I don't even know what 1D up/down change data means. I have been using openCV however I wouldn't know how to write the code to execute face direction.

    In the past five days I've re-watched the majority of shiffmans tutorials, merely creating basic sketches that don't hold much relevance, installed and set up kinect and employed shiffmans examples, in the meantime to take a break from my confusion I have just been catching up on book work and stressing out.

    Thank you for taking your time

  • @bude
    I understand you want to make use of the kinect but id be surprised if it's required for your final project to be interactive.
    Showing and explaining your terrain sketch might be more than good enough?

  • It's not a requirement in terms of it as necessary on the spec however I made that my original objective and I essentially am required to for fill the proposal. The sketch listed above is a copy of someone elses, I haven't even achieved this. Would you be able help? P.S like your audio visualization is that using minim?

  • oh, I did not read the whole thread before I answered, I can't do this for you but have you tried replacing the noise in that sketch with the mouse controlled 2D terrain that TfGuy44 posted, as you don't have all the time in the world and not everyone has a kinect but everyone has a mouse I think you would be better off doing that than a kinect, and doing the kinect thing when you got more time and experience

  • I've tried multiple times but I haven't had any success, thanks for the advice, no one would need a kinect it is for an exhibition piece, yeah think it was all just a bit unfeasible, thank you

  • edited May 2017

    Look, you need to break your project down into steps.

    When you are done, this will be the process:

    1) The kinect generates a 2D data set of depth data. You should get this directly from your kinect.

    2) The depth data is processed and turned into silhouette data, which is also 2D. Each point in the 2D data will either be detected as a person or not. People are detected because they will be significantly closer at some point when this data is compared to a background data set (one that has no people in it). You might have to write this, if there isn't already an example that does it.

    3) The 2D data is sampled in columns, counting up the points in each column that are detected as person. This results in a 1D array of values. You may need to write this too.

    4) That 1D array of values is then compared to the previous array of values, giving a difference. The difference data is also a 1D array of values.

    What you need to have done so far to get a 1D array of vales is A LOT OF WORK. That is why I have written you a sketch that allows you to generate a 1D array of values simply using the mouse.

    5) The difference data (or my mouse data) is then used to draw one row of terrain. This is essentially done for you in my sketch.

    6) The terrain sketch you have is currently using 2D data from noise(). You need to edit it so that the first row of data is the new 1D data from somewhere (be it the mouse input or steps 1-4). The remaining terrain then needs to be the previous values, so you will need to move the rest of the rows down.

    7) You then need to get a value that determines which direction the terrain should appear to be moving in. I have suggested that, for now, you get this from the keyboard, because that is easy. But you could also get it from detecting faces from a camera image and seeing which way the majority of the faces are pointing - there are examples for the open CV library that do this.

    8) based on the value from 7, move the terrain data in a way that makes it look like it is going in different directions.


    You say you have tried multiple times, but YOU HAVEN'T SHOWN US ANY OF THOSE ATTEMPTS. We have heard NOTHING from you for FIVE DAYS. There is NOTHING that is infeasible here. You could get this up and running in under an hour if you UNDERSTAND how it needs to work.

  • I'm currently going through the tutorials one last time tonight and I will have a go at tackling this. This is shiffman's code for enabling the kinect to create a silhouette, does it give the 1D array of values? p.s thank you for being so informative.

  • import org.openkinect.processing.*;
    
    Kinect kinect;
    
    void setup(){
      size(600,400);
      kinect = new Kinect (this);
      kinect.initDepth();
    }
    
    void draw(){
      background(255);
    
      PImage img = kinect. getDepthImage ();
      image ( img ,0,0);
      int skip = 20;
      for (int x= 0; x < img.width; x++) {
      for (int y= 0; y <img.height; y++) {
        int index = x + y * img.width;
        float b = brightness (img.pixels [index]);
        fill (b);
        rect (x,y,skip ,skip) ;
        }
     }
    }
    
  • edited May 2017

    does it give the 1D array of values?

    Yes. It loads the depth image into an image object -- a picture.

    ...like this:

    PImage img = kinect.getDepthImage ();
    

    It then loops through the rows and columns of that picture, one pixel at a time:

    for (int x= 0; x < img.width; x++) {
      for (int y= 0; y <img.height; y++) {
    

    Any PImage contains a 1D array of pixels -- PImage.pixels[].

    The code uses the x and why to look up the 1D index of a specific pixel in the depth image:

    int index = x + y * img.width;
    float b = brightness (img.pixels[index]);
    

    ...and find the brightness:

    This is then used to set the drawing fill color, and draw a 20x20 rectangle:

    ...like this:

    fill (b);
    rect (x,y,skip ,skip) ;
    
  • thank you for breaking it down

  • edited June 2017
    `int columns, rows;
    int scale = 14;
    int wid = 3800;
    int hei = 100;
    float inter, drift;
    float[][] i;
    float ioff;
    void setup() {
      size(1000, 600, P3D);
      columns = wid/scale;
      rows = wid/scale;
      colorMode(RGB);
      i = new float[columns][rows];
    }
    void draw() {
      drift = map(sin(ioff), -1, 25, -400, width) * 0.06;
      float yoff = drift;
           for (int x = 0; x < columns; x ++) {
          i[x][y] = map(noise(xoff, yoff), 0, 1, -120, 180);
          xoff += 0.1;
        }
        yoff += 0.08;
      }
      ioff += 0.01;
      background( 130);
      strokeWeight(1);
      translate(width/2, height/2);
      rotateX(PI/3.6);
      rotate(mouseX * 0.008);
      inter += .9;
      translate(-wid/2, -wid/2);
      for (int y = 2; y < rows - 1; y++) {
        beginShape(LINES);
        for (int x = 2; x < columns; x++) {
          stroke(200);
          vertex(x * scale, y * scale, i[x][y]);
          vertex(x * scale, (y + 1) * scale, i[x][y + 1]);
        }
        endShape();
      }
    }`
    

    This is as far as i've got using the still based on shiffman's however I've managed to implement the mouseX variable to enable navigation, any added help will be appreciated

  • edited June 2017

    That's not really going to help you, as it's still generating the terrain height from the 2D data that noise() provides. Plus it just oscillates back and forth over the same area, so i have no idea how you are going to incorporate your processed Kinect's depth data into that.

    Here is as far as I took it:

    int direction = 0;
    
    int cols, rows;
    int scl = 20;
    int w = 3000;
    int h = 1600;
    
    float flying = 0;
    
    float[][] terrain;
    
    void setup() {
      size(1700, 1000, P3D);
      cols = w / scl;
      rows = h/ scl;
      terrain = new float[cols][rows];
      ys = new float[cols];
    }
    
    
    void draw() {
    
      for (int i = 0; i < ys.length-1; i++) {
        //line(10*i, ys[i], 10*(i+1), ys[i+1]);
        ys[i]*=.9;
      }
      ys[ys.length-1]*=.9;
    
      flying -= 0.035;
    
      float yoff = flying;
      for (int y = rows-1; y >=0; y--) {
        float xoff = 0;
        for (int x = cols-1; x >= 0; x--) {
          if ( y > 0 ) { 
            terrain[x][y] = terrain[constrain(x+direction,0,cols-1)][y-1];
          } else { 
            terrain[x][y] = ys[x] + map(noise(xoff, yoff), 0, 1, -100, 100);
          }
          xoff += 0.20;
        }
        yoff += 0.1;
      }
    
      background(100);
      stroke(200);
      noFill();
    
      translate(width/2, height/2+130);
      rotateX(PI/3);
      translate(-w/2, -h/2);
      for (int y = 0; y < rows-1; y++) {
        beginShape(TRIANGLE_STRIP);
        for (int x = 0; x < cols; x++) {
          vertex(x*scl, y*scl, terrain[x][y]);
    
          vertex(x*scl, (y+1)*scl, terrain[x][y+1]);
        }
        endShape();
      }
    }
    
    float[] ys;
    
    void mouseMoved() {
      int px = int(map(mouseX,0,width,0,cols));
      float dy = 2*(pmouseY - mouseY);
      if (px>0&&px<ys.length) {
        ys[px]+=dy;
      }
      for(int i = 1; i<20;i++){
        if (px-i>=0) { 
          ys[px-i]+=dy/(i+1);
        }
        if (px+i<ys.length) { 
          ys[px+i]+=dy/(i+1);
        }
      }
    }
    
    void keyPressed(){
      if(keyCode == LEFT ){
        direction = -1;
      } else if (keyCode == RIGHT ){
        direction = 1;
      } else if (keyCode == UP ){
        direction = 0;
      }
    }
    

    Notice that the mouse Pointer still maps changes in mouseY to the ys array, and the 0'th row of terrain is generated by the data in the ys array. The remaining terrain comes from the previous iteration. I kept the noise in to keep it from being boring.

    The keyPressed() function also lets you set a direction for the terrain to move in, using the UP, LEFT, or RIGHT arrow keys. This direction is used to offset the column that the terrain is copied from, causing it to appear to shift left or right (or not).

    As I do not have a Kinect or a camera, I am both unable and unwilling to help any more. If you are going to try to add more to this, sample your depth data in columns and get it into the ys array instead of using the mouse, and find an openCV example for face tracking to determine which way faces are facing, and use that instead of keyPressed().

    I've been sitting on this code for 6 days. It didn't really take long. I'm not doing anything more with it.

  • edited June 2017

    Thank you for sharing how you incorporated the keyPressed, okay I will spend my last day today trying to implement the kinect, it would have taken me ages to figure it out your advice and help has been extremely helpful, thank you for all your time.

Sign In or Register to comment.