Real time typography with kinect or webcam

edited April 2017 in Kinect

Hey everyone. I'm really new to processing so excuse me if this question sounds vague. Basically - I am trying to create a real-time typographic piece similar to the one below using either a Kinect camera or a webcam. I just don't know where to start. The other example feels similar to Shiffman's pointcloud example. I have attempted to implement text into his example but unfortunately I don't really know what I'm doing!

Could someone please help me out!

Thank you

Tagged:

Answers

  • Start with something simple. Here's something simple:

    int[][] vals = new int[10][10];
    
    void setup() {
      size(400, 400);
      textSize(20);
      textAlign(CENTER);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {
          vals[y][x] = int(random(10));
        }
      }
    }
    
    void draw() {
      background(0);
      fill(255);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {
          text(vals[y][x], 20+40*x, 28+40*y);
        }
      }
    }
    
  • Make small changes. Like, can I do this in 3D? Yes:

    int[][] vals = new int[10][10];
    
    void setup() {
      size(400, 400,P3D);
      textSize(20);
      textAlign(CENTER);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {
          vals[y][x] = int(random(10));
        }
      }
    }
    
    void draw() {
      background(0);
      fill(255);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {
          text(vals[y][x], 20+40*x, 28+40*y, -40*x);
        }
      }
    }
    
  • More small changes. Make Z-position depend on the value. Make the values depend on something not random. Yep:

    int[][] vals = new int[10][10];
    
    void setup() {
      size(400, 400,P3D);
      textSize(20);
      textAlign(CENTER);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {
          vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
        }
      }
    }
    
    void draw() {
      background(0);
      fill(255);
      for (int y = 0; y < 10; y++ ) {
        for (int x = 0; x < 10; x++ ) {      
          text(vals[y][x], 20+40*x, 28+40*y, -40*vals[y][x]);
        }
      }
    }
    
  • Make more small changes. Scale it down. Ramp it up. Add some way to view it in 3D. Nice:

    int[][] vals = new int[100][100];
    
    void setup() {
      size(400, 400,P3D);
      textSize(8);
      textAlign(CENTER);
      for (int y = 0; y < 100; y++ ) {
        for (int x = 0; x < 100; x++ ) {
          vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
        }
      }
    }
    
    void draw() {
      background(0);
      fill(255);
      translate(200,200,0);
      rotateX(map(mouseY,0,height,-PI,PI));
      rotateY(map(mouseX,0,width,-PI,PI));
      translate(-200,-200,0);
      for (int y = 0; y < 100; y++ ) {
        for (int x = 0; x < 100; x++ ) {      
          text(vals[y][x], 20+4*x, 28+4*y, -40*vals[y][x]);
        }
      }
    }
    
  • Answer ✓

    Next, I would get a video working. Sample the values from the video - perhaps the brightness at a set of points. Or hook up a Kinect and get distance data, and sample that. Try it yourself. Post your attempt and real questions for more help.

  • TfGuy44 - thank you so so much for responding. You have been so helpful. The problem is - I just don't know how to implement video into the code. I wouldn't know where to start! I have a Kinect V1 and am using Processing 3. I imagine I would need to collect the data from depth points of the video and assign them certain depths to numbers/ letters? I'm sorry - I'm so new to this but really would love to learn! Thanks for your help so far. Oliver

  • edited April 2017

    Hey! So I think I have it working! I just need to improve it slightly! Thank you so much for your suggestions previously.

    So here is the code that I am using now.....

    import org.openkinect.processing.*;
    
    Kinect kinect;
    
    void setup() {
      size(640, 480, P3D);
      kinect = new Kinect(this)
      ;
      kinect.initDepth();
      textSize(8);
      textAlign(CENTER);
    }
    
    void draw() {
      background(0);
    
    
      PImage img = kinect.getDepthImage();
      //image(img, 0, 0);
    
      int skip = 20;
      for (int x = 0; x < img.width; x+=skip) {
        for (int y = 0; y < img.height; y+=skip) {
    
          int index = x + y * img.width;
          float b = brightness(img.pixels[index]);
          float z = map(b, 0, 255, 250, -250);
          fill(255-b);
          pushMatrix();
          translate(x, y, z);
          text(0,0,skip/2,skip/2);
          popMatrix();
    
        }
      }
    }
    

    I know it's probably not perfect but it works reasonably well. I can't work out how to add a screen shot in here to shot you how it works but.. currently the only numbers showing up are 0's. I think I need to adjust and play with the depth as well to get a better result. How much I change the text that is showing up to be random numbers/ letters (instead of just 0's) depending on something (depth, brightness, etc)?

    Thanks!

  • text(0,0,skip/2,skip/2);

    Well, that's why you only get 0's! (The first parameter to text() is the text to display!)

    Make that parameter depend on the value - or be random. Or change the fill() color randomly.

  • Ohhhhhh! what is the language to use for random? Or is there a page you could point me to that I could study language/ parameters + their functions etc. Thanks

Sign In or Register to comment.