Real time typography with kinect or webcam

olieast

Hey everyone. I'm really new to processing so excuse me if this question sounds vague. Basically - I am trying to create a real-time typographic piece similar to the one below using either a Kinect camera or a webcam. I just don't know where to start. The other example feels similar to Shiffman's pointcloud example. I have attempted to implement text into his example but unfortunately I don't really know what I'm doing!

Could someone please help me out!

Thank you

TfGuy44

Start with something simple. Here's something simple:

int[][] vals = new int[10][10];

void setup() {
  size(400, 400);
  textSize(20);
  textAlign(CENTER);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {
      vals[y][x] = int(random(10));
    }
  }
}

void draw() {
  background(0);
  fill(255);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {
      text(vals[y][x], 20+40*x, 28+40*y);
    }
  }
}

TfGuy44

Make small changes. Like, can I do this in 3D? Yes:

int[][] vals = new int[10][10];

void setup() {
  size(400, 400,P3D);
  textSize(20);
  textAlign(CENTER);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {
      vals[y][x] = int(random(10));
    }
  }
}

void draw() {
  background(0);
  fill(255);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {
      text(vals[y][x], 20+40*x, 28+40*y, -40*x);
    }
  }
}

TfGuy44

More small changes. Make Z-position depend on the value. Make the values depend on something not random. Yep:

int[][] vals = new int[10][10];

void setup() {
  size(400, 400,P3D);
  textSize(20);
  textAlign(CENTER);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {
      vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
    }
  }
}

void draw() {
  background(0);
  fill(255);
  for (int y = 0; y < 10; y++ ) {
    for (int x = 0; x < 10; x++ ) {      
      text(vals[y][x], 20+40*x, 28+40*y, -40*vals[y][x]);
    }
  }
}

TfGuy44

Make more small changes. Scale it down. Ramp it up. Add some way to view it in 3D. Nice:

int[][] vals = new int[100][100];

void setup() {
  size(400, 400,P3D);
  textSize(8);
  textAlign(CENTER);
  for (int y = 0; y < 100; y++ ) {
    for (int x = 0; x < 100; x++ ) {
      vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
    }
  }
}

void draw() {
  background(0);
  fill(255);
  translate(200,200,0);
  rotateX(map(mouseY,0,height,-PI,PI));
  rotateY(map(mouseX,0,width,-PI,PI));
  translate(-200,-200,0);
  for (int y = 0; y < 100; y++ ) {
    for (int x = 0; x < 100; x++ ) {      
      text(vals[y][x], 20+4*x, 28+4*y, -40*vals[y][x]);
    }
  }
}

TfGuy44

Next, I would get a video working. Sample the values from the video - perhaps the brightness at a set of points. Or hook up a Kinect and get distance data, and sample that. Try it yourself. Post your attempt and real questions for more help.

olieast

TfGuy44 - thank you so so much for responding. You have been so helpful. The problem is - I just don't know how to implement video into the code. I wouldn't know where to start! I have a Kinect V1 and am using Processing 3. I imagine I would need to collect the data from depth points of the video and assign them certain depths to numbers/ letters? I'm sorry - I'm so new to this but really would love to learn! Thanks for your help so far. Oliver

olieast

Hey! So I think I have it working! I just need to improve it slightly! Thank you so much for your suggestions previously.

So here is the code that I am using now.....

import org.openkinect.processing.*;

Kinect kinect;

void setup() {
  size(640, 480, P3D);
  kinect = new Kinect(this)
  ;
  kinect.initDepth();
  textSize(8);
  textAlign(CENTER);
}

void draw() {
  background(0);


  PImage img = kinect.getDepthImage();
  //image(img, 0, 0);

  int skip = 20;
  for (int x = 0; x < img.width; x+=skip) {
    for (int y = 0; y < img.height; y+=skip) {

      int index = x + y * img.width;
      float b = brightness(img.pixels[index]);
      float z = map(b, 0, 255, 250, -250);
      fill(255-b);
      pushMatrix();
      translate(x, y, z);
      text(0,0,skip/2,skip/2);
      popMatrix();

    }
  }
}

I know it's probably not perfect but it works reasonably well. I can't work out how to add a screen shot in here to shot you how it works but.. currently the only numbers showing up are 0's. I think I need to adjust and play with the depth as well to get a better result. How much I change the text that is showing up to be random numbers/ letters (instead of just 0's) depending on something (depth, brightness, etc)?

Thanks!

TfGuy44

text(0,0,skip/2,skip/2);

Well, that's why you only get 0's! (The first parameter to text() is the text to display!)

Make that parameter depend on the value - or be random. Or change the fill() color randomly.

olieast

Ohhhhhh! what is the language to use for random? Or is there a page you could point me to that I could study language/ parameters + their functions etc. Thanks

kfrajer

https://processing.org/reference/

https://processing.org/reference/random_.html

Kf