Howdy, Stranger!

We are about to switch to a new forum software. Until then we have removed the registration on this forum.

Real time typography with kinect or webcam

edited April 2017 in Kinect

Hey everyone. I'm really new to processing so excuse me if this question sounds vague. Basically - I am trying to create a real-time typographic piece similar to the one below using either a Kinect camera or a webcam. I just don't know where to start. The other example feels similar to Shiffman's pointcloud example. I have attempted to implement text into his example but unfortunately I don't really know what I'm doing!

Thank you

Tagged:

• Start with something simple. Here's something simple:

int[][] vals = new int;

void setup() {
size(400, 400);
textSize(20);
textAlign(CENTER);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
vals[y][x] = int(random(10));
}
}
}

void draw() {
background(0);
fill(255);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
text(vals[y][x], 20+40*x, 28+40*y);
}
}
}

• Make small changes. Like, can I do this in 3D? Yes:

int[][] vals = new int;

void setup() {
size(400, 400,P3D);
textSize(20);
textAlign(CENTER);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
vals[y][x] = int(random(10));
}
}
}

void draw() {
background(0);
fill(255);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
text(vals[y][x], 20+40*x, 28+40*y, -40*x);
}
}
}

• More small changes. Make Z-position depend on the value. Make the values depend on something not random. Yep:

int[][] vals = new int;

void setup() {
size(400, 400,P3D);
textSize(20);
textAlign(CENTER);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
}
}
}

void draw() {
background(0);
fill(255);
for (int y = 0; y < 10; y++ ) {
for (int x = 0; x < 10; x++ ) {
text(vals[y][x], 20+40*x, 28+40*y, -40*vals[y][x]);
}
}
}

• Make more small changes. Scale it down. Ramp it up. Add some way to view it in 3D. Nice:

int[][] vals = new int;

void setup() {
size(400, 400,P3D);
textSize(8);
textAlign(CENTER);
for (int y = 0; y < 100; y++ ) {
for (int x = 0; x < 100; x++ ) {
vals[y][x] = int(10 * noise(x/10.0,y/10.0)); //int(random(10));
}
}
}

void draw() {
background(0);
fill(255);
translate(200,200,0);
rotateX(map(mouseY,0,height,-PI,PI));
rotateY(map(mouseX,0,width,-PI,PI));
translate(-200,-200,0);
for (int y = 0; y < 100; y++ ) {
for (int x = 0; x < 100; x++ ) {
text(vals[y][x], 20+4*x, 28+4*y, -40*vals[y][x]);
}
}
}

• Next, I would get a video working. Sample the values from the video - perhaps the brightness at a set of points. Or hook up a Kinect and get distance data, and sample that. Try it yourself. Post your attempt and real questions for more help.

• TfGuy44 - thank you so so much for responding. You have been so helpful. The problem is - I just don't know how to implement video into the code. I wouldn't know where to start! I have a Kinect V1 and am using Processing 3. I imagine I would need to collect the data from depth points of the video and assign them certain depths to numbers/ letters? I'm sorry - I'm so new to this but really would love to learn! Thanks for your help so far. Oliver

• Hey! So I think I have it working! I just need to improve it slightly! Thank you so much for your suggestions previously.

So here is the code that I am using now.....

import org.openkinect.processing.*;

Kinect kinect;

void setup() {
size(640, 480, P3D);
kinect = new Kinect(this)
;
kinect.initDepth();
textSize(8);
textAlign(CENTER);
}

void draw() {
background(0);

PImage img = kinect.getDepthImage();
//image(img, 0, 0);

int skip = 20;
for (int x = 0; x < img.width; x+=skip) {
for (int y = 0; y < img.height; y+=skip) {

int index = x + y * img.width;
float b = brightness(img.pixels[index]);
float z = map(b, 0, 255, 250, -250);
fill(255-b);
pushMatrix();
translate(x, y, z);
text(0,0,skip/2,skip/2);
popMatrix();

}
}
}

I know it's probably not perfect but it works reasonably well. I can't work out how to add a screen shot in here to shot you how it works but.. currently the only numbers showing up are 0's. I think I need to adjust and play with the depth as well to get a better result. How much I change the text that is showing up to be random numbers/ letters (instead of just 0's) depending on something (depth, brightness, etc)?

Thanks!

• text(0,0,skip/2,skip/2);

Well, that's why you only get 0's! (The first parameter to text() is the text to display!)

Make that parameter depend on the value - or be random. Or change the fill() color randomly.

• Ohhhhhh! what is the language to use for random? Or is there a page you could point me to that I could study language/ parameters + their functions etc. Thanks