Creating a 3d slice from a image silhouette?

General Discussion Other

johngomm

Creating a 3d slice from a image silhouette?

in General Discussion • Other • 2 years ago

Hi All,

I want to take a silhouette image (black hand on white background) and make a 3d "slice" that has thickness 1 and the outline of the hand. Is there a simple way to extract the black pixels, find the outline and then give it some thickness?

Ideally, I want to do this with a video of a hand moving and build up a stack of slices into a column, so I want to keep the vertex count manageable. I did think about making a 1x1x1 box for every black pixel, but the vertex count gets huge very quickly (8 for every black pixel rather than finding an outline).

Alternatively, creating a 3D mesh that connects the first outline with the second outline with the third outline directly into a water-tight, 3D-printable mesh would be the holy grail for me, but I think this is too hard a problem.

If anyone knows of some similar projects or useful libraries, please let me know.

Thanks, DrJohn

Replies(5)

tfguy44

Re: Creating a 3d slice from a image silhouette?

2 years ago

The hard step here is taking the 3D image and working out where the difference between black and white pixels is. Once you can do that, you can get pre-compute an outline of the hand - for each frame - that you can draw easily in 3D as a set of vertexes.

That alone is a lot of work.

Do you really really need a truly 3D model of this?

Couldn't you just modify the pixels[] array so that values near white are totally transparent when they are rendered (on a non-black background, of course)?

johngomm

Re: Creating a 3d slice from a image silhouette?

2 years ago

I already have a sketch that pulls in the video and thresholds the pixels to black and white, so I'm dealing with a 2D image to analyze. I think that I'm looking for a bitmap-to-polygon type example to learn from/library to use. Something like an edge-detection - hmm that sounds like a promising search term...

Yes I truly need a 3D model if I'm going to real-world 3D fabricate the result (my entire goal). I have sketches that make some 3D meshes from logo-type images and saves them as .stl files and have successfully manufactured them on a MakerBot, so that part I am comfortable with if I can get the polygon conversion (or whatever) figured out. I'll post that code over at Thingiverse.com and link to it from here when I tidy it up for public consumption.

Thanks for your thoughts,

John

tfguy44

Re: Creating a 3d slice from a image silhouette?

2 years ago

Well, if you've already threshold-ed it so that you only have black and white pixels, that already makes it a step easier.
What you could do is make a 2D iso-surface. Imagine you're looking really, really closely at a group of pixels, so that they appear like a grid of squares, like in the left image here:

Now imagine that each pixel is a node, and there are graph lines connecting each pixel to it's immediate neighbors, as on the right.

Color these graph lines green if the are between two pixel nodes of the same color, and red otherwise.
Now the center points of the red graph lines can be connected in each "square" formed by for adjoining pixels.

This will give you the points you need to draw the outline of the hand for one frame. Repeat this for all the frames, and then add a Z-coordinate based on the frame number.

Five things to note:

1) There is a square that has an ambiguous case - when all four of its graph lines are red, as in the bottom center square. In this case, I suggest you join the corners as I have above, as it's more likely that the black squares are joined, not the white ones (black squares being the solid hand). That's up to you, of course.
2) This is a lot of work, both terms of coding and computing power. You probably will not be able to do this for live video!

3) You might not have to do this for every pixel - you might be satisfied if you sampled the image at a less fine rate.
4) Since you want to draw the black part as solid, you'll need to work out which side of the line is the black half, and draw the appropriate shape in each square too. For an all black-region, this will be a square. For one black point, a triangle. For three, an odd pentagram. For two, either a rectangle or an odd hexagram.

5) You don't have to use the midpoint of each graph line - you can use a point a fixed distance away from the black side (probably a very small distance). This will give you a very tight edge.

Thoughts/questions?

johngomm

Re: Re: Creating a 3d slice from a image silhouette?

2 years ago

That looks really useful, conceptually, thanks. Also looks like a bunch of work (unless there's a library or example sketch that already does this) that might result in an elegant solution and there might be dirtier, but easier to code solutions. I was looking at Toxiclibs voxels and wondered if creating a voxel for each black pixel would keep the overhead small, but output a watertight .stl file by definition.

I don't have time right now to whip up a sketch, but I'll give it a try later and let you know how it goes. Otherwise I might have a crack at implementing the approach you describe.

Thanks for your help

DrJohn

toxmeister

Re: Creating a 3d slice from a image silhouette?

2 years ago

Here's an old example of the voxel approach updated from my answer to a similar question on the old forum:

http://processing.org/discourse/yabb2/YaBB.pl?num=1276630718/6#6

/**
* Loads a B&W image and extrudes non-black pixels brighter than a given threshold into
* a watertight 3D mesh.
*
* @author Karsten Schmidt // LGPL2 licensed
*
* Dependencies: toxiclibscore-0020
* (or newer, available from: http://toxiclibs.org/ )
*/
import toxi.geom.*;
import toxi.geom.mesh.*;
import toxi.volume.*;
int SCALE=8;
int DEPTH=4;
int THRESH=32;
boolean FIXED_DEPTH=true;
PImage img=loadImage("hand.png");
// setup volumetric space
Vec3D worldSize = new Vec3D(img.width, img.height, DEPTH).scale(SCALE);
VolumetricSpace volume = new VolumetricHashMap(worldSize, img.width, img.height, DEPTH, 0.33);
VolumetricBrush brush = new RoundBrush(volume, SCALE);
brush.setMode(VolumetricBrush.MODE_PEAK);
// parse the image
for (int y = 0; y < img.height; y ++) {
for (int x = 0; x < img.width; x ++) {
int col=img.pixels[y * img.width + x] & 0xff;
if (col > THRESH) {
for (int z = 0, d=FIXED_DEPTH ? DEPTH : (int)(col/255.0*DEPTH); z < d; z++) {
brush.drawAtGridPos(x, y, z, 1);
}
}
}
}
// make volume watertight
volume.closeSides();
// compute mesh
WETriangleMesh mesh = new WETriangleMesh();
new HashIsoSurface(volume).computeSurfaceMesh(mesh, 0.25f);
// apply vertex smoothing
new LaplacianSmooth().filter(mesh,2);
mesh.saveAsSTL(sketchPath("test.stl"), true);
exit();

http://postspectacular.com | http://toxiclibs.org

Top Reply