Control a 3d human mesh with kinect

General Discussion Other

noncom

Control a 3d human mesh with kinect

in General Discussion • Other • 1 year ago

Hello!

I am trying to animate a 3d character with kinect. I have been successfull in getting all the joints data ok and I can make a stick man to reproduce my moves. The problem is that I cannot get a 3d model to do the same. I know, it is much harder. I am using objimport lib for importing OBJ files and the OpenGL renderer. Can anyone suggest anything about this? So far, I have been unable to find any relevant material on this topic. I would appreciate anything - suggestions, libraries, links...

Replies(6)

mrweix

Re: Control a 3d human mesh with kinect

1 year ago

Did you manage to get this working? I want to do something similar, both with a stick man and a 3D character. Not necessarily Kinect, but to show movements of a stickman, in a 3D model and am also quite clueless how to manage this.
Any input highly appreciated!

Thanks!

noncom

Re: Control a 3d human mesh with kinect

1 year ago

Well, yes the stickman was ok from the very beginning.. I was using kinect. Kinect makes it plain simple - you just get the exact coordinates of all the joints and that's it. Drawing a figure with them is not a big trick. However, I was unable to do the same with a 3d mesh. Mostly because Processing lacks importers for 3D objects, and I do not even speak for importers of characters with bones. I am myself not a math or file format guru, so I could not implement them for myself. Since then I did not return to this task. However, now I am using JMonkeyEngine3 for 3D (I still use Processing for 2D), and I think that it could manage that task since it has the necessary importer, but I did not try.

If you are interested, I can share my code... or better for you - I can help you if you have any questions, because sharing code is not so exciting... except for when you are at a deadline.. besides, my code itself is such a mess that you will spend much more time and nerves trying to even read it, not even mentioning about making it work like you want it to)))... anyway.. I will be glad to help no matter which way you choose...

mrweix

Re: Control a 3d human mesh with kinect

1 year ago

Thanks for the answer and your offer to share the code!

Doesn't Processing have the OBJLoader http://code.google.com/p/saitoobjloader/
With that you should be able to load 3D .obj files, for example from Maya, 3ds or such into Processing. Its pretty straightforward.

But then I am stuck at that point: I can change the objects in Maya to resemble a movement, but I don't know how to do that inside Processing.
As in: I have a stickman movement in Processing. How to get the corresponding 3D movement in Processing.

Cheers!

noncom

Re: Control a 3d human mesh with kinect

1 year ago

Yep if you need the code just tell me i will email it to you (an Eclipse project).

As for the OBJ loader - the OBJ files do NEVER contain any animation. They only contain one or several meshes with materials and textures (in separate files). Try opening an OBJ file in Notepad++... The OBJ format itself, being very simple, is one of the furthest from being best for production. OBJ files typically take several times more space than any binary format (say, if FBX takes 5 mb, an OBJ may take 50 mb), are very slow to load and do not contain any advanced information on objects. I have tried two OBJ loaders for Processing that I have found including the one you have mentioned.

The best result I was able to get is 14 models of 800mb of disk space in total, loading for about 20 seconds in my application and with no animation. With textures applied to the objects, the application memory footprint was almost going through the roof. I got crashes if I tried loading more models since 32 bit Java is limited to somewaht 1500Mb or so. And all this - without effects or shaders or something else. This is not satisfactory as I work for a company and such a solution is totally unsuitable for production. I have found two engines to be possible replacements for processing in 3D - Bonzai engine (which has many loaders) and the JME3 engine which I am exploring right now. Processing remains my best choise for any 2D graphics, be it a plain 2D application, or drawing to a texture in a 3D application.

BTW, I advice you to read about 3D file formats to get a general impression of the realm.

mrweix

Re: Control a 3d human mesh with kinect

1 year ago

Thanks so much. I am not sure if it will help me, but now I am curious to see it ;) I sent you a private message with my email address. Does one need kinect to run the code? Thanks a bunch!

noncom

Re: Control a 3d human mesh with kinect

1 year ago

Yes, I have got the email, I will send the project very soon. Answering your question: yes, you need a kinect. Beside that, you also need to install the kinect drivers from OpenNI website. Be sure to instal dev versions. There are 3-4 installations that you have to do. Actually, motion capture without kinect is quite hard. Kinect was specially made for that type of work and it handels it relatively easy due to the technology that it uses. Tracking joints on a simple webcam is not so easy at all.

Top Reply