Processing 1.0 - Processing Discourse - Tricks for enhancing tracking?

RickiG

Tricks for enhancing tracking?
Aug 31^st, 2009, 11:28am

Hi

I am starting a small project with a couple of friends. Basically we want to track a hacky sack live with processing.

First up is the speed of the camera, 20-30 fps is nowhere fast enough for this sort of thing. Does anyone know of cameras (within a $100 budget) with faster frame rates, that will work with Processing?

I was planning on stripping the IR filter from the camera and flood the surroundings with light in the IR spectrum to further improve the tracking.
Then I came to think of black light, does anyone know of some substance I could spray or otherwise apply to the hacky sack to make it easier to track, when lighting it with IR light?

I did stuff like this before with hands, people and faces, but nowhere near these speeds and never in lighting conditions completely out of my control. I used both JMyron and OpenCV, but if I can "pimp" the physical conditions so that I can use a simple "find the color" algorithm, i.e. the hacky sack will stand out from everything else in the frame, I would believe this to be many times faster.

I was hoping someone did something like this or just had some good advise before it start out buying stuff and coding away for days

Cheers!

Regards Ricki

st33d God Member Offline Posts: 666	Re: Tricks for enhancing tracking? Reply #1 - Aug 31^st, 2009, 11:39am I used a tower PC with a video card in it for tracking once. Got blinding framerates because it was hooked up to a camcorder and didn't have to use any funny drivers, I just got a video feed in real time.

RickiG Junior Member Offline Posts: 81	Re: Tricks for enhancing tracking? Reply #2 - Aug 31^st, 2009, 6:16pm Hi Sounds interesting:) but what is a video card? is it something other than a graphics card? If I were to buy a web cam that promised frame rates up to ≈100 fps, do you think I would ever get that using Processing? cheers:)
	Regards Ricki

subpixel

Re: Tricks for enhancing tracking?
Reply #3 - Sep 1^st, 2009, 2:55am

Check this article about the Sony PS3 Eye camera:

http://createdigitalmotion.com/2009/08/06/trick-out-your-ps3-eye-webcam-best-cam-for-vision-augmented-reality/
Quote:

It’s just US$40, and it’s your best ticket to creating your own computer vision and augmented reality projects, imagining stuff before big game console makers do. It’s the Sony PlayStation 3 Eye. Sony intended it to be used with their flagship game console. (The guy working at my local GameStop looked emotionally dejected when I told him I don’t actually own a PS3.) But thanks to community-developed, open-source drivers for all three platforms, the PS3 Eye has become a big choice for interactive applications, from tracking fingers and objects on tables to projection mapping and augmented reality.

Why choose the PS3 Eye over another webcam Because it was built for CV applications, the camera performs well in variable lighting, has rock-solid, low-latency USB performance, and is capable of high framerates (60-75 fps at normal resolution or even 125-150 fps if you can sacrifice resolution, which might be okay for tracking).

If you don’t already own a PS3 Eye, you should. Go get one — I’ll wait. Now, the hackers among you aren’t done yet. You’ll want to trick out the PS3 Eye to make it really yours, with drivers, infrared-tracking capability, and a beautiful, new custom housing.

Also, the Windows driver has apparently been updated, and the author talks of using it with Processing in a recent post...

http://createdigitalmotion.com/2009/08/27/ps3-eye-camera-drivers-updated-for-windows-fixes-performance-options-awesomeness/
Quote:

The new driver is performing beautifully. I’m using it in Processing with the aid of GSVideo. Updated in this release:

Improved driver stability (this relates to the freezing of the camera image on some machines)
Overall CPU usage was greatly reduced by optimizing the critical code. This code is now implemented using hand optimized MMX/SSE instructions.
Various fixes in PS3Eye.ax direct show component. This should improve compatibility with many programs that use this feature (i.e. Skype, MSN, Flash)
Direct show applications can now use full set of PS3Eye camera frame rate capabilities. Here is the list of supported formats: 320×240 @ 15, 30, 60, 75, 100, 125 fps and 640×480 @ 15, 30, 40, 50, 60 fps
Full XP/Vista/Windows 7 support

In fact, to me this gives the driver near-parity with the Linux version (though it’s still easier to install GStreamer on Linux), and makes it worth running in Windows if you’ve got a Boot Camp install on your Mac.

Grab the new version at the very bottom of the ongoing page; 3.0.0.0825 at press time.

Sony PS3Eye Camera DirectShow Capture Source Filter *Update* v3.0.0.0825

-spxl

Processing: subpixels.com, OpenProcessing.org | VJ portfolio: SPXL.TV

RickiG Junior Member Offline Posts: 81	Re: Tricks for enhancing tracking? Reply #4 - Sep 1^st, 2009, 1:52pm Hi subpixel! Bamm! right on:) sounds exactly like something I could use.. of to the internets to see if this is still Mac compatible! Cheers mate:)
	Regards Ricki

RickiG

Re: Tricks for enhancing tracking?
Reply #5 - Sep 2^nd, 2009, 5:14am

Hi

Just a follow up:)

I found a EyeCreate PS3 camera at my local electronics shop for $38.
Ran Macam and got 120 fps crystal video at 320x240 and 90 fps at 640x480!
http://webcam-osx.sourceforge.net/

Im having trouble getting GSCapture to find the camera, but nothing critical, it does not show up in Capture.list(). But I just unwrapped it so I'll just test some things out:)

Thanks again subpixel!

Regards Ricki

doros YaBB Newbies Offline Posts: 3	Re: Tricks for enhancing tracking? Reply #6 - Sep 7^th, 2009, 9:41pm Hey guys, I am trying to use the ps3 camera with GSCapture but I am facing the same problem. RickiG have u managed to get the GSCapture to find the camera? If yes, how? Thanks!

RickiG

Re: Tricks for enhancing tracking?
Reply #7 - Sep 9^th, 2009, 9:24am

Hi

Actually I got distracted by other issues:) but the camera now shows up using the Capture class, and I guess the GSCapture get's it's data from the same place:)

I'll test it for you tonight when I get back ( 3 hours or so ) I did get some good advise in this thread I posted: http://processing.org/discourse/yabb2/num_1251903672.html

If it works out with GSCapture let me know, if not then I'll give it another try:)

I went OpenCV instead, but I'm really interested in testing the "reception" and speed of one vs. the other.

Regards Ricki

doros

Re: Tricks for enhancing tracking?
Reply #8 - Sep 10^th, 2009, 12:56am

Hi RickiG,

I haven't actually made it work with GSCapture, but it does however work with the normal processing classes (whatever includes import processing.video.*) as well as with the JMyron library.

Funny thing is it does not work with this :

myCapture = new Capture(this, width, height, 30);

but it does work with this :

myCapture = new Capture(this, width, height, "PS3Eye Camera-WDM", 30);

Have you made it work with OpenCV? I am trying but it keep getting :

Error while starting capture : device 0

Maybe I need to specify which camera to use, like above? Any idea how to do that? Maybe a sample of ur code that works?

Thanks,

Doros

RickiG

Re: Tricks for enhancing tracking?
Reply #9 - Sep 10^th, 2009, 2:29pm

Hi

That is strange:)

Usually the camera needs to be known to the system, then:
println(Capture.list() );

would produce a list of the cameras available.
My is done like this:
video = new Capture(this, width, height, "Sony HD Eye for PS3 (SLEH 00201)", 120);

Maybe we have different models, I think mine is new, I bought it only a couple of days ago?

I was doing some research into why the PS3 cam would run on my workstation but not on my laptop. I tried synchronizing the quicktime components between the two computers, this resulted in the PS3 cam not working anywhere :/ So until I want to spend half a day getting it up and running again, I'll have to work without it. Im doing the skeleton for my classes etc. using a different $15 camera.. that works jut fine

Regards Ricki

subpixel Senior Member Offline Posts: 261	Re: Tricks for enhancing tracking? Reply #10 - Sep 22^nd, 2009, 9:10pm No problems. Don't thank me, thank the peeps who did the work that I found. Had my own issues with GSvideo on my machine - had it working (eventually!) on some day, and another day trying to run the same sketch (no changes) get back to some nightmare of the library not loading (for no given reason). sigh -spxl
	Processing: subpixels.com, OpenProcessing.org \| VJ portfolio: SPXL.TV

knutl YaBB Newbies Offline Posts: 37 Bremen, Germany	Re: Tricks for enhancing tracking? Reply #11 - Nov 14^th, 2009, 7:00am this discussion helped me a lot in using this great camera within processing !! thnx. Any ideas how to control Capture Format or framerate out of a sketch ? knutl

liudr

Re: Tricks for enhancing tracking?
Reply #12 - Nov 17^th, 2009, 8:36pm

Hey guys, has anyone tried this Sony PS3 cam with regular capture programs on PC like virtual-dub or msn online chatting? I suppose it will work but if anyone has a confirmation, I would appreciate it. I'm considering buying one but just in case it won't work on processing at the very least I get a webcam out of it. Thanks.

BTW, here is my story behind the need for a cheap camera. I'm (still) developing a physics lab with cameras. I bought a $400 industrial camera with 640*480 @60fps but this sony cam may be a better choice if it works stable enough with winxp and vista (anyone confirm this?). I'm going to buy this sony cam out of my own pocket to play with. In case this works, I may be able to convince my colleagues to equip 10-20 of these in our labs, which is exciting. A part of my code is tracking motion on live video, which is similar to what PS3 can do but I got my ideas from places other than their E3 or 3E show. I am, however using glowing stuff as well, like their orbs.

liudr

Re: Tricks for enhancing tracking?
Reply #13 - Nov 18^th, 2009, 10:48am

Realized my last post was off topic. Here's some on-topic reply:
I'm tracking juggling balls with live video. All I need is that the balls are uniform in prime color (say red) and the background without this color. I calculate what's so called center of mass of red in each frame and get a fairly good result/not too heavy on calculation even with myself moving in the frame (I track green better since my skin also has some redness that may be affecting red tracking).

I can track as many as three objects in red, green, and blue but may track more with more calculations that slow down my fps.

Another suggestion is to use a lit juggling ball. Google LED juggling balls. They're easier to track since they emit color instead of reflecting.

gcolefla YaBB Newbies Offline Posts: 25	Re: Tricks for enhancing tracking? Reply #14 - Nov 28^th, 2009, 11:04pm Solved - see http://processing.org/discourse/yabb2/num_1259097615.html Has anyone been able to make JMyron recognize the PS3 eye yet (in windows) Tonight I was able to do it on a mac (using macam, which is very easy to do), but I need the power of my pc desktop to do the processing. Thanks for any help.
	« Last Edit: Nov 29^th, 2009, 1:56pm by gcolefla »