Processing 1.0 - Processing Discourse - sound vs video interaction

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

Index › Programming Questions & Help › Programs › sound vs video interaction

‹ Previous Topic | Next Topic ›

Pages: 1

sound vs video interaction (Read 1453 times)

guybowden

sound vs video interaction
Apr 29^th, 2005, 1:31pm

Hi,

I'm creating a fairly hefty virtual life simulation - its a farmyard of animals roaming round the fields, eating, sleeping etc etc.

I'm fairly certain about how I'm going to create these animals, using a combination of some Boid code along with adding more behaviours such as hunger / sleepyness etc. What I'd like to add is some form of human interaction:

Basically I want the creatures to be aware that someone is watching them - and be scared/curious of the human onlookers - at its most basic, I would like to a: detect if there is a human presence, and then b: react.

The reaction will be both dependant on the characteristics of the creatures (will they run and hide ? or stop and stare) and the level of interaction from the humans. I'd also like the creatures to know where you are across the screen - to the left or right.

Two ways I've thought of doing this:
1 - sound:
have a pair of microphones - one on each side of the projection, the level of sound + the position of the sound will be used to determine the position of the human, and how aggressive the human is.

2 - video:
Have a camera mounted above the area in front of the screen, and write some code to determine where there is a presence, and how aggressive - i.e. if there's lots of movement in one part of the camera image, there's an aggressive human there.

I'm fairly confident of doing it with video - however I think the sound method will give better results i.e. people could creep up on the unsuspecting creatures and then shout "boo!" - to scare them all off.

my question is - is the second method possible using P5?

And any thoughts on any of the above would be great!

Cheers
Guy

v3ga

Re: sound vs video interaction
Reply #1 - Apr 29^th, 2005, 4:01pm

Hello Guy,

a solution would be to use "computer vision". Presence/activity in front of the camera could be detected by comparing video frames ( supoosing your camera is not moving).
Some ideas :

0 - at the start of your program, simply save a reference frame (ie background image).
1 - Presence could be detected by making the difference between the current frame and the reference frame.
2 - Activity could be detected by making the difference between the current frame and the previous frame, when a presence has been detected.

I've been doing some computer vision stuff (edge+blob detection stuff), and am actually building a library for Proceesing (Beta) > http://v3ga.net/show.php?id=12&type=0

JMyron is also doing this kind of stuff in a different approach > http://processing.org/discourse/yabb_beta/YaBB.cgi?board=VideoCamera;action=display;num=1114529869

Hope this helps !

Ps : by the way, may be this thread should have been started in Video Capture subject.

site | blog&&

guybowden

Re: sound vs video interaction
Reply #2 - Apr 29^th, 2005, 5:13pm

cheers for the advice - sorry about it being in the wrong topic.. wasn't sure where to put it!

The part I'm interested in would be the sound input method... is it possible to do it that way? i.e. a pair of microphones? I guess I'd need some sound card that can have two inputs perhaps.. a line in and a microphone socket and then compare the two?

Daniel Shiffman

Re: sound vs video interaction
Reply #3 - Apr 29^th, 2005, 6:22pm

You may want to investigate "webcamxtra" a real-time computer vision library . . .

http://webcamxtra.sourceforge.net/

There is a java version and with some fiddling, it'll work with processing. . .

Some simple CV examples can be found here too (with a few leaps, you can turn this into blob tracking, etc.):

http://stage.itp.tsoa.nyu.edu/~dts204/ppaint/week11/index.html

(sorry, these are still for the old processing, will update soon!)

Best,
Dan

http://www.learningprocessing.com | http://www.shiffman.net | http://itp.nyu.edu

philworthy YaBB Newbies Offline Posts: 19	Re: sound vs video interaction Reply #4 - May 5^th, 2005, 2:50pm Hi. Do you have any idea of how your glob detection comares with Jmyron for speed? Thanx.
	~~>

Pages: 1

‹ Previous Topic | Next Topic ›