I forgot to mention here that I used Processing to build the following installation for the Gameworld exhibit at the Laboral art center in Gijon, Spain:

http://www.youtube.com/watch?v=I6KnLWLgK_4

I'm very happy with it, as it's my first full-scale public installation using Processing. It's a very simple piece, but I wanted to start slow.

You can see videos of the other works from the exhibit and from the neighboring exhibits (including a lovely work from Casey Reas) here:

http://www.abstractmachine.net/blog/gameworld-feedback-labcyberspaces/

As for the development details, I used two mac Minis, just to keep things simple on the development front. OSC connects the two computers: #1 for video tracking -> #2 sound/graphics/gameplay. I used the ESS library for sound generation, OpenGL for nice full-screeness, and the Camera library for grabbing live video from the camera.

The entire development was completed in about 4 days, on site. It's pretty amazing how fast you can work with Processing: 2 days negotiating with the construction crew to get a camera up in the rafters, getting the right lighting, etc, and 2 days programming. The video tracking (see video) was programmed in a just over an hour using some old-skool tracking technics (looking for pixel changes). As with all video tracking projects it was harder to actually setup the camera + lighting than programming it. The algorithm in this instance is more in the design of the space than the actual code.

If you have any development questions, fire away.