Processing 1.0 - Processing Discourse - Communication between EyesWeb and Processing

st33d

Re: Communication between EyesWeb and Processing
Reply #15 - Sep 8^th, 2005, 7:24pm

Just so the answer to this is online:

http://www.robotacid.com/misc/matrixSplit.eywx

Been having a chat with the EyesWeb guys. A Matrix to String converter is in the works but for now there is a method for extracting info from the Matrix block. The example was sent to me by Roberto who showed us the EyesWeb 4 in Prague. It shows how you can separate info in a matrix into scalar elements. You could concatenate these into a String. A bit lengthy but I believe you can make a block which is effectively another patch.

beachmeat

Re: Communication between EyesWeb and Processing
Reply #16 - Sep 9^th, 2005, 11:10am

Very well...lets keep this alive and available to the public...
The provided patch shows how to getEntries of a matrix the old-school way...entry by entry...we need something quicker, especially for matrices with 10000+ entries Wink

Paolo Coletta, one of the members of the EyesWeb-team has replied to this also in the EyesWeb newsgroup.

This is his input: http://www.dr3.de/processing/MatrixToScalar.eywx

As we can see he uses a timebased loop to getEntries of the Matrix. Since we need something that loops over the whole matrix per each frame and concats them to a string, this is not really suitable.
Anybody out here who knows C++ and could program the block to do that? I imagine that if you know C++, it should be a piece of cake but I may be mistaken

home sweet home

Alterscape

Re: Communication between EyesWeb and Processing
Reply #17 - Sep 12^th, 2005, 5:47pm

Hey everybody,
I just thought I'd chime in and acknowledge how useful this topic has been to me. I'm working on implementing a system wherein Eyesweb feeds motion-analysis data to both Processing and MAX/MSP via OSC for an interactive performance under development at Cornell University, and reading what has been accomplished so far has been very useful. If I make any progress working at this thing on my own, I'll be sure to post it here. Thanks for all your work thusfar; I can say with certainty that at least one member of the community appreciates it greatly. Wink

Cheers,
Ryan Spicer

beachmeat Full Member Offline Posts: 105	Re: Communication between EyesWeb and Processing Reply #18 - Sep 13^th, 2005, 8:49am Very good to hear! How much data do you send per frame? I am still stuck at the same problem, that I cant implode a matrix, which sounds incredibly ridicolous to me...but thats how it is when you suck at c++ Post some link to your installation when youre finished!
	*home sweet home*

Alterscape

Re: Communication between EyesWeb and Processing
Reply #19 - Sep 13^th, 2005, 11:11pm

Ideally, I'd like to have two systems running Eyesweb -- one camera above the performers aiming downward to capture their positions on the stage, and one camera in front, capturing their gestures. About six dancers are involved, so I'm hoping that OSC is up to streaming the complete ManParams matrix for each dancer, plus their X and Y coordinates on the stage, per frame. I know that sounds crazy and its probably going to be scaled back, but for now I might as well shoot for the moon.

My software development experience is entirely in Java (thus why Processing makes such a good fit for me) but I'm hoping I can muddle through setting up GCC to compile an Eyesweb block to string-ify a matrix. I wouldn't hold my breath, though, since my C++ development experience can be summarized by the word "nonexistant" .. Wink

If I do come up with something workable, I'll post it here, of course.

beachmeat Full Member Offline Posts: 105	Re: Communication between EyesWeb and Processing Reply #20 - Sep 14^th, 2005, 9:59am The datarate you aim at shouldnt be a problem - if just the block were available! So if you go for making that block in EyesWeb - please do!!! Thats the most required block in the history of computation ever!
	*home sweet home*

st33d God Member Offline Posts: 666	Re: Communication between EyesWeb and Processing Reply #21 - Sep 14^th, 2005, 5:57pm Drop me a line if the String-Matrix block is formed. I'm sure I'll be able to get the EyeWeb guys to help. In the mean time - good luck and good work!

beachmeat

Re: Communication between EyesWeb and Processing
Reply #22 - Sep 19^th, 2005, 12:44pm

Hey gals!

I just stumbled across a javasocketserver that receives data from EyesWeb network_send block - and this block can send any kind of data!

I put this example together fast, where you can see it working for real - EyesWeb-matrix to Processing Cheesy

Woohoooo

http://www.unshift.de/admin/SAM//www/index.php?remoteID=468

Edit: as always, if you solve one problem, the next one is already waiting patiently in the line. The Socketserver seems to halt everything so that the draw method is not being called anymore - the data get polled in correctly. That is weird. Any ideas?
Ugly workaround would be to translate the socket-input to OSC and send it to another applet. Not really cool.

« Last Edit: Sep 19^th, 2005, 4:31pm by beachmeat »

home sweet home

cosmin

Re: Communication between EyesWeb and Processing
Reply #23 - Sep 20^th, 2005, 4:57am

yup, I'm there too, just noticed it and dropped it. The good thing is that I can receive a matrix in P5, but with all the hassle it's not worth it anymore. I'll guess when the time comes I'll write the EW block myself and save the trouble. I don't know why you won;t do it yourself, if you have a fair understanding of processing I doubt it'll take you more then 2 days to write the block. The tutorials are pretty good.

http://10megatons.com

beachmeat

Re: Communication between EyesWeb and Processing
Reply #24 - Sep 20^th, 2005, 8:17am

I already had my hands on the tutorial and yes I also guess it should work, but it is also the 2nd most interesting solution, since it wastes CPU with converting to String/from String - and that with something around 640x480 Entries. So the networksend thingie is just sending the raw data, and the socket reads it from the buffer. Better. But well, when it halts the applet (I assume it halts everything to continuosly read whats incoming to the buffer), what can you do...
Another idea is (its getting complicated) to make the socket-standalone. What would be the fastest and less CPU-eating way to communicate that kind of data from Processing to Processing?

home sweet home

st33d

Re: Communication between EyesWeb and Processing
Reply #25 - Sep 20^th, 2005, 1:29pm

http://www.robotacid.com/misc/Matrix2String.zip

Wink

Why is it necessary to scan a 640x320 image? Are you performing a transformation on the image or just looking for something in the picture? If it's the latter you can get away with a much lower resolution or at least skip every second pixel. Another thing you can do to get some performance out of your machine is to drop your applet size to 640x480 and drop the display (monitor) to 800x600. Also there's how much memory Processing has to use (defined in the run-expert.bat file).

I know the matrix 2 string patch isn't top speed but there's probably some data that could be ommited, (plus it might help if I wasn't listening to Kings of Leon at the same time).

The data can be plugged into a String2Osc block. That data can be extracted with
Code:


//lets say our OSC string is called "numbers"
int list[] = int(split(numbers),":");

For the best performance you're gonna have to get another machine.

beachmeat

Re: Communication between EyesWeb and Processing
Reply #26 - Sep 20^th, 2005, 3:31pm

St33d, your matrix2string patch is basically not usable in this case I think.
Imagine I even go to a lower resolution like 240x160 - it will take you days to make a patch that has 38400 blocks PLUS i bet EW will crash then Cheesy

And I really need to transfer the whole 1bit matrix (with some simple string compression).
And I will work on two 3.4 Ghz machines

Got my hands on coding the block in C++ myself...can only take weeks....

home sweet home

st33d

Re: Communication between EyesWeb and Processing
Reply #27 - Sep 21^st, 2005, 3:03am

Dude, that's a silly amount of data to send down a network cable.

Is it possible send the video to a TV card and let Win VDig read the video off that? - it reads my TV card quite happily. Would seem a tad more sense than busting one's noodle getting the pixels to queue up and go down the network. Or could you just send the video over the network? Why decode and recode if it's pixels you're after? Why not transform them at the source into an appreciable data image and then process it on the other machine?

What the are you trying to do anyway?

beachmeat

Re: Communication between EyesWeb and Processing
Reply #28 - Sep 21^st, 2005, 9:40am

Its not that silly as it seems - at least I hope so Wink

First: good news, I fixed the socketserver issue, so that is working fine now with really big matrices as well.
http://www.unshift.de/admin/SAM/www/index.php?remoteID=468

What I wanna do:
Analyse the videoimage from a camera and extract smoothened silhouettes of people without the background. People can then make gestures to control the installation and to interact with it. This interaction is highly visual and needs a lot of CPU power, so I have to do the videoanalysis on another machine, its quite impossible to do it on one.
The imageanalysis of course works very good in EyesWeb, so i just need to send the result over to P5 - not the image, just 1/0 if theres a person.

For that I have 2 ideas:

1)Socket send all the matrix.
advantage: few CPU power on target machine required to extract the data.
disdavantage: big amount of data

2) Write an own block in EyesWeb that compressses the matrix. Since the silhouettes consist of big continuosly parts, its very easy to compress the data (count continuos same values) and send that via socket or OSC.
advantage: less data
disadvantage: more PCU power required on target machine.

Of course I can make the image much smaller and resize it in processing, that is obvious that I do not need the full resolution lateron, but I first want to shoot for the best possible result and then see whats realisitic

home sweet home

Alterscape

Re: Communication between EyesWeb and Processing
Reply #29 - Sep 26^th, 2005, 3:22pm

Manuel,
The issue with P5 not drawing has to do with threading issues. The way the example is coded right now, the socketserver monopolizes all the CPU time for the program, preventing execution from ever reaching draw(); what we need to do is code the server stuff to run in another thread and exchange data with the applet's thread. I've never coded threaded apps before, but I'm working away at this one.

Ryan