Processing 1.0 - Processing Discourse

Menekis YaBB Newbies Offline Posts: 5	Webcam via UDP Oct 14^th, 2007, 4:57am I was wondering, Do you think that it would be possible to transfer a webcam via udp so that 2 person could see the images live?

nitelite

Re: Webcam via UDP
Reply #1 - Oct 17^th, 2007, 12:34pm

Shouldn't be a problem in theory at least ...

Seems the processing.net library only supports TCP, but there is a UDP library you can try out at http://hypermedia.loeil.org/processing/

I haven't been using Processing for very long, but if possible you should probably apply some compression to the image before you send it. An uncompressed 320 x 240 image with 16 bits of color info would be around 150kb pr frame if i am not mistaken... (please correct me if any of my assumptions are flawed)

You could possibly use the JPEGImageEncoder class from the com.sun.image.codec.jpeg package to get a BufferedOutputStream that you could send as an UDP datagram. With JPEG it should be easy to get one frame in under 65kb, which is the limit for one UDP datagram as far as I know.

Menekis YaBB Newbies Offline Posts: 5	Re: Webcam via UDP Reply #2 - Oct 18^th, 2007, 6:12am thanks , i'll try that!

Menekis YaBB Newbies Offline Posts: 5	Re: Webcam via UDP Reply #3 - Oct 19^th, 2007, 5:16pm The problem qith that is I would need a webserver to put the jpeg on it and retrieve the file at each frame.. There is no way to have a direct connection? With a special plugin maybe?

nitelite

Re: Webcam via UDP
Reply #4 - Oct 20^th, 2007, 10:08pm

No need to have a web server, you build a server into your app using the UDP library. You would also need to make a client that can connect to your server and talk the language of the processing server.

The data would flow like this:

1. Read Webcam
2. Read frame into PImage
3. Encode to JPEG
4. Send the JPEG binary data as a datagram via UDP
5. Any UDP clients connected would receive the datagram
6. Datagram contains JPEG data so we decode the data into a PImage
7. Display on screen on other computer

Points 5. - 7. could exist on as many computers you would want, depending on how many are "connected" to the UDP port

nitelite

Re: Webcam via UDP
Reply #5 - Oct 21^st, 2007, 2:12am

I was a bit facinated by your project so I sat down this evening and tried to make a web cam over udp style thing with multicast, so that you can send one cam to 20 people if you like ...

This uses the UDP library at http://hypermedia.loeil.org/processing/ and the ImageIO that comes with JDK 1.4.4 and up.

If I havent screwed anything up, you should be able to start this on two different machine on the same network and when you press C on one of the machines, it should start transmitting its webcam to the network. Anyone listening (other computers or itself) should start displaying the image. Local web cam is left part of window, remote frames is right part.

There is an issue if more then one of these programs are transmitting at the same time. Since its multicasting, it receives its own frame as if someone else sent them, so two apps transmitting will result in flickering between the two sources on both machines. This is solvable though. You could probably use a header at the start of the datagrams to identify who the sender is and allocate a part of the window for each machine, but this is left as an exercise for the user Tongue

I have tried to stuff some comments into the Java code, if anyone has any questions, comments or improvements, please do respond. Feel free to use this code as you please, its there for people to learn

Source Code: http://prosjekt.be/WebCamOverUDP.pde

Menekis YaBB Newbies Offline Posts: 5	Re: Webcam via UDP Reply #6 - Oct 27^th, 2007, 4:58am damn thx ! You are a beast! Well I've tried with my system of using an apache and php server. it works.. but it is damn slow. I din't know that my project could be that facinating ! Well thx again il try to post later what I have done with that!

sowhat YaBB Newbies Offline Posts: 37	Re: Webcam via UDP Reply #7 - Nov 21^st, 2007, 12:12pm Hey, I tested your programm, works fine! Now I got the multicast issue How can i seperate between incoming datagrams and the one I sent?

sowhat YaBB Newbies Offline Posts: 37	Re: Webcam via UDP Reply #8 - Nov 21^st, 2007, 4:15pm I solved the flicker problem: simply add this to the initial setup: udp.loopback(false); cheers, flo

sowhat

Re: Webcam via UDP
Reply #9 - Nov 21^st, 2007, 4:51pm

Another Problem:

Where to process the incoming data?

In my opinion right here:

synchronized void captureEvent( Capture c ) {
if(c.available()) {
// Read frame from camera
c.read();
// process the image data
......
workspace.set( 0, 0, c );
localFrame = workspace.get();
sendFrame();
}

But the operations aren't working...any suggestions?
}

sowhat

Re: Webcam via UDP
Reply #10 - Nov 22^nd, 2007, 1:59pm

Okay, I found out that I can work on the pixels like that:

synchronized void draw(){

if (cam.available()) {
cam.read();
cam.loadPixels();
int detail = 10;
for ( int i=0; i<width;i+=detail) {
for (int j=0;j<height;j+=detail) {
color d = cam.get(i,j);
fill(d);
rect(i,j,detail,detail);
}
}
}
workspace.set( 0, 0, cam );
localFrame = workspace.get();
sendFrame();

// If we are not currently getting a new frame, we wanna show what we have.
if(!gettingNewImage && remoteFrame != null)
{
image( remoteFrame, w, 0 );
}
}

But the frames, which are sent, are not the pixels I changed. I guess it's because of this argument:
workspace.set( 0, 0, cam );

Why doesn't the method use the updated pixels?

hopefully someone might help me.

JosephDuchesne

Re: Webcam via UDP
Reply #11 - Dec 5^th, 2007, 6:01am

I have code that does what you want, but it uses the regular processing net library, rather than UDP (because the UDP library site seems to be down at the moment, so I can't get it.) I do plan to convert this to UDP when I can get the library of course, for speed reasons.
With jpg compression 320x240px color image is only a few kb. You can change the image quality in the jpgFromPImage function where setQuality(0.4... is 40% quality.

Here's the server:
Code:


//TODO: Rewrite using UDP... TCP is SLOOOOOOOOW

import processing.net.*;
import processing.video.*;
import com.sun.image.codec.jpeg.*;

Server myServer;
Capture video;
boolean startServing=false;

void setup() {

  size(320,240);
  frameRate(30);
  video = new Capture(this, 640, 480);
  myServer = new Server(this,9090);
  background(0);
}

void serverEvent(Server s, Client c)
{
  startServing=true;
  println("Client connected. Server transmitting...");
}

void draw(){

  if(video.available()){
    video.read();
    //image(video,0,0,width,height);
    if(startServing){
	byte[] jpgBytes=jpgFromPImage(video);
	//println("Sending image of size "+jpgBytes.length);
	myServer.write(jpgBytes.length/256); //high bit
	myServer.write(jpgBytes.length%256);//low bit (length in bytes)
	myServer.write(jpgBytes); 
    }
  }

}

//following function modified from forum post by seltar
//http://processing.org/discourse/yabb_beta/YaBB.cgi?board=Syntax;action=display;num=1138221586

 byte[] jpgFromPImage(PImage srcimg){
  ByteArrayOutputStream out = new ByteArrayOutputStream();
  BufferedImage img = new BufferedImage(srcimg.width, srcimg.height, 2);
  img = (BufferedImage)createImage(srcimg.width, srcimg.height);
  for(int i = 0; i < srcimg.width; i++)
  {
    for(int j = 0; j < srcimg.height; j++)
    {
     int id = j*srcimg.width+i;
     img.setRGB(i,j, srcimg.pixels[id]);  
    }
  }
  try{
    JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out);
    JPEGEncodeParam encpar = encoder.getDefaultJPEGEncodeParam(img);
    encpar.setQuality(0.4,false); // 0.0-1.0, force baseline
  encoder.setJPEGEncodeParam(encpar);
    encoder.encode(img);
  }
  catch(FileNotFoundException e){
    System.out.println(e);
  }
  catch(IOException ioe){
    System.out.println(ioe);
  }
  return out.toByteArray();
}

JosephDuchesne

Re: Webcam via UDP
Reply #12 - Dec 5^th, 2007, 6:03am

Here's the client side of things:
(I'm double posting because I had the code for the two parts on different computers Tongue

)
Again, I'm hoping to rewrite this with UDP ASAP, but it works well enough over lan.

Note that this is really a rough draft. You must run the server before running the client and only one client should connect.

Code:


import processing.net.*;
Client myClient; 

int W=640;
int H=480;

void setup(){
  size(W,H);
  myClient = new Client(this, "192.168.0.101", 9090); 
}

void draw(){
  
  
  if(myClient.available()>1){
    int imageSize=myClient.read()*256+myClient.read();
    //println("Reading jpg frame of bytesize "+imageSize);
    while(myClient.available()<imageSize);
	byte[] dataIn=new byte[imageSize];
	myClient.readBytes(dataIn);
	PImage i=loadPImageFromBytes(dataIn,this);
	image(i,0,0);
  }

}

//function based the processing libraries new PImage function
PImage loadPImageFromBytes(byte[] b,PApplet p){
  Image img = Toolkit.getDefaultToolkit().createImage(b);
  MediaTracker t=new MediaTracker(p);
  t.addImage(img,0);
  try{
    t.waitForAll();
  }
  catch(Exception e){
    println(e);
  }
  return new PImage(img);
}

bigbadotis

Re: Webcam via UDP
Reply #13 - Mar 5^th, 2008, 5:53am

Here's a UDP version of the code above. You have to have the UDP library to run it... search the forums and you'll find a cached link to download it, I can't remember where it is right now.

It looks like using the UDP library simplifies the code a lot. The files are setup to run locally; you can start them in any order. The broadcaster will simply display a blank screen, the receiver will display the streamed video when the broadcaster is running.

Receiver provided first, broadcaster second.

Quote:

// UDPReceiver
// based on code found at:
/* http://processing.org/discourse/yabb_beta/YaBB.cgi?board=Video;action=display;num=1192330628;start=5#5 */

import hypermedia.net.*;

UDP udp;

void setup(){
size(320,240);
udp = new UDP( this, 9091, "127.0.0.1"); // this, port, ip address
udp.listen(true);
background(0);
}

void draw() {}

void receive( byte[] data, String ip, int port ) {
// <-- extended handler
PImage i = loadPImageFromBytes(data, this);
image(i,0,0);
}

//function based the processing libraries new PImage function
PImage loadPImageFromBytes(byte[] b, PApplet p){
Image img = Toolkit.getDefaultToolkit().createImage(b);
MediaTracker t=new MediaTracker(p);
t.addImage(img,0);
try{
t.waitForAll();
}
catch(Exception e){
println(e);
}
return new PImage(img);
}

bigbadotis

Re: Webcam via UDP
Reply #14 - Mar 5^th, 2008, 5:56am

Code:



// UDPBroadcaster  
// based on code found at:
// http://processing.org/discourse/yabb_beta/YaBB.cgi?board=Video;action=display;num=1192330628;start=5#5

import hypermedia.net.*;
import processing.video.*; 
import com.sun.image.codec.jpeg.*; 
 
UDP udp;
Capture video;

void setup() { 
  size(320,240); 
  udp = new UDP( this);

  frameRate(30);
  video = new Capture(this, 320, 240); 
  background(0); 
} 

void draw(){ 
  if(video.available()){ 
    video.read(); 
    //image(video,0,0,width,height);
    String ip	 = "localhost";
// the remote IP address
    int port	  = 9091;

// the destination port
    byte[] jpgBytes=jpgFromPImage(video); 
    udp.send( jpgBytes, ip, port );	 
  }
} 

//following function modified from forum post by seltar 
//http://processing.org/discourse/yabb_beta/YaBB.cgi?board=Syntax;action=display;num=1138221586 

byte[] jpgFromPImage(PImage srcimg){ 
  ByteArrayOutputStream out = new ByteArrayOutputStream(); 
  BufferedImage img = new BufferedImage(srcimg.width, srcimg.height, 2); 
  img = (BufferedImage)createImage(srcimg.width, srcimg.height); 
  
  try {  // make sure we can access the buffered image
    img.setRGB(0,0, 0);
  } catch (Exception e) {
    return null;
  }
  
  for(int i = 0; i < srcimg.width; i++) { 
    for(int j = 0; j < srcimg.height; j++) { 
	int id = j*srcimg.width+i; 
	try {
	  img.setRGB(i,j, srcimg.pixels[id]);
	}catch (Exception e) {}   
    }
  }
  
  try{ 
    JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out); 
    JPEGEncodeParam encpar = encoder.getDefaultJPEGEncodeParam(img); 
    encpar.setQuality(0.4,false); // 0.0-1.0, force baseline 
    encoder.setJPEGEncodeParam(encpar); 
    encoder.encode(img); 
  }catch(FileNotFoundException e){ 
    System.out.println(e); 
  }catch(IOException ioe){ 
    System.out.println(ioe); 
  }
  
  return out.toByteArray(); 
}