Processing 1.0 - Processing Discourse - Arduino (via Serial library) + video library?

c0sis

Arduino (via Serial library) + video library?
Jun 19^th, 2008, 5:21pm

Hey everyone,
I've been fooling around with a sketch that controls an arduino (via the serial library) and also takes images via the video library. However, I can only do one at a time. If I try to take images AND control the arduino in the same sketch, it hangs. I've traced this as far as the hang being in arduino.available() (Serial.available()).

So my question: is this a bug, or am I doing something very wrong?

Sketch code (ignore any commented code lines, they're mostly there for debugging):
Quote:

import processing.video.*;

import processing.serial.*;

Serial arduino;

PImage nolaser;
PImage laser;

Capture video;

void setup() {
println(Serial.list());
println(Capture.list());
println("started");
size(640, 240); //So we can display two side-by-side images

video = new Capture(this, 320, 240, 12); //About the size my camera is, and on a slow framerate.
arduino = new Serial(this, "/dev/cu.usbserial-A4001tLy", 9600); // Start the arduino
arduino.write('A'); //Initialize communication
println("setup done");
}

void draw() {
//println("begin draw loop");
while(!video.available()) {
//idle
}
println("nolaser frame available...");
video.read();
//nolaser = video;
//println("read.");
image(video,0,0);
//arduino.write('G');
laserOn();
println("laser is now on!");
println("starting laser frame. Idling.");
while(!video.available()) {
//idle
}
//println("laser frame available...");
video.read();
//laser = video;
//println("read.");
//arduino.write('F');
laserOff();
println("laser is now off");
image(video,320,0);
}

boolean laserOn() {
println("attempting to turn on the laser...");
arduino.write('G');
//println("wrote!");
while(arduino.available()<1) {
//We wait!
println("waiting");
}
//Ok, the next piece is somewhat superflous now.
//But this makes sure the arduino is able to delay properly.
println("reading");
if(arduino.read()=='K') {
println("laser turned on");
return(true);
}
else {
return(false);
}
}

boolean laserOff() {
//println("attempting to turn of the laser...");
arduino.write('F');
//println("wrote!");
if (arduino.available()>1) {
//We wait!
//println("waiting");
}
println("reading");
if(arduino.read()=='K') {
//println("laser turned off");
return(true);
}
else {
return(false);
}
}

The specific point of failure is in laserOn() and laserOff() - the two lines which read Quote:

if (arduino.available()>1) {

. The sketch runs properly without the calls to laserOn() and laserOff(), but hangs as soon as it hits that specific line.

Finally, some machine specific info: Mac OS X 10.4.11, G4. The camera is a lego-branded Logitech quickcam web (as reported by Macam). Processing shows it as
[2] "QuickCam Web"
[3] "USB Video Class Video"

Comments, help, suggestions? Perhaps I should try putting the camera stuff and the laser stuff into a class so theres absolutely no way they can interfere with eachother?

rrrufusss

Re: Arduino (via Serial library) + video library?
Reply #1 - Jun 19^th, 2008, 7:57pm

Can you describe the desired behavior of the sketch? It's a little difficult to figure out what you're attempting to do.
Are you just trying to turn the laser on and off each time a new frame is available from the camera?

My guess is that the problem stems from using those empty "while" loops. I'd strongly recommend that you get rid of those since they are essentially locking up processing until the while condition is met. For example, if the arduino never sends a result back, you're stuck in that while loop forever.

In general, using an empty loop as a means of causing a delay is a dangerous practice. You're just asking for strange behavior such as what you've seen. If you want to wait until a certain event happens, there are many other ways of doing that which won't cause the sketch to hang.

c0sis

Re: Arduino (via Serial library) + video library?
Reply #2 - Jun 19^th, 2008, 8:27pm

The desired behavior is to repeatedly take two images, one with the laser off, one with the laser on, and display them side by side. this means taking one frame with the laser off, turning the laser on (and waiting for its small warmup period, at the end of which arduino sends back a single K), taking another fram, and turning the laser of (again, the response is a K). (notice that this means there is ALWAYS a response).

The issue is not that it doesnt work PERIOD (as I've tested each component separately), but that these two components don't work together. I can make the laser work, and I can make the video they do not work together.

Besides, how would you ask the sketch to wait without an empty loop? Keep in mind that however you wait has to be able to reliably wait for several events in order AND ignore irrelevant events.

rrrufusss

Re: Arduino (via Serial library) + video library?
Reply #3 - Jun 20^th, 2008, 12:11am

I would suggest a state machine of sorts, so you let "draw" run over and over and check what state you're in each time. Then, depending on the state you take action accordingly. That way you aren't freezing up the sketch, but are just checking the current state each time "draw" is invoked.

Here's an example. I haven't tested this but I think it should produce the desired results..

Code:


import processing.video.*;
import processing.serial.*;
 
Serial arduino;
 
PImage nolaser;
PImage laser;
 
Capture video;

int currentState = 0;
 
void setup() {
  size(640, 240); //So we can display two side-by-side images
 
  video = new Capture(this, 320, 240, 12); //About the size my camera is, and on a slow framerate.
  arduino = new Serial(this, "/dev/cu.usbserial-A4001tLy", 9600); // Start the arduino
  arduino.write('A'); //Initialize communication

  println("setup done");
}
 
void draw() 
{ 
  switch( currentState )
  {
    case 0: // laser off, get a frame (when available)
	  if( video.available() )
	  {
	    video.read();
	    image(video, 0, 0);
	    arduino.write('G'); 
	    
	    currentState = 1;
	  }
	break;
	
    case 1: // waiting for "laser on" confirmation
	  if( arduino.available() > 0 )
	  {
	    currentState = 2;
	  }
	break;
	
    case 2: // laser is on, get a frame (when available)
	  if( video.available() )
	  {
	    video.read();
	    image(video, 320, 0);
	    arduino.write('F');
	    
	    currentState = 3;
	  }
	break;
	
    case 3: // waiting for "laser off" confirmation
	  if( arduino.available() > 0 )
	  {
	    currentState = 0; // repeat the cycle..
	  }
	break;
  } 
}

c0sis

Re: Arduino (via Serial library) + video library?
Reply #4 - Jun 20^th, 2008, 12:49am

rrrufusss, thanks for your help. I copied your code (very elegant. Wish I'd thought of doing it like that) int processing and ran it. It stalled, just as before. I added a println("case 1"); just inside case 1, and tried again. It stalled, and just endlessly repeated that line (no, it never got to case 2).

I'm about ready to file a bug report on this, if I can get someone to replicate the behavior, or confirm it with other hardware/operating systems. I have done enough experimentation to determine that it may be related to my camera driver. When the camera is tied up doing something (and quicktime returns the frames that say so), the code will work perfectly. I've got another webcamera I'll dig up (unfortunately also a logitech) and try the code with, and a firewire video camera I can try it with. If all of these fail, I'll be filing a bug report.

If anyone has suggestions for how to get around this problem, I'd love to hear them. I (in my amateurish way) tried to get multi-threading to work, but it also failed there. More likely a problem with my code however...

c0sis

rrrufusss

Re: Arduino (via Serial library) + video library?
Reply #5 - Jun 20^th, 2008, 12:56am

Are you 100% certain that the arduino is sending the response? I can't think of any other reason that the code would hang there.. Perhaps you could post the arduino code as well? I'm familiar with arduino so maybe I can spot a bug in there?

One other thing, I realized there's a potential bug in my example code. If you don't call arduino.read(), then the buffer won't be cleared out and arduino.available() will always return a number higher than zero (after the first data is sent from arduino, I mean).

To fix this, modify the switch statement for cases 1 and 3
like so:

Code:


case 1: 
  if(arduino.available() > 0)
  {
    arduino.clear();
    currentState = 2;
  }
break;

and

Code:


case 3: 
  if(arduino.available() > 0)
  {
    arduino.clear();
    currentState = 0;
  }
break;

c0sis

Re: Arduino (via Serial library) + video library?
Reply #6 - Jun 20^th, 2008, 2:45am

I made those chnges; it still stalled in case 1.

I've used the Serial Monitor in arduino-10 to make sure data is being sent as processing expects it - and there is in fact a single K returned before processing locks up.

Finally, arduino code (really simple):
Quote:

/*
Laser Control Board

Controls a laser line diode by switching the power on and off
(using a relay in my case)
When it recieves an 'G' (for go on) via serial, it turns the laser on.
When it gets an 'F' (for off), it turns it off.
Oh, and the bord acknolages the change in status with 'K'. If
somehow communication fails, print 'N'.
*/

int relayPin = 13; // LED connected to digital pin 13
char command;
int success=0;

void setup()
{
pinMode(relayPin, OUTPUT); // sets the digital pin as output
beginSerial(9600);
}

void loop()
{
if(serialAvailable()) {
command = serialRead();
if (command=='G') {
digitalWrite(relayPin,HIGH);
success=1;
} else if (command=='F') {
digitalWrite(relayPin,LOW);
success=1;
} else {
success=0;
}

delay(20);
//Delays 2/10ths of a second to make sure the laser has come on

if (success==1) {
serialWrite('K');
} else {
serialWrite('N');
}
success=0;
}
}

rrrufusss YaBB Newbies Offline Posts: 32 Oregon, USA	Re: Arduino (via Serial library) + video library? Reply #7 - Jun 20^th, 2008, 6:14am Odd.. I just hooked up my arduino and tried the code, cut and pasted directly from this post, and it works fine for me. The LED (instead of a laser) blinks and the video shows up just fine in processing. No stalling, etc..

c0sis YaBB Newbies Offline Posts: 7	Re: Arduino (via Serial library) + video library? Reply #8 - Jun 20^th, 2008, 2:35pm What system are you on? What type of camera did you use? What type of arduino? Did you run anything through a USB hub? I'd like to add all of this to my bug report (and perhaps just switch development to a system which is known to work..) Thanks for all your help, C0sis

rrrufusss YaBB Newbies Offline Posts: 32 Oregon, USA	Re: Arduino (via Serial library) + video library? Reply #9 - Jun 20^th, 2008, 5:57pm computer: Macbook pro, intel cpu, running OS 10.4 arduino: usb, the older kind with the ATmega8 camera: built-in iSight in the laptop No usb hub, arduino is plugged directly into the usb port on the laptop.

c0sis

Re: Arduino (via Serial library) + video library?
Reply #10 - Jun 23^rd, 2008, 12:56am

Will update the bugreport to include this info.

I managed to work around this issue in my case by simply sleeping the thread for a bit. The purpose of the arduino was just to delay enough so the laser would be able to come fully on.

Heres some code to do a delay:
try {
Thread.sleep(500);
//println("slept");
} catch (InterruptedException e) { println("interupted");}

telharmonium YaBB Newbies Offline Posts: 7	Re: Arduino (via Serial library) + video library? Reply #11 - Jul 2^nd, 2009, 4:50pm I incorporated rrrufusss buffer class into a video sampler I built, and wanted to thank him here, where I saw the code. Much thanks. I signed up today and would have wanted to include a link to documentation of it, but apparently I can't do that here?