using serial input from arduino to execute events in processing

liveoak

Hi all, I'm new to processing and I'm having trouble getting something to work when I try to base its execution on input from arduino. I can, however, get it to work work when I use keyPressed(). In both cases, the code loads a few images in the window without issue. In the keyPressed() version (works as intended), I use switch to pick the set of images to display, as defined by two other functions, femaleAngerTest and schematicAngerTest. When those appear, you either click on the correct image and get one image on top of the others, or you get it wrong and get a different image on top of the others. This is the user feedback. But I really want the arduino to trigger the test functions, not a keypress.

In the keyPressed() version, I can hit 'a' on the keyboard, and the appropriate set of images will sit there till you click on something, then the feedback will occur. If I then hit 'b', the other set of images will appear wait for you to click on something, then display the feedback image. Both of those are good things. But when I send trigger values from the arduino and read from the serial port in processing, the images appear, but I can only get feedback from a click if I click my mouse button super fast. The mouse press only gets noticed if I can do it right when the set of images loads. I can see that the value I'm sending (simple transmission of a 1 or 2, alternating every 5 seconds via delay for testing purposes) is being received, and the correct image test is occurring, but that isn't doing me any good if I can't get a response from processing when I mouse click on them. So, I'm just wondering what the issue is, and how I can fix it. I want to be able to do the following:

1) Send a value from Arduino to Processing to trigger one of the tests. Eventually, there will be more than 20, I just want to get 2 working for now.

2) Have Processing display the test image and display the feedback image in response to mousePressed().

3) After feedback has occurred, have Processing send a value back to arduino that it will read, and send a new value back to Processing after doing something else. This will trigger the next test in the list. I'm pretty comfortable with arduino, so I'm not too worried about this; I'm able to send values that trigger responses now, so I don't think the problem is on that end. Overall, the answer to this problem is probably something that will make me want to stick my head in the microwave when I hear it.

Here is the keyPressed() version, minus the image declaration and loading stuff:

import processing.serial.*; //import the Serial library
 Serial myPort;  //the Serial port object
 String val;

void setup() {
  //  initialize your serial port and set the baud rate to 9600
  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.bufferUntil('\n'); 
  size(1640, 1360); // window size; change to suit screen
}

void draw() { 
  keyPressed();
}

void keyPressed(){ // detect keypresses to trigger tests; ultimately this will be  changed to look for input from the robot
// switch between tests based on input from robot
switch(key) {
  case 'a': 
    femaleAngerTest();
    println("Female Anger Test");  // executes test if key = a, prints test name
    break;
  case 'b': 
    schematicAngerTest();
    println("Schematic Anger Test");  // executes test if key = b, prints test name
    break;
  }
}

// displays female faces, 'angry' label pops up if the angry face is clicked, 'different' pops up for a wrong click
void femaleAngerTest(){
  background(51);
  femaleImageDisplay(femaleAngry, femaleDisgust, femaleFear, femaleHappy, femaleSad, femaleSurprise);
    if((mousePressed) && mouseX<390 && mouseY<390){  
      image(labelAngry, 0, 0);
    }
    if(((mousePressed) && mouseX>390) || ((mousePressed) && mouseY>390)){
      image(labelDifferent, 500, 170);
    }
}

// displays schematic faces, 'angry' label pops up if the angry face is clicked, 'different' pops up for a wrong click
void schematicAngerTest(){
  background(51);
  schematicImageDisplay(schematicAngry, schematicDisgust, schematicFear, schematicHappy, schematicSad, schematicSurprise);
    if((mousePressed) && mouseX<390 && mouseY<390){  
    image(labelAngry, 0, 0);
  }
    if(((mousePressed) && mouseX>390) || ((mousePressed) && mouseY>390)){
      image(labelDifferent, 500, 170);
    }
}
  //schematicImageDisplay(schematicAngry, schematicDisgust, schematicFear, schematicHappy, schematicSad,     schematicSurprise);


void femaleImageDisplay(PImage a, PImage b, PImage c, PImage d, PImage e, PImage f) {
  image(a, 0, 0);
  image(b, 0, 580);
  image(c, 626, 0);
  image(d, 626, 580);
  image(e, 1254, 0);
  image(f, 1254, 580);
}

void schematicImageDisplay(PImage a, PImage b, PImage c, PImage d, PImage e, PImage f) {
  image(a, 0, 0, a.height/1.5, a.width/1.5);
  image(b, 0, 580, b.height/1.5, b.width/1.5);
  image(c, 626, 0, c.height/1.5, c.width/1.5);
  image(d, 626, 580, d.height/1.5, d.width/1.5);
  image(e, 1254, 0, e.height/1.5, e.width/1.5);
  image(f, 1254, 580, f.height/1.5, f.width/1.5);
} 

Here is the version that takes serial input:

import processing.serial.*; //import the Serial library
Serial myPort;  //the Serial port object

String val;
int v;

void setup() {
  //  initialize your serial port and set the baud rate to 9600
  myPort = new Serial(this, Serial.list()[0], 9600);
  myPort.bufferUntil('\n'); 
  size(1640, 1360); // window size; change to suit screen

void draw() { 
  serialEvent(myPort);
}

// communication between arduino and processing; detect string input and convert to int for execution
// of taskSelect function
void serialEvent(Serial myPort) { 
  val = myPort.readStringUntil('\n');
  //make sure our data isn't empty before continuing
  if (val != null) {
    //trim whitespace and formatting characters (like carriage return)
    myPort.clear();
    val = trim(val);
    println(val);
    v = int(val);
    taskSelect(v);
  }
}

void taskSelect(int v){ // detect serial input from arduino
switch(v) {
  case 1: 
    femaleAngerTest();
    println("Female Anger Test");  // executes test if v = 1, prints test name
    break;
  case 2: 
    schematicAngerTest();
    println("Schematic Anger Test");  // executes test if v = 2, prints test name
    break;
  }
}

The rest of the functions are the same, so I didn't paste them again.

Any help will be GREATLY appreciated. Thanks much.

liveoak

I've now noticed another problem. When I use the keyPress() version, each track that I play can only be played once. This is not optimal, since there are a few tracks that get repeated throughout. I'd rather not copy and rename all of those differently. Any ideas on that would be much appreciated as well. Thanks.

TechWiz777

For the playing once thing, try adding .loop at the end of each of each sound.

Hope it helps

liveoak

Thanks, TechWiz. I ended up using .rewind() just before each instance of .play(), but you put me on the right track to find it. So, thank you very much!

To everyone else, the first problem persists, so I'm still wanting advice on that. Thanks.