I have been playing with the Leap Motion for the last couple of days ever since I discovered the "Leap Motion for Processing" library.

The biggest problem that you have is that sometimes the Leap Motion library does not immediately provide data. To demonstrate this I modified example sketch "LM_1_Basics". It can take up to 99 msecs for the Leap Motion to connect, that's a really long time from the sketches point of view.

In your original code you were assuming the the right-hand's position data was available from the very first iteration of the draw(). Unfortunately, initially it isn't, I modified your code to illustrate this problem:

import de.voidplus.leapmotion.*; 
import processing.sound.*; 
import punktiert.math.Vec; 
import punktiert.physics.*;

LeapMotion leap; 
SinOsc sine; 
Hand leftHand; 
Hand rightHand;

float freq; 
float amp; 
float pos; 
PVector hL; 
PVector hR;

int teller = 0; 
int amount = 100;

// world object
VPhysics physics;

// attractor
BAttraction attr;

void setup() { 
  //size(displayWidth, 700);
  size(1280, 720); // HD 720P

  leap = new LeapMotion(this); 
  sine = new SinOsc(this);

  //Start the Sine Oscillator.
  sine.play();
  sine.amp(amp);

  noStroke(); 
  background(#57385c); 
  //frameRate(30); 
  smooth();

  //set up physics
  physics = new VPhysics();
  physics.setfriction(0.4f);

  // new AttractionForce: (Vec pos, radius, strength)
  attr = new BAttraction(new Vec(width * .5f, height * .5f), 400, .01f);
  physics.addBehavior(attr);
  //
  // val for arbitrary radius
  float rad = attr.getRadius()/4;
  // vector for position
  Vec pos = new Vec (random(rad, width/2 - rad), random(rad, height/2 - rad));
  // create particle (Vec pos, mass, radius)
  VParticle particle = new VParticle(pos, 9, rad);
  // add Collision Behavior
  particle.addBehavior(new BCollision());
  // add particle to world
  physics.addParticle(particle);

  for (int i = 1; i < amount; i++) {
    // val for arbitrary radius
    float rad2 = random(2, 20);
    // vector for position
    Vec pos2 = new Vec(random(rad2, width - rad2), random(rad2, height - rad2));
    // create particle (Vec pos, mass, radius)
    VParticle particle2 = new VParticle(pos2, 8, rad2);
    // add Collision Behavior
    particle2.addBehavior(new BCollision());
    // add particle to world
    physics.addParticle(particle2);
  }
}

void draw() { 
  background(255); 
  smooth();

  for (Hand hand : leap.getHands()) { 
    boolean handIsLeft = hand.isLeft(); 
    boolean handIsRight = hand.isRight();

    leftHand = leap.getLeftHand(); 
    hL = leftHand.getPosition();
    if (hL == null) println("No LH data!");

    rightHand = leap.getRightHand(); 
    hR = rightHand.getPosition();
    if (hR == null) println("No RH data!");

    //----------------------- LEFT HAND ----------------------------------
    if (handIsLeft) {
      text("leftHand-X=" + hL.x, 10, 15); //shows hL.x in sketch
      text("leftHand-Y=" + hL.y, 10, 30); //shows hL.y in sketch
      text("leftHand-Z=" + hL.z, 10, 45); //shows hL.z in sketch
      text("volume(in %)= " + nf(amp*100, 1, -3), 10, 60);

      leftHand.draw();
      // Map leftHandY van 0.0 to 1.0 voor amplitude (volume)
      amp = map(hL.y, 0, height, 1.0, 0.0);
    } 

    sine.amp(amp);

    //----------------------- RIGHT HAND ----------------------------------  
    if (handIsRight) {
      text("rightHand-X= " + hR.x, width-160, 15); // shows hL.x in sketch
      text("rightHand-Y= " + hR.y, width-160, 30); // shows hL.y in sketch
      text("rightHand-Z= " + hR.z, width-160, 45); // shows hL.z in sketch
      text("freq(in Hz)= " + nf(freq, 3, -3), width-160, 60);

      rightHand.draw();
      // Map rightHandX van 100Hz to 800Hz voor frequency
      freq = map(hR.x, 0, width, 100.0, 800.0);
    }

    if (!handIsLeft && handIsRight) {
      amp = 0;
    }

    sine.freq(freq);
  } // end for - hand

  physics.update();

  noFill(); 
  stroke(200, 0, 0);

  // set pos to mousePosition
  Vec mousePos;

  // Deal with right-hand position not yet ready?
  // Note: The Leap Motion can take a fraction of a second to start providing data and
  //       this code is outside the "for (Hand hand : leap.getHands())" for-loop ;-)
  if (hR == null) {
    println("hR is null!");
    mousePos = new Vec(mouseX, mouseY); // Added by Niels
  } else {
    println("hand x,y = " + hR.x + "," + hR.y);
    mousePos = new Vec(hR.x, hR.y); // Added by Niels
  }

  attr.setAttractor(mousePos);

  //ellipse(attr.getAttractor().x, attr.getAttractor().y, attr.getRadius(), attr.getRadius());

  noStroke(); 
  fill(0, 125);

  teller = 0; 
  for (VParticle p : physics.particles) { 
    //println("teller= " + teller);

    if (teller == 0) {
      p.set(mouseX, mouseY);
      ellipse(p.x, p.y, p.getRadius() * 2, p.getRadius() * 2);
      teller++;
    }

    if (teller == 1); {
      ellipse(p.x, p.y, p.getRadius() * 2, p.getRadius() * 2);
    }
  } // end for - p
}

With no call to frameRate() specified Processing will call draw() as quickly as it can. When I run the modified sketch I see the following in the console window:

# Leap Motion Library v2.3.1.6 - Leap Motion SDK v2.3.1+31549 - https://github.com/nok/leap-motion-processing hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! hR is null! translate(), or this particular variation of it, is not available with this renderer. hand x,y = 647.9134,592.4121 hand x,y = 786.3234,530.3577 hand x,y = 767.5345,525.84015 hand x,y = 745.8991,519.5922

So you see you need to handle the situation where you don't immediately get data from the Leap Motion.

Can you post the Processing code that you have at the moment?

I have more experience with Processing than with the Leap Motion--I confess I've had the Leap Motion for over a year now, but it was only today that I discovered that Processing has the "Leap Motion for Processing" library. :-bd

EDIT: Ok, after getting a bit more up to speed on the "Leap Motion for Processing" library I think I can offer a solution. The confusing part is that your source code snippet is in C++ and for a completely different Leap Motion library. :-(

The code below is borrowed from the example sketch "LM_2_Gestures", based on the code snippet you posted it looks like it testing to determine the duration of the swipe.

As for getting only one reading, I have to use millis() to record the time between when the swipe gestures stop. Though, I occasionally still see duplicates, try reducing the allow number of milliseconds (see if ( (currentTime - previousTime) < 100)) until it works the way you want it to. :-??

// LM_2_Gestures Modified
// Only report swipe gestures. Distinguish between slow and quick swipes (less than 0.1 seconds)

import de.voidplus.leapmotion.*;

// ======================================================
// Table of Contents:
// ├─ 1. Swipe Gesture
// ├─ 2. Circle Gesture (REMOVED FOR BREVITY)
// ├─ 3. Screen Tap Gesture (REMOVED FOR BREVITY)
// └─ 4. Key Tap Gesture (REMOVED FOR BREVITY)
// ======================================================

LeapMotion leap;

int previousTime;

void setup() {
  size(800, 500);
  background(255);
  // ...

  previousTime = millis();

  leap = new LeapMotion(this).allowGestures("swipe");  // Leap detects only swipe gestures
}

void draw() {
  background(255);
  // ...
}

// ======================================================
// 1. Swipe Gesture

void leapOnSwipeGesture(SwipeGesture g, int state) {
  int     id               = g.getId();
  Finger  finger           = g.getFinger();
  PVector position         = g.getPosition();
  PVector positionStart    = g.getStartPosition();
  PVector direction        = g.getDirection();
  float   speed            = g.getSpeed();
  long    duration         = g.getDuration();
  float   durationSeconds  = g.getDurationInSeconds();

  int currentTime = millis();
  boolean flagDuplicateSwipe = false;

  switch(state) {
  case 1: // Start
    break;

  case 2: // Update
    break;

  case 3: // Stop
    if ( (currentTime - previousTime) < 100) {
      flagDuplicateSwipe = true;
    }

    if (durationSeconds < 0.1) {
      println( (flagDuplicateSwipe ? "IGNORE " : "") + "Quick Swipe Gesture: " + id + " [duration = " + durationSeconds + "]");
    } else {
      println( (flagDuplicateSwipe ? "IGNORE " : "") + "Slow Swipe Gesture: " + id + " [duration = " + durationSeconds + "]");
    }

    previousTime = currentTime; // Refresh time stamp
    break;
  }
}

I'm wanna do something like this:

Leap::Frame sinceFrame;
int currentGestureID = -1;
Leap::Gesture currentGesture = frame.gesture(currentGestureID);
if(currentGesture.isValid()){
    //Update swipe action
} else {
    // Pick a new swipe
    Leap::GestureList gestures = frame.gestures(sinceFrame);
    for(auto gesture : gestures){
        if(gesture.type() == Leap::Gesture::TYPE_SWIPE
           && gesture.durationSeconds() < .1){
            currentGestureID = gesture.id();
            break;
        }
    }
}
sinceFrame = frame;

I got this code from this Leap Motion forum: https://forums.leapmotion.com/t/increase-range-and-decrease-sensitivity-for-swipe-gesture/3064/4

But I am unable to convert this code to use it in Processing due to my poor programming skill.

Fewer people has this experience as this technology is not as common here in the forum. Consider exploring previous posts: https://forum.processing.org/two/search?Search=leapmotion

Can you display your leapMotion together with video streaming? Would this multiple finger detection be caused by the system getting confused and detecting the other fingers as the index finger?

Kf

Hello,

Just to register here, I decided to do a test, I removed the Libraries from Leap Motion, which was what error was presented when creating the Executable.

I left in the Project the Libraries of Processing, ControlP5, G4P and Minim, and I was able to create both .jar and .exe, using launch4j, it worked normal.

So the problem is with the LeapMotion Library, it uses beyond .jar, 2 dlls, it looks like it is not finding their way.

I generated the .jar exporting as it is selected in the image below:

But this is already a great progress, thank you very much for the help here.

Thanks anyway, thank you!

• We can't directly use Java libraries on JS! @-)
• You're gonna need to find a LeapMotion library written in JS instead.
• For example, LeapJS: https://Developer-Archive.LeapMotion.com/javascript
• Since you're gonna need to interact w/ JS code, you're better off re-writing your sketch in p5.js: https://GitHub.com/processing/p5.js/wiki/Processing-transition
• And btW, I've just found out a sketch which uses the LeapJS library under p5.js: :-bd
  https://GitHub.com/cozmoc/LeapMotion-Matter.JS

How would I go about including the Leap library when using Processing.js to put the sketch on the web?

When I try, I get this:

Uncaught ReferenceError: LeapMotion is not defined at Processing.Processing.setup (eval at attach (processing.js:885), :12:3) at executeSketch (processing.js:21571) at new Processing.Processing (processing.js:21598) at callback (processing.js:964) at XMLHttpRequest.xhr.onreadystatechange (processing.js:943)

Which to me looks like it fails to load the library, and thus stops loading the rest of the sketch due to an error. Can anyone help me out?

I'm new to processing, and I've been struggling with removing the text when I deploy the sketch (I tried commenting out all the print lines but I don't think that's correct). The sketch works beautifully, I'm simply trying to remove the text and modify the hand design... I will be using this with wekinator.

I tired commenting out all portions of the code such as (println("Sent finger names" + n). I've also tried removing text in the skech- but the text still appears when I deploy the sketch. Perhaps there is something I'm not quite getting. Any guidance would be greatly appreciated.

Also the formatting of the markdown in this post is buggy, so here is the githubpage I am referencing: https://github.com/fiebrink1/wekinator_examples/blob/master/inputs/LeapMotion/LeapMotionViaProcessing/LeapMotion_Fingertips_15Inputs/LeapMotion_Fingertips_15Inputs.pde

` //adapted from https://github.com/nok/leap-motion-processing/blob/master/examples/e1_basic/e1_basic.pde //Sends 15 features ((x,y,z) tip of each finger) to Wekinator // sends to port 6448 using /wek/inputs message

import de.voidplus.leapmotion.*;

import oscP5.*; import netP5.*;

int num=0; OscP5 oscP5; NetAddress dest; LeapMotion leap; int numFound = 0;

float[] features = new float[15];

void setup() { size(800, 500, OPENGL); background(200, 231, 255 ); // ...

/* start oscP5, listening for incoming messages at port 12000 */ oscP5 = new OscP5(this,9000); dest = new NetAddress("127.0.0.1",6448);

leap = new LeapMotion(this); sendInputNames(); }

void draw() { background(200, 231, 255); // ... int fps = leap.getFrameRate();

// ========= HANDS ========= numFound = 0; for (Hand hand : leap.getHands ()) { numFound++; // ----- BASICS -----

//int     hand_id          = hand.getId();
//PVector hand_position    = hand.getPosition();
//PVector hand_stabilized  = hand.getStabilizedPosition();
//PVector hand_direction   = hand.getDirection();
//PVector hand_dynamics    = hand.getDynamics();
//float   hand_roll        = hand.getRoll();
//float   hand_pitch       = hand.getPitch();
//float   hand_yaw         = hand.getYaw();
//boolean hand_is_left     = hand.isLeft();
//boolean hand_is_right    = hand.isRight();
//float   hand_grab        = hand.getGrabStrength();
//float   hand_pinch       = hand.getPinchStrength();
//float   hand_time        = hand.getTimeVisible();
//PVector sphere_position  = hand.getSpherePosition();
//float   sphere_radius    = hand.getSphereRadius();

// ----- SPECIFIC FINGER -----

//Finger  finger_thumb     = hand.getThumb();
//// or                      hand.getFinger("thumb");
//// or                      hand.getFinger(0);

//Finger  finger_index     = hand.getIndexFinger();
//// or                      hand.getFinger("index");
//// or                      hand.getFinger(1);

//Finger  finger_middle    = hand.getMiddleFinger();
//// or                      hand.getFinger("middle");
//// or                      hand.getFinger(2);

//Finger  finger_ring      = hand.getRingFinger();
//// or                      hand.getFinger("ring");
//// or                      hand.getFinger(3);

//Finger  finger_pink      = hand.getPinkyFinger();
//// or                      hand.getFinger("pinky");
//// or                      hand.getFinger(4);        

// ----- DRAWING -----

hand.draw();
// hand.drawSphere();

// ========= ARM =========

if (hand.hasArm()) {
  Arm     arm               = hand.getArm();
  float   arm_width         = arm.getWidth();
  PVector arm_wrist_pos     = arm.getWristPosition();
  PVector arm_elbow_pos     = arm.getElbowPosition();
}

// ========= FINGERS =========

for (Finger finger : hand.getFingers()) {
  // Alternatives:
  // hand.getOutstrechtedFingers();
  // hand.getOutstrechtedFingersByAngle();

  // ----- BASICS -----

  int     finger_id         = finger.getId();
  PVector finger_position   = finger.getPosition();
  PVector finger_stabilized = finger.getStabilizedPosition();
  PVector finger_velocity   = finger.getVelocity();
  PVector finger_direction  = finger.getDirection();
  float   finger_time       = finger.getTimeVisible();


  // ----- SPECIFIC FINGER -----

  switch(finger.getType()) {
  case 0:
    // System.out.println("thumb");
    PVector pos = finger.getPosition();
    features[0] = pos.x;
    features[1] = pos.y;
    features[2] = pos.z;

    break;
  case 1:
    // System.out.println("index");
            pos = finger.getPosition();
    features[3] = pos.x;
    features[4] = pos.y;
    features[5] = pos.z;
    break;
  case 2:
    // System.out.println("middle");
            pos = finger.getPosition();
    features[6] = pos.x;
    features[7] = pos.y;
    features[8] = pos.z;
    break;
  case 3:
    // System.out.println("ring");
     pos = finger.getPosition();
    features[9] = pos.x;
    features[10] = pos.y;
    features[11] = pos.z;
    break;
  case 4:
    // System.out.println("pinky");
     pos = finger.getPosition();
    features[12] = pos.x;
    features[13] = pos.y;
    features[14] = pos.z;
    break;
  }

  // ----- SPECIFIC BONE -----

  Bone    bone_distal       = finger.getDistalBone();
  // or                       finger.get("distal");
  // or                       finger.getBone(0);

  Bone    bone_intermediate = finger.getIntermediateBone();
  // or                       finger.get("intermediate");
  // or                       finger.getBone(1);

  Bone    bone_proximal     = finger.getProximalBone();
  // or                       finger.get("proximal");
  // or                       finger.getBone(2);

  Bone    bone_metacarpal   = finger.getMetacarpalBone();
  // or                       finger.get("metacarpal");
  // or                       finger.getBone(3);

  // ----- DRAWING -----

  // finger.draw(); // = drawLines()+drawJoints()
  // finger.drawLines();
  // finger.drawJoints();

  // ----- TOUCH EMULATION -----

  int     touch_zone        = finger.getTouchZone();
  float   touch_distance    = finger.getTouchDistance();

  switch(touch_zone) {
  case -1: // None
    break;
  case 0: // Hovering
    // println("Hovering (#"+finger_id+"): "+touch_distance);
    break;
  case 1: // Touching
    // println("Touching (#"+finger_id+")");
    break;
  }
}

// ========= TOOLS =========

for (Tool tool : hand.getTools ()) {


  // ----- BASICS -----

  int     tool_id           = tool.getId();
  PVector tool_position     = tool.getPosition();
  PVector tool_stabilized   = tool.getStabilizedPosition();
  PVector tool_velocity     = tool.getVelocity();
  PVector tool_direction    = tool.getDirection();
  float   tool_time         = tool.getTimeVisible();


  // ----- DRAWING -----

  // tool.draw();


  // ----- TOUCH EMULATION -----

  int     touch_zone        = tool.getTouchZone();
  float   touch_distance    = tool.getTouchDistance();

  switch(touch_zone) {
  case -1: // None
    break;
  case 0: // Hovering
    // println("Hovering (#"+tool_id+"): "+touch_distance);
    break;
  case 1: // Touching
    // println("Touching (#"+tool_id+")");
    break;
  }
}

}

// ========= DEVICES =========

for (Device device : leap.getDevices ()) { float device_horizontal_view_angle = device.getHorizontalViewAngle(); float device_verical_view_angle = device.getVerticalViewAngle(); float device_range = device.getRange(); }

// =========== OSC ============ if (num % 3 == 0) { sendOsc(); } num++; }

// ========= CALLBACKS =========

void leapOnInit() { // println("Leap Motion Init"); } void leapOnConnect() { // println("Leap Motion Connect"); } void leapOnFrame() { // println("Leap Motion Frame"); } void leapOnDisconnect() { // println("Leap Motion Disconnect"); } void leapOnExit() { // println("Leap Motion Exit"); }

//====== OSC SEND ====== void sendOsc() { OscMessage msg = new OscMessage("/wek/inputs"); if (numFound > 0) { for (int i = 0; i < features.length; i++) { msg.add(features[i]); } } else { for (int i = 0; i < features.length; i++) { msg.add(0.); } } oscP5.send(msg, dest); }

void sendInputNames() { OscMessage msg = new OscMessage("/wekinator/control/setInputNames"); String[] fingerNames = {"smile", "index", "middle", "ring", "pinky"}; String coordinates[] = {"_x", "_y", "_z"}; int n = 0; for (int i = 0; i < fingerNames.length; i++) { for (int j = 0; j< coordinates.length; j++) { msg.add(fingerNames[i] + coordinates[j]); n++; } } oscP5.send(msg, dest); println("Sent finger names" + n); }`

yes, because I've set times for my looped code, so I want after my leapmotion code ends, my original code continues where it should be.

for example, the original code is led light up for five times in 10 sec, so when my leapmotion code triggered and last for 3sec, I want the last led ends at 10sec instead of 13 sec.

hope its clear and sorry for making you misunderstanding :(

Something is not clear:

my original looped code don't pause or stop, just covered by the Leapmotion code.

So you have both codes merged already? But the n you say:

I want to put these code together as one

Which code are you referring to?

So, processing handles leapMotion and Processing drives the arduino. If the arduino receives data from leapMotion (via Processing) the arduino should do both the original code and the interactive(leapMotion) code?

Kf

First, Sorry for my bad English,

hello, I've post about this several month ago, but as I continued my project, the more I need to figure out.

I'm doing an installation which is looped/interactive , when there's no hand detected by Leap Motion, my installation only display the looped code which is very simple delay code, (Arduino only) but if there's hand detect by Leap Motion, then convert to the code which I wrote for interactive (Processing+Arduino+leapmotion), but as the interactive part start, I want my original looped code still running,

for example , {A} is my original looped display, {B} is Leap motion interactive code,

{A}------(hand detected)----------------{hand removed}-----------------{A}
                                       {B}------------{B} 

my original looped code don't pause or stop, just covered by the Leapmotion code.

and here is my looped code (Arduino, nothing special, only delay) :

int ledPin = 1;
int ledPin2 = 2;
int ledPin3 = 3;
int ledPin4 = 4;
int ledPin5 = 5;
int ledPin6 = 6;
int ledPin7 = 7;
int ledPin8 = 8;
int ledPin9 = 9;
int ledPin10 = 10;
int ledPin11 = 11;
int ledPin12 = 12;
int ledPin13 = 13;
int ledPin14 = 14;

int laser = 15;
int laser2 = 16;
int laser3 = 17;
int laser4 = 18;
int laser5 = 19;
int laser6 = 20;
int laser7 = 21;
int laser8 = 22;



void setup() {

  pinMode(ledPin, OUTPUT);
  pinMode(ledPin2, OUTPUT);
  pinMode(ledPin3, OUTPUT);
  pinMode(ledPin4, OUTPUT);
  pinMode(ledPin5, OUTPUT);
  pinMode(ledPin6, OUTPUT);
  pinMode(ledPin7, OUTPUT);
  pinMode(ledPin8, OUTPUT);
  pinMode(ledPin9, OUTPUT);
  pinMode(ledPin10, OUTPUT);
  pinMode(ledPin11, OUTPUT);
  pinMode(ledPin12, OUTPUT);
  pinMode(ledPin13, OUTPUT);
  pinMode(ledPin14, OUTPUT);

  pinMode(laser, OUTPUT);
  pinMode(laser2, OUTPUT);
  pinMode(laser3, OUTPUT);
  pinMode(laser4, OUTPUT);
  pinMode(laser5, OUTPUT);
  pinMode(laser6, OUTPUT);
  pinMode(laser7, OUTPUT);
  pinMode(laser8, OUTPUT);

}

void loop() {

  // laser part 01



  delay(15000);
  digitalWrite(laser, HIGH);
  delay(1500);
  digitalWrite(laser, LOW);
  delay(6500);
  digitalWrite(laser3, HIGH);
  delay(1500);
  digitalWrite(laser3, LOW);
  delay(6500);
  digitalWrite(laser5, HIGH);
  delay(1500);
  digitalWrite(laser5, LOW);
  delay(6500);
  digitalWrite(laser7, HIGH);
  delay(1500);
  digitalWrite(laser7, LOW);
  delay(3000);
  digitalWrite(laser2, HIGH);
  delay(700);
  digitalWrite(laser2, LOW);
  delay(3000);
  digitalWrite(laser4, HIGH);
  delay(700);
  digitalWrite(laser4, LOW);
  delay(3000);
  digitalWrite(laser6, HIGH);
  delay(700);
  digitalWrite(laser6, LOW);
  delay(3000);
  digitalWrite(laser8, HIGH);
  delay(700);
  digitalWrite(laser8, LOW);



  delay(34000);
  digitalWrite(ledPin, HIGH);
  digitalWrite(ledPin2, HIGH);
  digitalWrite(ledPin3, HIGH);
  digitalWrite(ledPin4, HIGH);
  digitalWrite(ledPin5, HIGH);
  digitalWrite(ledPin6, HIGH);
  digitalWrite(ledPin7, HIGH);
  delay(500);
  digitalWrite(ledPin, LOW);
  digitalWrite(ledPin2, LOW);
  digitalWrite(ledPin3, LOW);
  digitalWrite(ledPin4, LOW);
  digitalWrite(ledPin5, LOW);
  digitalWrite(ledPin6, LOW);
  digitalWrite(ledPin7, LOW);
  delay(500);

  digitalWrite(laser, HIGH);
  delay(500);
  digitalWrite(laser, LOW);

  delay(500);
  digitalWrite(ledPin8, HIGH);
  digitalWrite(ledPin9, HIGH);
  digitalWrite(ledPin10, HIGH);
  digitalWrite(ledPin11, HIGH);
  digitalWrite(ledPin12, HIGH);
  digitalWrite(ledPin13, HIGH);
  digitalWrite(ledPin14, HIGH);
  delay(500);
  digitalWrite(ledPin8, LOW);
  digitalWrite(ledPin9, LOW);
  digitalWrite(ledPin10, LOW);
  digitalWrite(ledPin11, LOW);
  digitalWrite(ledPin12, LOW);
  digitalWrite(ledPin13, LOW);
  digitalWrite(ledPin14, LOW);
  delay(1500);
  digitalWrite(ledPin, HIGH);
  digitalWrite(ledPin2, HIGH);
  digitalWrite(ledPin3, HIGH);
  digitalWrite(ledPin4, HIGH);
  digitalWrite(ledPin5, HIGH);
  digitalWrite(ledPin6, HIGH);
  digitalWrite(ledPin7, HIGH);
  delay(500);
  digitalWrite(ledPin, LOW);
  digitalWrite(ledPin2, LOW);
  digitalWrite(ledPin3, LOW);
  digitalWrite(ledPin4, LOW);
  digitalWrite(ledPin5, LOW);
  digitalWrite(ledPin6, LOW);
  digitalWrite(ledPin7, LOW);
  delay(500);

  digitalWrite(laser2, HIGH);
  delay(500);
  digitalWrite(laser2, LOW);

  delay(500);

  digitalWrite(ledPin8, HIGH);
  digitalWrite(ledPin9, HIGH);
  digitalWrite(ledPin10, HIGH);
  digitalWrite(ledPin11, HIGH);
  digitalWrite(ledPin12, HIGH);
  digitalWrite(ledPin13, HIGH);
  digitalWrite(ledPin14, HIGH);
  delay(500);
  digitalWrite(ledPin, LOW);
  digitalWrite(ledPin2, LOW);
  digitalWrite(ledPin3, LOW);
  digitalWrite(ledPin4, LOW);
  digitalWrite(ledPin5, LOW);
  digitalWrite(ledPin6, LOW);
  digitalWrite(ledPin7, LOW);
  delay(1500);

  digitalWrite(laser3, HIGH);

  digitalWrite(ledPin, HIGH);
  digitalWrite(ledPin2, HIGH);
  digitalWrite(ledPin3, HIGH);
  digitalWrite(ledPin4, HIGH);
  digitalWrite(ledPin5, HIGH);
  digitalWrite(ledPin6, HIGH);
  digitalWrite(ledPin7, HIGH);
  digitalWrite(ledPin8, HIGH);
  digitalWrite(ledPin9, HIGH);
  digitalWrite(ledPin10, HIGH);
  digitalWrite(ledPin11, HIGH);
  digitalWrite(ledPin12, HIGH);
  digitalWrite(ledPin13, HIGH);
  digitalWrite(ledPin14, HIGH);
  delay(500);

  digitalWrite(laser3, LOW);

  digitalWrite(ledPin, LOW);
  digitalWrite(ledPin2, LOW);
  digitalWrite(ledPin3, LOW);
  digitalWrite(ledPin4, LOW);
  digitalWrite(ledPin5, LOW);
  digitalWrite(ledPin6, LOW);
  digitalWrite(ledPin7, LOW);
  digitalWrite(ledPin8, LOW);
  digitalWrite(ledPin9, LOW);
  digitalWrite(ledPin10, LOW);
  digitalWrite(ledPin11, LOW);
  digitalWrite(ledPin12, LOW);
  digitalWrite(ledPin13, LOW);
  digitalWrite(ledPin14, LOW);
  delay(500);

  digitalWrite(laser4, HIGH);
  delay(500);
  digitalWrite(laser, LOW);

  delay(500);
  digitalWrite(ledPin, HIGH);
  digitalWrite(ledPin2, HIGH);
  digitalWrite(ledPin3, HIGH);
  digitalWrite(ledPin4, HIGH);
  digitalWrite(ledPin5, HIGH);
  digitalWrite(ledPin6, HIGH);
  digitalWrite(ledPin7, HIGH);
  digitalWrite(ledPin8, HIGH);
  digitalWrite(ledPin9, HIGH);
  digitalWrite(ledPin10, HIGH);
  digitalWrite(ledPin11, HIGH);
  digitalWrite(ledPin12, HIGH);
  digitalWrite(ledPin13, HIGH);
  digitalWrite(ledPin14, HIGH);
  delay(500);

}

and this is my interactive code (Arduino) :

int val; // Data received from the serial port
int ledPin = 13; // Set the pin to digital I/O 13
int ledPin2 = 12;
int ledPin3 = 11;
int ledPin4 = 10;
int ledPin5 = 9;
int ledPin6 = 8;

void setup() {
  pinMode(ledPin, OUTPUT); // Set pin as OUTPUT
  pinMode(ledPin2, OUTPUT);
  pinMode(ledPin3, OUTPUT);
  pinMode(ledPin4, OUTPUT);
  pinMode(ledPin5, OUTPUT);
  pinMode(ledPin6, OUTPUT);
  Serial.begin(9600); // Start serial communication at 9600 bps
}
void loop() {
  if (Serial.available())
  { // If data is available to read,
    val = Serial.read(); // read it and store it in val
  }
  if (val == 1)
  { // If 1 was received
    digitalWrite(ledPin, HIGH);
    digitalWrite(ledPin2, HIGH);
    digitalWrite(ledPin3, HIGH);
    digitalWrite(ledPin4, HIGH);
    digitalWrite(ledPin5, HIGH);
    digitalWrite(ledPin6, HIGH);// turn the LED on
  } else if (val == 0) {
    digitalWrite(ledPin, LOW); // otherwise turn it off
    digitalWrite(ledPin2, LOW);
    digitalWrite(ledPin3, LOW);
    digitalWrite(ledPin4, LOW);
    digitalWrite(ledPin5, LOW);
    digitalWrite(ledPin6, LOW);
  }
  delay(10); // Wait 10 milliseconds for next reading
} 

this is my interactive code (Processing):

import de.voidplus.leapmotion.*;
import processing.serial.*;

Serial myPort;

LeapMotion leap ; 

void setup () {
  size(800, 500);
  background(255);
  leap = new LeapMotion(this);
  String portName = Serial.list()[1]; //change the 0 to a 1 or 2 etc. to match your port
  myPort = new Serial(this, portName, 9600);
}
boolean rightHand,leftHand;
float posy ;
float posx ;
void draw() {

    int number=0;
  for (Hand hand : leap.getHands() ) {

    rightHand = hand.isRight();
    posy = hand.getPosition().y;

    number=hand.countFingers();

    hand.draw();
    //println("Hand palm position: "+hand.getPalmPosition());
  }

  //println(number);
  println(posy);

  if(rightHand){
    if ((number!= 0)&&(posy < 450 && posy > 350))
    {
      timer(128);        //send a 1
      println("1");
    } else if ((number!= 0)&&(posy < 350 && posy > 250))
    {
      timer(32);
      println("2");
    } else if ((number!= 0)&&(posy < 250 && posy > 150))
    {
      timer(8);
      println("3");
    } else if ((number!= 0)&&(posy < 150 && posy > -120))
    {
      timer(4);
      println("4");
    } else 
    {                           //otherwise
      myPort.write(0);          //send a 0
    }
  }else if(leftHand){


  }
  delay(30);
}



void timer(int step) {
  if (frameCount % step ==0) {
    myPort.write(1);
  } else if (frameCount % step == step/2) {
    myPort.write(0);
  }
}

I want to put these code together as one, please help me, thank you for your reading and apologize for my bad english again.

Now I want to make when use key play the music if the leap motion move to right square will output Right，contrary be wrong. How can I do that?

Here is my code.

import com.onformative.leap.*;
import com.leapmotion.leap.Finger;
import ddf.minim.*;
LeapMotionP5 leap;
Minim minim;
/////////////////// here is + music
AudioPlayer Bongo;
AudioPlayer cello;
AudioPlayer drumset;
AudioPlayer flute;
AudioPlayer Guitar;
AudioPlayer handbells;
AudioPlayer harmonica;
AudioPlayer Harp;
AudioPlayer marimba;
AudioPlayer oboe;
AudioPlayer piano;
AudioPlayer saxphone;
AudioPlayer Trombone;
AudioPlayer trumpet;
AudioPlayer Tuba;
AudioPlayer violin;

void setup()
{
  size(1280, 720, P3D);
  minim = new Minim(this);
  leap = new LeapMotionP5(this);

  Bongo = minim.loadFile("Bongo.wav");
  cello = minim.loadFile("cello.wav");
  drumset = minim.loadFile("drumset.wav");
  flute = minim.loadFile("flute.wav");
  Guitar = minim.loadFile("Guitar.wav");
  handbells = minim.loadFile("handbells.wav");
  harmonica = minim.loadFile("harmonica.wav");
  Harp = minim.loadFile("Harp.wav");
  marimba = minim.loadFile("marimba.wav");
  oboe = minim.loadFile("oboe.wav");
  piano = minim.loadFile("piano.wav");
  saxphone = minim.loadFile("saxphone.wav");
  Trombone = minim.loadFile("Trombone.wav");
  trumpet = minim.loadFile("trumpet.wav");
  Tuba = minim.loadFile("Tuba.wav");
  violin = minim.loadFile("violin.wav");
}

void draw() {                                           
  background(200, 200, 200);
  text("Press r to reset.", 10, 20 );
  fill (255);
  for (Finger f : leap.getFingerList()) {
    PVector position = leap.getTip(leap.getFinger(1));

    if (position.x > 250 && position.x <400 && position.y > 150 && position.y < 250)
    {
      Bongo.play();
    } else
      Bongo.pause();

    if (position.x > 650 && position.x <800 && position.y > 150 && position.y < 250) {
      cello.play();
    } else
      cello.pause();

    if (position.x > 1050 && position.x <1200 && position.y > 150 && position.y < 250) {
      drumset.play();
    } else
      drumset.pause();

    if (position.x > 1450 && position.x <1600 && position.y > 150 && position.y < 250) {
      flute.play();
    } else
      flute.pause();

    if (position.x > 250 && position.x <400 && position.y > 350 && position.y < 500) {
      Guitar.play();
    } else
      Guitar.pause();

    if (position.x > 650 && position.x <800 && position.y > 350 && position.y < 500) {
      handbells.play();
    } else
      handbells.pause();

    if (position.x > 1050 && position.x <1200 && position.y > 350 && position.y < 500) {
      harmonica.play();
    } else
      harmonica.pause();

    if (position.x > 1450 && position.x <1600 && position.y > 350 && position.y < 500) {
      Harp.play();
    } else
      Harp.pause();

    if (position.x > 250 && position.x <400 && position.y > 550 && position.y < 750) {
      marimba.play();
    } else
      marimba.pause();

    if (position.x > 650 && position.x <800 && position.y > 550 && position.y < 750) {
      oboe.play();
    } else
      oboe.pause();

    if (position.x > 1050 && position.x <1200 && position.y > 550 && position.y < 750) {
      piano.play();
    } else
      piano.pause();

    if (position.x > 1450 && position.x <1600 && position.y > 550 && position.y < 750) {
      saxphone.play();
    } else
      saxphone.pause();

    if (position.x > 250 && position.x <400 && position.y > 750 && position.y < 950) {
      Trombone.play();
    } else
      Trombone.pause();

    if (position.x > 650 && position.x <800 && position.y > 750 && position.y < 950) {
      trumpet.play();
    } else
      trumpet.pause();

    if (position.x > 1050 && position.x <1200 && position.y > 750 && position.y < 950) {
      Tuba.play();
    } else
      Tuba.pause();

    if (position.x > 1450 && position.x <1600 && position.y > 750 && position.y < 950) {
      violin.play();
    } else
      violin.pause();

    stroke(0);
    strokeWeight(5);
    ellipse(position.x, position.y, 20, 20);
  }

  // Here is draw 16 square
  stroke(0, 0, 0);                                    // square
  strokeWeight(5);
  fill(255, 255, 255);
  rect(250, 100, 150, 150);

  fill(255, 255, 255);
  rect(650, 100, 150, 150);

  fill(255, 255, 255);
  rect(1050, 100, 150, 150);

  fill(255, 255, 255);
  rect(1450, 100, 150, 150);
  //-----------------------------------------------------------
  fill(255, 255, 255);
  rect(250, 350, 150, 150);

  fill(255, 255, 255);
  rect(650, 350, 150, 150);

  fill(255, 255, 255);
  rect(1050, 350, 150, 150);

  fill(255, 255, 255);
  rect(1450, 350, 150, 150);
  //-----------------------------------------------------------
  fill(255, 255, 255);
  rect(250, 600, 150, 150);

  fill(255, 255, 255);
  rect(650, 600, 150, 150);

  fill(255, 255, 255);
  rect(1050, 600, 150, 150);

  fill(255, 255, 255);
  rect(1450, 600, 150, 150);
  //-----------------------------------------------------------

  fill(255, 255, 255);
  rect(250, 850, 150, 150);

  fill(255, 255, 255);
  rect(650, 850, 150, 150);

  fill(255, 255, 255);
  rect(1050, 850, 150, 150);

  fill(255, 255, 255);
  rect(1450, 850, 150, 150);
}

void keyPressed()
{
  if ( key == 'r' ) {
    Bongo.rewind();
    cello.rewind();
    drumset.rewind();
    flute.rewind();
    Guitar.rewind();
    handbells.rewind();
    harmonica.rewind();
    Harp.rewind();
    marimba.rewind();
    oboe.rewind();
    piano.rewind();
    saxphone.rewind();
    Trombone.rewind();
    trumpet.rewind();
    Tuba.rewind();
    violin.rewind();

    if ( key == 'z' ) {
      Bongo.play();
    } else {
      Bongo.pause();
    }
    if ( key == 'x' ) {
      cello.play();
    } else {
      cello.pause();
    }
    if ( key == 'c' ) {
      drumset.play();
    } else {
      drumset.pause();
    }
    if ( key == 'v' ) {
      flute.play();
    } else {
      flute.pause();
    }
    if ( key == 'b' ) {
      Guitar.play();
    } else {
      Guitar.pause();
    }
    if ( key == 'n' ) {
      handbells.play();
    } else {
      handbells.pause();
    }
    if ( key == 'm' ) {
      harmonica.play();
    } else {
      harmonica.pause();
    }
    if ( key == 'a' ) {
      Harp.play();
    } else {
      Harp.pause();
    }
    if ( key == 's' ) {
      marimba.play();
    } else {
      marimba.pause();
    }
    if ( key == 'd' ) {
      oboe.play();
    } else {
      oboe.pause();
    }
    if ( key == 'f' ) {
      piano.play();
    } else {
      piano.pause();
    }
    if ( key == 'g' ) {
      saxphone.play();
    } else {
      saxphone.pause();
    }
    if ( key == 'h' ) {
      Trombone.play();
    } else {
      Trombone.pause();
    }
    if ( key == 'j' ) {
      trumpet.play();
    } else {
      trumpet.pause();
    }
    if ( key == 'k' ) {
      Tuba.play();
    } else {
      Tuba.pause();
    }  
    if ( key == 'l' ) {
      violin.play();
    } else {
      violin.pause();
    }
  }
}

My Program is want to make a game When the leap motion move to the square will be playing music. But now I cannot play the music. So i want to know why and how to fix it. Thanks

Here is my code......

    import com.onformative.leap.*;
    import com.leapmotion.leap.Finger;
    import ddf.minim.*;
    LeapMotionP5 leap;
    Minim minim;
    /////////////////// here is + music
    AudioPlayer Bongo;
    AudioPlayer cello;
    AudioPlayer drumset;
    AudioPlayer flute;
    AudioPlayer Guitar;
    AudioPlayer handbells;
    AudioPlayer harmonica;
    AudioPlayer Harp;
    AudioPlayer marimba;
    AudioPlayer oboe;
    AudioPlayer piano;
    AudioPlayer saxphone;
    AudioPlayer Trombone;
    AudioPlayer trumpet;
    AudioPlayer Tuba;
    AudioPlayer violin;

    void setup()
    {
      size(1280, 720, P3D);
      minim = new Minim(this);
      leap = new LeapMotionP5(this);

      Bongo = minim.loadFile("Bongo.wav");
      cello = minim.loadFile("cello.wav");
      drumset = minim.loadFile("drumset.wav");
      flute = minim.loadFile("flute.wav");
      Guitar = minim.loadFile("Guitar.wav");
      handbells = minim.loadFile("handbells.wav");
      harmonica = minim.loadFile("harmonica.wav");
      Harp = minim.loadFile("Harp.wav");
      marimba = minim.loadFile("marimba.wav");
      oboe = minim.loadFile("oboe.wav");
      piano = minim.loadFile("piano.wav");
      saxphone = minim.loadFile("saxphone.wav");
      Trombone = minim.loadFile("Trombone.wav");
      trumpet = minim.loadFile("trumpet.wav");
      Tuba = minim.loadFile("Tuba.wav");
      violin = minim.loadFile("violin.wav");
    }

    void draw() {                                           
      background(200, 200, 200);
      text("Press r to reset.", 10, 20 );
      fill (255);
      for (Finger f : leap.getFingerList()) {
        PVector fingerPos = leap.getTip(leap.getFinger(2));

        if (fingerPos.x > 100 && fingerPos.x <180 && fingerPos.y > 100 && fingerPos.y < 180) {
          Bongo.play();
          Bongo.rewind( );
        } else
          Bongo.pause();

        if (fingerPos.x > 650 && fingerPos.x <800 && fingerPos.y > 250 && fingerPos.y < 150) {
          cello.play();
          cello.rewind( );
        } else
          cello.pause();

        if (fingerPos.x > 1050 && fingerPos.x <1200 && fingerPos.y > 250 && fingerPos.y < 150) {
          drumset.play();
        } else
          drumset.pause();

        if (fingerPos.x > 1450 && fingerPos.x <1600 && fingerPos.y > 250 && fingerPos.y < 150) {
          flute.play();
        } else
          flute.pause();

        if (fingerPos.x > 250 && fingerPos.x <400 && fingerPos.y > 350 && fingerPos.y < 150) {
          Guitar.play();
        } else
          Guitar.pause();

        if (fingerPos.x > 650 && fingerPos.x <800 && fingerPos.y > 350 && fingerPos.y < 150) {
          handbells.play();
        } else
          handbells.pause();

        if (fingerPos.x > 1050 && fingerPos.x <1200 && fingerPos.y > 350 && fingerPos.y < 150) {
          harmonica.play();
        } else
          harmonica.pause();

        if (fingerPos.x > 1450 && fingerPos.x <1600 && fingerPos.y > 350 && fingerPos.y < 150) {
          Harp.play();
        } else
          Harp.pause();

        if (fingerPos.x > 250 && fingerPos.x <400 && fingerPos.y > 600 && fingerPos.y < 150) {
          marimba.play();
        } else
          marimba.pause();

        if (fingerPos.x > 650 && fingerPos.x <800 && fingerPos.y > 600 && fingerPos.y < 150) {
          oboe.play();
        } else
          oboe.pause();

        if (fingerPos.x > 1050 && fingerPos.x <1200 && fingerPos.y > 600 && fingerPos.y < 150) {
          piano.play();
        } else
          piano.pause();

        if (fingerPos.x > 1450 && fingerPos.x <1600 && fingerPos.y > 600 && fingerPos.y < 150) {
          saxphone.play();
        } else
          saxphone.pause();

        if (fingerPos.x > 250 && fingerPos.x <400 && fingerPos.y > 850 && fingerPos.y < 150) {
          Trombone.play();
        } else
          Trombone.pause();

        if (fingerPos.x > 650 && fingerPos.x <800 && fingerPos.y > 850 && fingerPos.y < 150) {
          trumpet.play();
        } else
          trumpet.pause();

        if (fingerPos.x > 1050 && fingerPos.x <1200 && fingerPos.y > 850 && fingerPos.y < 150) {
          Tuba.play();
        } else
          Tuba.pause();

        if (fingerPos.x > 1450 && fingerPos.x <1600 && fingerPos.y > 850 && fingerPos.y < 150) {
          violin.play();
        } else
          violin.pause();

        stroke(0);
        strokeWeight(5);
        ellipse(fingerPos.x, fingerPos.y, 20, 20);
      }

      // Here is draw 16 square
      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(250, 100, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(650, 100, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1050, 100, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1450, 100, 150, 150);
      //-----------------------------------------------------------
      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(250, 350, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(650, 350, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1050, 350, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1450, 350, 150, 150);
      //-----------------------------------------------------------
      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(250, 600, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(650, 600, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1050, 600, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1450, 600, 150, 150);
      //-----------------------------------------------------------
      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(250, 850, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(650, 850, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1050, 850, 150, 150);

      stroke(0, 0, 0);                                    // square
      strokeWeight(5);
      fill(255, 255, 255);
      rect(1450, 850, 150, 150);
    }

    void keyPressed()
    {
      if ( key == 'r' ) Bongo.rewind();
      if ( key == 'r' ) cello.rewind();
      if ( key == 'r' ) drumset.rewind();
      if ( key == 'r' ) flute.rewind();
      if ( key == 'r' ) Guitar.rewind();
      if ( key == 'r' ) handbells.rewind();
      if ( key == 'r' ) harmonica.rewind();
      if ( key == 'r' ) Harp.rewind();
      if ( key == 'r' ) marimba.rewind();
      if ( key == 'r' ) oboe.rewind();
      if ( key == 'r' ) piano.rewind();
      if ( key == 'r' ) saxphone.rewind();
      if ( key == 'r' ) Trombone.rewind();
      if ( key == 'r' ) trumpet.rewind();
      if ( key == 'r' ) Tuba.rewind();
      if ( key == 'r' ) violin.rewind();

      if ( key == 'z' ) {
        Bongo.play();
      } else {
        Bongo.pause();
      }
      if ( key == 'x' ) {
        cello.play();
      } else {
        cello.pause();
      }
      if ( key == 'c' ) {
        drumset.play();
      } else {
        drumset.pause();
      }
      if ( key == 'v' ) {
        flute.play();
      } else {
        flute.pause();
      }
      if ( key == 'b' ) {
        Guitar.play();
      } else {
        Guitar.pause();
      }
      if ( key == 'n' ) {
        handbells.play();
      } else {
        handbells.pause();
      }
      if ( key == 'm' ) {
        harmonica.play();
      } else {
        harmonica.pause();
      }
      if ( key == 'a' ) {
        Harp.play();
      } else {
        Harp.pause();
      }
      if ( key == 's' ) {
        marimba.play();
      } else {
        marimba.pause();
      }
      if ( key == 'd' ) {
        oboe.play();
      } else {
        oboe.pause();
      }
      if ( key == 'f' ) {
        piano.play();
      } else {
        piano.pause();
      }
      if ( key == 'g' ) {
        saxphone.play();
      } else {
        saxphone.pause();
      }
      if ( key == 'h' ) {
        Trombone.play();
      } else {
        Trombone.pause();
      }
      if ( key == 'j' ) {
        trumpet.play();
      } else {
        trumpet.pause();
      }
      if ( key == 'k' ) {
        Tuba.play();
      } else {
        Tuba.pause();
      }  
      if ( key == 'l' ) {
        violin.play();
      } else {
        violin.pause();
      }
    }

You can explore prev posts:

https://forum.processing.org/two/search?Search=leapmotion
https://forum.processing.org/two/search?Search=arduino

Kf

No problem. This is how our code works, it's a bit of an adaption, using the example as inspiration, but build from scratch.

import de.voidplus.leapmotion.*; import processing.sound.*;

LeapMotion leap; SinOsc sine; Hand leftHand; Hand rightHand;

float freq; float amp; float pos; PVector hL; PVector hR;

void setup() { size(displayWidth, 700);

leap = new LeapMotion(this); sine = new SinOsc(this);

//Start the Sine Oscillator. sine.play(); sine.amp(amp); }

void draw() { background(255); smooth();

for (Hand hand : leap.getHands()) { boolean handIsLeft = hand.isLeft(); boolean handIsRight = hand.isRight();

leftHand = leap.getLeftHand();
hL   = leftHand.getPosition();
rightHand = leap.getRightHand();
hR   = rightHand.getPosition();
//----------------------- LEFT HAND ----------------------------------

if (handIsLeft) {
  text("leftHand-X=" +hL.x, 10, 15); //shows hL.x in sketch
  text("leftHand-Y=" +hL.y, 10, 30); //shows hL.y in sketch
  text("leftHand-Z=" +hL.z, 10, 45); //shows hL.z in sketch
  text("volume(in %)= " +nf(amp*100, 1, -3), 10, 60);

  leftHand.draw();
  // Map leftHandY van 0.0 to 1.0 voor amplitude (volume)
  amp=map(hL.y, 0, height, 1.0, 0.0);
} 

sine.amp(amp);

//----------------------- RIGHT HAND ----------------------------------  
if (handIsRight) {
  text("rightHand-X= " +hR.x, width-160, 15); //shows hL.x in sketch
  text("rightHand-Y= " +hR.y, width-160, 30); //shows hL.y in sketch
  text("rightHand-Z= " +hR.z, width-160, 45); //shows hL.z in sketch
  text("freq(in Hz)= " +nf(freq, 3, -3), width-160, 60);

  rightHand.draw();
  // Map rightHandX van 100Hz to 800Hz voor frequency
  freq=map(hR.x, 0, width, 100.0, 800.0);
}
if (!handIsLeft && handIsRight) {
  amp = 0;
}
sine.freq(freq);

} }

The theremin we build using Leap Motion and processing is pretty far, now we are trying to make a visual enhancement accompany the build.

I made the visual part based on a punktierd example called behavior attractor. I basically reversed the process and made it a behavior DEtractor.

Now i'd like to combine the two together. First by adapting the mousx, mousy to righthandx and righthandy. But for some reason it's not working correctly yet using this approach. It might the fault of the code being in the wrong order. Any ideas or suggestions?

Here's the code we are working with (note that both the mouse and hands in leap motion are enabled) (for some reason the forum is not showing the code correctly using the coding raster, so i used the quoting tool but it's still a bit of a mess... excuse me).

import de.voidplus.leapmotion.*; import processing.sound.*; import punktiert.math.Vec; import punktiert.physics.*;

LeapMotion leap; SinOsc sine; Hand leftHand; Hand rightHand;

float freq; float amp; float pos; PVector hL; PVector hR;

int teller = 0; int amount = 100;

// world object VPhysics physics;

// attractor BAttraction attr;

void setup() { size(displayWidth, 700);

leap = new LeapMotion(this); sine = new SinOsc(this);

//Start the Sine Oscillator. sine.play(); sine.amp(amp);

noStroke(); background (#57385c); frameRate (30); smooth();

//set up physics physics = new VPhysics(); physics.setfriction(.4f);

// new AttractionForce: (Vec pos, radius, strength) attr = new BAttraction(new Vec(width * .5f, height * .5f), 400, .01f); physics.addBehavior(attr);

//

// val for arbitrary radius float rad = attr.getRadius()/4; // vector for position Vec pos = new Vec (random(rad, width/2 - rad), random(rad, height/2 - rad)); // create particle (Vec pos, mass, radius) VParticle particle = new VParticle(pos, 9, rad); // add Collision Behavior particle.addBehavior(new BCollision()); // add particle to world physics.addParticle(particle);

for (int i = 1; i < amount; i++) { // val for arbitrary radius float rad2 = random(2, 20); // vector for position Vec pos2 = new Vec(random(rad2, width - rad2), random(rad2, height - rad2)); // create particle (Vec pos, mass, radius) VParticle particle2 = new VParticle(pos2, 8, rad2); // add Collision Behavior particle2.addBehavior(new BCollision()); // add particle to world physics.addParticle(particle2); } }

void draw() { background(255); smooth();

for (Hand hand : leap.getHands()) { boolean handIsLeft = hand.isLeft(); boolean handIsRight = hand.isRight();

leftHand = leap.getLeftHand(); hL = leftHand.getPosition(); rightHand = leap.getRightHand(); hR = rightHand.getPosition(); //----------------------- LEFT HAND ----------------------------------

if (handIsLeft) {
  text("leftHand-X=" +hL.x, 10, 15); //shows hL.x in sketch
  text("leftHand-Y=" +hL.y, 10, 30); //shows hL.y in sketch
  text("leftHand-Z=" +hL.z, 10, 45); //shows hL.z in sketch
  text("volume(in %)= " +nf(amp*100, 1, -3), 10, 60);

  leftHand.draw();
  // Map leftHandY van 0.0 to 1.0 voor amplitude (volume)
  amp=map(hL.y, 0, height, 1.0, 0.0);
} 

sine.amp(amp);

//----------------------- RIGHT HAND ----------------------------------  
if (handIsRight) {
  text("rightHand-X= " +hR.x, width-160, 15); //shows hL.x in sketch
  text("rightHand-Y= " +hR.y, width-160, 30); //shows hL.y in sketch
  text("rightHand-Z= " +hR.z, width-160, 45); //shows hL.z in sketch
  text("freq(in Hz)= " +nf(freq, 3, -3), width-160, 60);

  rightHand.draw();
  // Map rightHandX van 100Hz to 800Hz voor frequency
  freq=map(hR.x, 0, width, 100.0, 800.0);
}
if (!handIsLeft && handIsRight) {
  amp = 0;
}
sine.freq(freq);

} physics.update();

noFill(); stroke(200, 0, 0); // set pos to mousePosition attr.setAttractor(new Vec(hR.x, hR.y)); //ellipse(attr.getAttractor().x, attr.getAttractor().y, attr.getRadius(), attr.getRadius());

noStroke(); fill(0, 125);

teller = 0; for (VParticle p : physics.particles) { println(teller);

if (teller == 0) {
  p.set(mouseX, mouseY);
  ellipse(p.x, p.y, p.getRadius() * 2, p.getRadius() * 2);
  teller++;
}
if (teller == 1);
{
  ellipse(p.x, p.y, p.getRadius() * 2, p.getRadius() * 2);
}

} }

The user experience is a big problem, IMO. I'm not familiar with p5.js and VR, but I have tried to implement a browser-based drawing app using leapmotion, it has a poor performance compared to the native implementation :(

But It makes sense, I like this idea :)

I believe that the leap motion returns a hands list:

https://forum.processing.org/one/topic/leap-motion-tracking-both-hands.html

See also:

http://blog.leapmotion.com/two-handed-interactions-you-have-two-hands-use-them/

We are using the code of someone else https://forum.processing.org/two/discussion/20409/newbie-seeking-help-re-combining-example-code-using-a-leapmotion

What we would like to do is having your left AND your right hand control two seperate sounds and their frequency's. We are thinking of dividing the screen in two halves. And changing everything that's X axis to pitch the frequency into Z axis. And we need to let a seperate code of tone and frequency correspond with the right hand.

I'm not sure where to begin to tackle this problem, we highly appreciate it if there is someone who can give us any advice on where to look to solve this problem :)

how on earth do I set the volume (amp) to 0 when the project starts?

Add the following line between line 24 and 25: sine.amp(amp); However I am not sure if that will do the trick bc I can't run your code and bc it is very likely than when your device starts, even if your hands are not within the interactive space, the values that your device returns could be automatically initialize inside the interactive space. If this is so, then your draw() function will override my previous line and you will have a tone playing with non-zero volume.

I will suggest to approach your problem in a different way. Try using mouse positions in a different sketch where you control the sound bound by different regions in your sketch. Then handle the situation when the mouse leaves your sketch area fast, so the mouse positions are still returning points inside your sketch despite your mouse being outside. An idea to handle this situation is that instead of reacting to absolute pointer position, you react to differential positions aka. "previous MINUS current" position. In the trivial case, if the hand is in the active area of drawing, your hand jitters and introduces differential values. When the differential values are zero, either the hand is very still or the hand has left the active region being monitored by leapMotion.

Kf