We are about to switch to a new forum software. Until then we have removed the registration on this forum.
First up, hello everybody. I'm new, both to the forum, and to processing, but despite my very limited experience, what an awesome program - It's so exciting to tinker with!
I'm using processing 3.2.3, and for now, I'm learning the basics through modifying example code provided via example librarys, which is proving to be a slow but valuable learning experience.
I've now got a little project where I want to combine 2 parts of example code, and I've hit a road block - I simply can't seem to get it to work.
There is a nice SineWave demo in the official sound library (libraries/sound/examples/Oscillators/SineWave) which generates a tone, that tracks your mouse position and changes frequency, amplitude and panning based on your mouse position.
The second is from an example provided as part of the Library item: Leapmotion | Forwards Leap Motion controller events to a processing sketch by Michael Heuer, specifically, the leap_position_example. This example tracks finger tip position provided by the leap, and displays them in a processing window as circular vectors of a random colour.
What I would like to do, is to combine both parts of code - I'd like to bring the SineWave code into the Leap example and then map one of the values provided by the leap (Wrist/palm position ideally - anybody know how to visually display this also?) to SineWave code so that you are able to control the frequency etc. of the sound with you hard movements rather than the mouse. Like a simple theremin!
I'm clearly missing something fundamental, because no matter how I try to format it, I can't get both examples of working in the same processing window. As such, any help would be greatly appreciated - Thank you!
Rather than provide my numerous examples butchered code for you to untangle, here are the two examples of untouched code that I am hoping to use:
SOUND:
import processing.sound.*;
SinOsc sine;
float freq=400;
float amp=0.5;
float pos;
void setup() {
size(800, 600);
background(255);
// Create and start the sine oscillator.
sine = new SinOsc(this);
//Start the Sine Oscillator.
sine.play();
}
void draw() {
// Map mouseY from 0.0 to 1.0 for amplitude
amp=map(mouseY, 0, height, 1.0, 0.0);
sine.amp(amp);
// Map mouseX from 20Hz to 1000Hz for frequency
freq=map(mouseX, 0, width, 80.0, 1000.0);
sine.freq(freq);
// Map mouseX from -1.0 to 1.0 for left to right
pos=map(mouseX, 0, width, -1.0, 1.0);
sine.pan(pos);
}
LEAP:
import java.util.Map;
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ConcurrentHashMap;
import com.leapmotion.leap.Controller;
import com.leapmotion.leap.Finger;
import com.leapmotion.leap.Frame;
import com.leapmotion.leap.Hand;
import com.leapmotion.leap.Tool;
import com.leapmotion.leap.Vector;
import com.leapmotion.leap.processing.LeapMotion;
LeapMotion leapMotion;
ConcurrentMap<Integer, Integer> fingerColors;
ConcurrentMap<Integer, Integer> toolColors;
ConcurrentMap<Integer, Vector> fingerPositions;
ConcurrentMap<Integer, Vector> toolPositions;
void setup()
{
size(800, 600);
background(20);
frameRate(60);
ellipseMode(CENTER);
leapMotion = new LeapMotion(this);
fingerColors = new ConcurrentHashMap<Integer, Integer>();
toolColors = new ConcurrentHashMap<Integer, Integer>();
fingerPositions = new ConcurrentHashMap<Integer, Vector>();
toolPositions = new ConcurrentHashMap<Integer, Vector>();
}
void draw()
{
fill(20);
rect(0, 0, width, height);
for (Map.Entry entry : fingerPositions.entrySet())
{
Integer fingerId = (Integer) entry.getKey();
Vector position = (Vector) entry.getValue();
fill(fingerColors.get(fingerId));
noStroke();
ellipse(leapMotion.leapToSketchX(position.getX()), leapMotion.leapToSketchY(position.getY()), 24.0, 24.0);
}
for (Map.Entry entry : toolPositions.entrySet())
{
Integer toolId = (Integer) entry.getKey();
Vector position = (Vector) entry.getValue();
fill(toolColors.get(toolId));
noStroke();
ellipse(leapMotion.leapToSketchX(position.getX()), leapMotion.leapToSketchY(position.getY()), 24.0, 24.0);
}
}
void onFrame(final Controller controller)
{
Frame frame = controller.frame();
fingerPositions.clear();
for (Finger finger : frame.fingers())
{
int fingerId = finger.id();
color c = color(random(0, 255), random(0, 255), random(0, 255));
fingerColors.putIfAbsent(fingerId, c);
fingerPositions.put(fingerId, finger.tipPosition());
}
toolPositions.clear();
for (Tool tool : frame.tools())
{
int toolId = tool.id();
color c = color(random(0, 255), random(0, 255), random(0, 255));
toolColors.
(toolId, c);
toolPositions.put(toolId, tool.tipPosition());
}
}
Answers
please don't post duplicates.
Thanks for the warm welcome to my first post Koogs!
Of course I didn't intend to cause problems or break any rules, so for future ref. could you explain what the difference between 'ask a question' and 'start a discussion' is? I wasn't sure which was the correct one to use and am simply eager to learn (so I erroneously tried both).
As such, I would still really appreciate a little guidance re. my initial question too, I'm sure it's quite simple for somebody more knowledgeable than I.
you're welcome
combining sketches is theoretically just a case of combining imports and global variable definitions, the setup() section and the draw() sections. rename things that clash (nothing here, i don't think).
these
(and the corresponding y version) suggest to me that they are the leapMotion equivalent of mouseX in the first sketch. so, instead (or additional to) drawing an ellipse at the finger position you can modify the amp / freq / pan of your sine.
the problem is that you have one mouse and potentially 10 fingers. so, er, pick one? and think of a way of dealing with 0.
the leapmotion aspect of this means you've reduced the potential audience, possibly to zero, so don't expect too much. the above is a guess that i have no way of verifying.
Thanks Koogs, I'll do my best to give it another go tonight and report back.
I'm happy to substitute just one finger for the mouse input, it doesn't need to be super accurate, I just want to experiment with the magic of hands free control. I was just getting errors when trying to launch the project (after combining it), so I must be breaking something when combining the projects (user error related sure!).
Hey Koogs, thanks to your encouragement, I've now got both parts of code working independently within the same sketch, the mouse position controls frequency, and the leap draws ellipses of a random colour at fingertip locations.
However, I've spent the past 3 hours trying to map one of the leap finger positions to frequency (instead of mouseX), but I've had zero luck - the syntax is clearly beyond my current ability. I would be eternally grateful if you, or anybody else for that matter is able to help by way of complete example to try? :)
Here is the code with both of them working independently within the same sketch:
If you can draw ellipses with your fingers on the screen, then I would expect changing line 46 to
would do the trick, wouldn't it?
Kf
Thanks for the advice kfrajer, very much appreciated. I gave it a try and it throws up the error:
The name "position" cannot be recognized
It also refuses to launch the sketch.
Any ideas? :-/
Right, my advice was not proper. Instead of replacing line 46, insert my previous two lines between lines 58 and 59 where position is available. Alternatively, I present next another version that is somehow more efficient. Replace all your draw() function with this code:
You should be able to use whatFinger to select what finger you want to use for your operation. To clarify, this is untested code as I do not have the hardware to run your sketch.
Here are two links that you could review. The first one tells you how to get a finger. The second one is a running example.
https://github.com/nok/leap-motion-processing#usage
https://forum.processing.org/two/discussion/4328/using-leap-motion-instead-of-mousex-and-mousey
Kf
kfrajer, first up, thank you for trying, you're an absolute legend for doing so, and I fully appreciate that it must be very difficult without having a leap to test things yourself - So once again, thank you! ;;)
Sadly, unless I'm doing something really stupid, which is entirely possible, neither method works. :(
The first method (example 1) that introduces the code between 58 & 59 results in: Type mismatch, "float" does not match with "int".
The second more efficient code you provided (example 2) results in: The variable "apos" does not exist
Neither launch.
I have provided both examples in full below, in case I am clearly doing something wrong elsewhere that is causing problems.
Example 1:
Example 2:
I will have to check the documentation more closely. For the time being, check the link I provided before (here is again https://forum.processing.org/two/discussion/4328/using-leap-motion-instead-of-mousex-and-mousey) as there is some code that shows exactly what you want to do:
Try this in a new sketch and see if it works. I will get back to you in a few hours about example 1 and example 2.
Kf
kfrajer! I think you might have nailed where the problem is! If you search 'leap' in the processing library, there are 2 alternative libraries that you can install:
Leap Motion for Processing | Library to use the [Leap Motion]
&
Leap Motion | Forwards Leap Motion controller events to a processing sketch
Only one of these can be installed at a time, and I was using the latter, I think that your code changes might be based on the former?
I tried the code above, and it didn't work (wrong library installed), this prompted me to try the other library, and it worked once I had. Once it was working I was able to adapt the far more easily understood code and sort of get the desired effect using this:
It's not quite as pretty before (I can't seem to get fixed random colours assigned to each finger - they flash like crazy if I try), BUT fp.x did allow me to control the oscillator with lateral hand movements!
Interestingly and potentially problematically, the sound was not constrained by the sketch window. For example, while the above code should limit the frequency to a range of 100Hz (far left of the sketch), and 1000Hz (far right), the leapmotion kept tracking the position of the fingers as they left the borders of the sketch window, and the frequency also continued far beyond these confines, e.g. sub 20Hz as you broke off to the left, and ear piercingly high as you ventured further right!
Use constrain() on the value before you use it in your sine object. Check the reference for detrails.
Kf
@Detta
Example 1 is already deploy as part of the examples provided by the library. Here is the link: https://github.com/heuermh/leap-motion-processing/blob/master/examples/leap_position_example/leap_position_example.pde
You can access the library source and documentation by clicking in the proper libraries in the next link: https://processing.org/reference/libraries/
Most of the time, exploring the examples provides enough information to start a new project. You can explore the source code for extra comments associated to functions provided. Don't forget there are also videos posted online explaining processing+LeapMotion from the beginning. For instance,
Kf
Thanks again Kf,
I've come on leaps and bounds thanks to your advice!
I now have the leap tracking fingers and palm position in a full HD window and, am able to map sound(s) to these values. One rather simple thing seems to elude me now, how on earth do I set the volume (amp) to 0 when the project starts?
Currently the tone plays when the project opens, then when I bring a hand into play, and then move either the X or Z value of said hand out of bounds, the volume ramps down to 0, and stays and 0 until I bring a hand back into sensor range, at which point the volume ramps back up to 1 (100%).
What I would like to do is have the amp set to 0 until a hand (hp) is detected.
My code is as follows (please forgive newbie formatting):
Any pointers?
how on earth do I set the volume (amp) to 0 when the project starts?
Add the following line between line 24 and 25:
sine.amp(amp);
However I am not sure if that will do the trick bc I can't run your code and bc it is very likely than when your device starts, even if your hands are not within the interactive space, the values that your device returns could be automatically initialize inside the interactive space. If this is so, then your draw() function will override my previous line and you will have a tone playing with non-zero volume.I will suggest to approach your problem in a different way. Try using mouse positions in a different sketch where you control the sound bound by different regions in your sketch. Then handle the situation when the mouse leaves your sketch area fast, so the mouse positions are still returning points inside your sketch despite your mouse being outside. An idea to handle this situation is that instead of reacting to absolute pointer position, you react to differential positions aka. "previous MINUS current" position. In the trivial case, if the hand is in the active area of drawing, your hand jitters and introduces differential values. When the differential values are zero, either the hand is very still or the hand has left the active region being monitored by leapMotion.
Kf