Approach to utilise variable data from Draw

matt82au

Hello! New to Processing and have been experimenting with the tutorials and videos a bit, and have a hit a hurdle I am sure someone will have some insight to.

Essentially, I am trying to take the colour tracking algorithm from Tutorial 11.5, and utilise the avgX variable to control the panning of a sine wave. My code is below...

// Daniel Shiffman
// http://codingtra.in
// http://patreon.com/codingtrain
// Code for: 

import processing.video.*;
import processing.sound.*;

SinOsc sine;
Capture video;

color trackColor; 
float threshold = 25;

void setup() {
  size(640, 360);
  String[] cameras = Capture.list();
  //printArray(cameras);
  video = new Capture(this, cameras[3]);
  video.start();
  trackColor = color(255, 0, 0);
}

void captureEvent(Capture video) {
  video.read();
}

void draw() {
  video.loadPixels();
  image(video, 0, 0);

  //threshold = map(mouseX, 0, width, 0, 100);
  threshold = 20;

  float avgX = 0;
  float avgY = 0;

  int count = 0;

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d < threshold*threshold) {
        stroke(255);
        strokeWeight(1);
        point(x, y);
        avgX += x;
        avgY += y;
        count++;
      }
    }
  }

  // We only consider the color found if its color distance is less than 10. 
  // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
  if (count > 0) { 
    avgX = avgX / count;
    avgY = avgY / count;
    // Draw a circle at the tracked pixel
    fill(255);
    strokeWeight(4.0);
    stroke(0);
    ellipse(avgX, avgY, 24, 24);


  }
}

float distSq(float x1, float y1, float z1, float x2, float y2, float z2) {
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1);
  return d;

}

void mousePressed() {
  // Save color where the mouse is clicked in trackColor variable
  int loc = mouseX + mouseY*video.width;
  trackColor = video.pixels[loc];
}
  {
      sine = new SinOsc(this);
      sine.pan(map(avgX, 0, width, -1.0, 1.0));
      sine.play();
}  

The problem is that the variable avgX cannot exist outside the draw. It works if I place the Oscillator in Draw, but that causes a crash in a hurry!

Any help greatly appreciated!

Best, Matt

Lord_of_the_Galaxy

The problem is that the variable avgX cannot exist outside the draw.

If that's all your problem is, then add this to the start of the program -

float avgX = 0;
float avgY = 0;

and replace line 35-36 with -

avgX = 0;
avgY = 0;

matt82au

thanks for your suggestion, but it doesn't solve my problem. I am guessing because the variable avgX is evolving constantly within the draw window. If I println(avgX) inside the draw window, I will receive constantly evolving responses based on the positional tracking of the colour I have selected. If I println(avgX) outside, it remains a static figure.

I need to get that avgX from inside draw to be feeling my pan value outside of draw.

Am I right in thinking I just need to create that formula outside of draw and then obtain those number inside of draw? i.e., am I just doing this in the wrong order?

Thanks again for your help, Matt

Lord_of_the_Galaxy

I'm afraid I do not understand what your require. Can you please elaborate more? Thanks!

kfrajer

I think reviewing the functionality of draw could help in this case: https://processing.org/reference/draw_.html

Kf

matt82au

of course, thank you!

a large part of this sketch is from a processing tutorial about colour tracking. Using this sketch, I am able to assign a target colour, and have Processing render a ellipse around it and track it across the screen. You can see Dans example of this at his youtube link.

My goal was to take that tracking point, use (or map) its x axis to a variable, than I can use to control the panning functions of a sine wave.

I can achieve this crudely by assigning the sine wave code INTO the drawing window

      sine = new SinOsc(this);
      sine.pan(map(avgX, 0, width, -1.0, 1.0));
      sine.play();

When it is in the drawing window, avgX is obtaining the value from the main detection algorithm (which i believe is computed at the last moment of this section).

void draw() {
  video.loadPixels();
  image(video, 0, 0);

  //threshold = map(mouseX, 0, width, 0, 100);
  threshold = 20;

  float avgX = 0;
  float avgY = 0;

  int count = 0;

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d < threshold*threshold) {
        stroke(255);
        strokeWeight(1);
        point(x, y);
        avgX += x;
        avgY += y;
        count++;
      }
    }
  }

  // We only consider the color found if its color distance is less than 10. 
  // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
  if (count > 0) { 
    avgX = avgX / count;
    avgY = avgY / count;
    // Draw a circle at the tracked pixel
    fill(255);
    strokeWeight(4.0);
    stroke(0);
    ellipse(avgX, avgY, 24, 24);


  }
}

float distSq(float x1, float y1, float z1, float x2, float y2, float z2) {
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1);
  return d;

}

The problem with running the code like this, is that processing does not like the sine wave command being in the draw window, and crashes the runtime in around 5 seconds or so (but works great until then!)

So, I need to get the x co-ordinate of the ellipse I am drawing, and map and assign that to a new variable to control the pan aspect. Though, I can't figure out how to get that computed number from the DRAW section, so I can use it with my sine wav.

Hope that helps,

Matt

matt82au

Thank you kfrajer. What I think I am taking, and where I am going wrong, is that any computation of co-ordinates inside of the draw function are basically limited to there.

Is that correct?

If so, should I be attempting to compute the co-ordinates outside of Draw, in its own path of code, and then send that to Draw AND to my sine wave?

Best, Matt

matt82au

Ultimately, if I can track the x coordinate of my Ellipse, I will be set. is that possible if it is being generated in the draw section?

kfrajer

sine = new SinOsc(this);  
sine.pan(map(avgX, 0, width, -1.0, 1.0));
sine.play();

Is the first line on draw? I would say it is better in setup.

avgX and avgy are values provided by the detection algrotithm. They are updated in every draw cycle. The question here is how fast can you play the sine wave?

I am not familiar with sine.pan tbh... what does it do?

So let's say the object is static. it will produce one sound. If the object moves, it suppose to produce a sound with different frequency, higher frequency toward the right side of the screen?

Kf

matt82au

Close! Pan Is simply moving the output of the sine wave between left and right speaker. Crudely, I want my tracked object to control the sound moving left and right.

The sineOsc is currently at the end of the sketch. I tried placing it in draw which crashes. As long as it is outside of draw, the variable avgX is not following what occurs, as draw is consigned with a local variable which does not exit its bounds.

Thanks, Matt

kfrajer

You can try outside of draw... but where? Inside draw, you can trigger than when your avg position changed by more than, say, 15 pixels, you can trigger the panning. What you need is a global variable. Something like this below (partial code):

int globalX;
boolean trigger=false;
void draw(){
  avgX=...;
  avgY=...;
  if(abs(gloablX-avgX)>15)
     trigger=true;  //Trigger panning update
  gloablX=avgX;   //Update globalX position... you can do the same for Y

  ....

  if(trigger==true){
     //update your sin pan property
     trigger=false;  // Don't forget to reset your trigger
  }

}

However, I still think everything is happening inside draw. You don't need to instantiate an object every frame. Instead, you update your created object with a new pan value.

Kf

matt82au

Thanks Kf, I will try that some of that code / approach and see how we go.

Cheers, Matt

Lord_of_the_Galaxy

Based on my limited knowledge of processing sound, you should follow @kfrajer's approach.

Initialise the sine variable in setup, and play it. Inside draw, PAN it if there is sufficient difference between current position and previous update position.

matt82au

Thanks LotG!

matt82au

FYI - that worked very nicely. Thank you both!

Lord_of_the_Galaxy

Good Job!