How can I determine which spectrum is generated from which file on mouseclick?

Answers

• kfrajer

  June 2017

  If you plot both files, then you have access to the data for each source. When you have a mouseEvent, you compare the mouseX/mouseY points to your data points and you should determine which point is the closest to the mouse pointer. Then you need to figure out if that point belongs to the voice data set or from the second file.

  Kf
• mezbah

  June 2017

  Yes I have the data for each source..But how can I compare the data points and mouseX/mouseY...I mean where should I look into among the data.. @kfrajer
• kfrajer

  June 2017

  What structures are you using to store the data?

  Second question: How do you map your data to your window dimensions?

  Are you using translate/rotate?

  I would use the dist() function: https://processing.org/reference/dist_.html and find the point with the shortest distance to the mouse pointer's coordinates.

  Kf
• mezbah

  June 2017

  This is the structure I followed for 1 file..If you could Kindly look into it..And suggest which way I should go with this..

  import ddf.minim.analysis.*;
  import ddf.minim.*;
  
  Minim minim;
  AudioInput in;
  FFT fft;
  AudioPlayer ap;
   // how many individual peak bands we have (dep. binsperband)
  float gain = 0; // in dB
  float dB_scale = 2.0;  // pixels per dB
  int buffer_size = 1024;  // also sets FFT size (frequency resolution)
  float sample_rate = 44100;
  
  int spectrum_height = 200; // determines range of dB shown
  int legend_height = 20;
  int spectrum_width = 512; // determines how much of spectrum we see
  int legend_width = 40;
  int freqOffset = 0;
  
  void setup()
  {
    size(552, 220, P2D);
    textMode(SCREEN);
    textFont(createFont("Georgia", 12));
  
    minim = new Minim(this);
  
    //in = minim.getLineIn(Minim.MONO,buffer_size,sample_rate);
     ap = minim.loadFile("test.mp3");
    // create an FFT object that has a time-domain buffer 
    // the same size as line-in's sample buffer
    fft = new FFT(ap.bufferSize(), ap.sampleRate());
  
    // Tapered window important for log-domain display
    fft.window(FFT.HAMMING);
  
    //println("buffer:"+in.bufferSize());
    //println("sampleRate:"+in.sampleRate());
    //println("peaksize:"+peaksize);
  }
  
  
  void draw()
  {
    // clear window
    background(0);
  
    // perform a forward FFT on the samples in input buffer
    fft.forward(ap.mix);
    ap.play();
  
    // now draw current spectrum in brighter blue
    stroke(64,255,255);
    noFill();
    for(int i = 0; i < fft.specSize(); i++)  {
  
      // draw the line for frequency band i using dB scale
      float val = dB_scale*(20*((float)Math.log10(fft.getBand(i))) + gain);
  
      if (fft.getBand(i) == 0) {   val = -200;   }  // avoid log(0)
  
      int y = spectrum_height - Math.round(val);
  
      if (y > spectrum_height) { y = spectrum_height; }
  
      line(legend_width+i+freqOffset, spectrum_height, legend_width+i+freqOffset, y);
  
      // update the peak record
      // which peak bin are we in?
      //int peaksi = i/binsperband;
      //if (val > peaks[peaksi]) {
      //  peaks[peaksi] = val;
      //  // reset peak age counter
      //  peak_age[peaksi] = 0;
      //}
    }
    // add legend
    // frequency axis
    fill(255);
  
    stroke(255);
  
    int y = spectrum_height;
  
    line(legend_width,y,legend_width+spectrum_width,y); // horizontal line
  
    // x,y address of text is immediately to the left of the middle of the letters 
    textAlign(CENTER,TOP);
  
    int spFreq=0; //for spacing
  
    for (float freq = 0.0; freq < ap.sampleRate(); freq += 2000.0) {
      int x = legend_width+spFreq+freqOffset; // which bin holds this frequency
     //println(freq+"->"+fft.freqToIndex(freq));
      line(x,y,x,y+4); // tick mark
      text(Math.round(freq/1000) +"kHz", x, y+5); // add text label
      spFreq+=45;
    }
  
    // DBlevel axis
    int x = legend_width;
  
    line(x,0,x,spectrum_height); // vertictal line
  
    textAlign(RIGHT,CENTER);
  
    for (float level = -100.0; level < 100.0; level += 20.0) {
      y = spectrum_height - (int)(dB_scale * (level+gain));
  
      line(x,y,x-3,y);
  
      text((int)level+" dB",x-5,y);
    }
  
  }
  
  void keyReleased()
  {
    // +/- used to adjust gain on the fly
    if (key == '+' || key == '=') {
      gain = gain + 5.0;
    } else if (key == '-' || key == '_') {
      gain = gain - 5.0;
    }
    //(.)/(/) used to adjust frequency axis
    else if(key == '/')
    {
      freqOffset = freqOffset-4;
    }
  
    else if( key == '.')
    {
      freqOffset = freqOffset+4;
    }
  }
  
  void stop()
  {
    // always close Minim audio classes when you finish with them
    in.close();
    minim.stop();
  
    super.stop();
  }
  
• jeremydouglass

  June 2017 edited June 2017

  One easy solution:

  1. Create a PGraphics for spectrum1 and spectrum2
  2. Draw each spectrum to each PGraphics
  3. On the main canvas, display your multi-spectrum graph by drawing the axes and then pasting each spectrum layer with image().
  4. When the mouse is clicked, check the brightness of the pixel on spectrum1 for a hit, then check spectrum2. You have to decide what to do if both are a hit.

  Checking a pixel in a PGraphics with spectrum1.get() based on mouseX, mouseY is a simple array lookup -- you can also do this spectrum1.pixels[] -- and you were already keeping the spectrum data in the array for drawing.
• kfrajer

  June 2017

  On the global scope:

  final color colSelected=color(255,0,0);
  final color colPlain=color(255);
  final float distanceThreshold=20;
  

  Now replace line 66 with:

      float x1=legend_width+i+freqOffset;
      float y1=spectrum_height;
      float x2=x1;
      float y2=y;
  
      line(x1, y1, x2, y2);
  
      pushStyle();
      noStroke();
      if (dist(mouseX, mouseY, x2, y2)<distanceThreshold) 
        fill(colSelected);
      else
        fill(colPlain);
      ellipse(x2, y2, 3, 3);
      popStyle();
  

  As you see, the dist operation here determines if your mouse pointer is close to the data. The value of distanceThreshold can be adjusted to be more accurate in selecting the spectrum (or differentiating multiple spectra) but in your case, it is inconvenient because your spectrum changes really fast, in you might not have enough time to detect when the mouse pointer is on top of the graph.

  This code is valid if you don't use transformations. Notice you can adjust your offset and gain and the dist algorithm will still work. That is because and I am working using the data you are plotting and not on your raw data.

  Since one can change the color from red to white, referring to selected or non-selected resp., then it is in this part of your code where you can inquire if the data came from your first or second file.

  Kf