Help with FPS drops on my audio visualiser

CodexVII

Hi all,

This is a personal project I've been working on for the past few weeks and have just recently tried to tidy up everything last Sunday. The program itself works and will create an audio visualization in real time by listening to the lineIn.

My problem is that on execution frameRate tends to drop at certain times. Looking at the task manager usage also sits above 25% of the CPU consistently. Most of the time it sits at 60/59 fps but the drop is noticeable, especially if the sketch is just running on another monitor while I'm using something else on my primary. I do have to add that I'm using an Inspiron 17R with a mobile GPU from AMD.

Putting low end specs in the bench for now I think my code just isn't optimized and I may have unnecessary for loops present or I'm going about the wrong way for collecting current and previous FFT data to update the circles with.

Here are the code as well as the data folder in case anyone wants to try it out.

PS: Before running it, pick the resolution size from setup() from the four available. You must change the img being drawnin draw() to the corresponding img suitable for your resolution (img1, img2, im3, img3 - found just below where you set the size). Sorry I know it's clunky but it wasn't a high priority to do this automatically.

By default it runs at 1366x768, so if that fits in your screen then there's no need to change.

Cheers,

Ian

http://www.mediafire.com/download/i3apc17lw46q7ah/data.zip

//Bounce!
//The objective is to create an array of balls to react to music. Each ball has its own frequency band that it will react to.
//The higher the intensity of that frequency band the higher the ball will jump.
//
//Author: Ian Lodovica
//Date: 6th of February 2015

import ddf.minim.analysis.*;
import ddf.minim.*;

PImage img1, img2, img3, img4;
Minim minim;
AudioInput in;
FFT fft;

final int num_of_circles = 140;
FFTBall[] ball = new FFTBall[num_of_circles];

int gain = 220; //bigger the number the higher the balls jump 200~ 769p 400+ 1080p
int attack = 240; //time to reach goal in ms
int decay = 180;  //time to reach origin in ms

void setup()
{
  final float radius;
  final float origin;
  final float offset;

  ////////////////////////////////////////////////////////////////////////
  //Resolution of the App
  ////////////////////////////////////////////////////////////////////////
  frame.removeNotify();
  frame.setUndecorated(true);
  frame.addNotify(); 
  //size(1300, 600);
  //size(1920, 1080);
  size(1366, 768);
  //size(1600, 900);
  img1 = loadImage("BG.jpg");
  img2 = loadImage("BG 1300x600.jpg");
  img3 = loadImage("BG 1366x768.jpg");
  img4 = loadImage("BG 1600x900.jpg");

  ////////////////////////////////////////////////////////////////////////
  //Minim Stuff
  ////////////////////////////////////////////////////////////////////////
  minim = new Minim(this);  //initialise minim to use the class' methods
  in = minim.getLineIn(minim.STEREO, 2048);  //listen to the incoming audio
  fft = new FFT(in.bufferSize(), in.sampleRate());  //sets up the fft object
  fft.logAverages(120, 21); 
  fft.window(FFT.HAMMING);  //attempts to keep ends of window continuous

  ////////////////////////////////////////////////////////////////////////
  //Initialising Balls
  ////////////////////////////////////////////////////////////////////////
  ellipseMode(RADIUS);  
  noStroke();
  radius = (width/(float)num_of_circles)/2;
  origin = height + 2*radius;
  offset = 2*radius;

  for (int i = 0; i < num_of_circles; i++)  
  {
    ball[i] = new FFTBall(radius, radius*2*i + radius, offset, origin);
  }

  println(fft.avgSize());
}

void draw()
{
  image(img3, 0, 0, width, height);
  drawBalls();
}

//We want to check the avg value for this frame on each ball and see if it's < the previous frame
//If so then the ball should fall back down
//If not we set a new destination for it to rise to with the new avg calculated.  
void drawBalls()
{
  fft.forward(in.mix);

  //updates FFT data for all the balls at the frame the method is called
  for (int i = 0; i < map (fft.avgSize (), 0, fft.avgSize(), 0, num_of_circles); i++)
    ball[i].updateFFT(i);

  //this part draws the circles with the updated coordinates
  for (int i = 0; i < num_of_circles; i++)
  {
    ball[i].updateCoords();
    ball[i].display();
  }
}

class FFTBall
{
  //constant variables
  private final float radius;
  private final float offset;
  private final float origin;

  //used for tweening
  private float x;
  private float start_y;
  private float stop_y;
  private float current_y;
  private float pct;
  private float step;
  private float time;  //required time to hit peak, everything else is a fraction of it

  //color updates
  float sat;
  float brightness = 60;
  float hue;

  //used for calculating fft data
  float old_avg;
  float current_avg;

  FFTBall(float radius, float x, float offset, float origin)
  {
    this.radius = radius;
    this.x = x;
    this.offset = offset;
    this.origin = origin;
  }

  //keep fft data from one previous frame as well as the current frame
  void updateFFT(int index)
  {
    old_avg = current_avg;  //send data from last frame to old_avg
    current_avg = sqrt(fft.getAvg(index));
  }

  //sets the target coord for the circle to travel to using fft data collected
  void updateCoords()
  {
    time = attack;
    if (current_avg > old_avg &&  (height-radius) - current_y <= 0.5)
    {
      pct = 0;
      // start_y  = current_y ;  //set current y to start y
      start_y = height-radius;
      stop_y = height - radius - (current_avg *gain);  //set destination point
      time = attack;  //sets time to reach destination
    } 

    if (current_avg  < old_avg )  //start going down
    {
      pct  = 0;
      start_y  = current_y ;
      stop_y  = origin;  //origin
      time = decay;  //sets time to reach destination
    }

    step = 1.0/(60*((float)time/1000.0));  //time is capable of changing. see the two above if statements
    if (pct  < 1.0)
    {
      current_y = start_y  + ((stop_y - start_y)*pct );
      pct += step;
    }

    //    //allows intensity of sounds to reflect colors of the circles
    colorMode(HSB, 360, 99, 99);
    sat  = map(current_y, (.8)*height, height+radius, 20, 80);
    brightness  = map(current_y, height+radius, (.8)*height, 20, 99);
    hue  = map(current_y, height+radius, (.8)*height, 160, 220);
    fill(color(hue, sat, brightness ));
  }

  //should only be displayed when all previous updates are completed
  void display()
  {
    ellipse(x, current_y +offset, radius, radius);
  }
} 

GoToLoop

These are most obvious offenders for fix I've managed to spot in a quick review.
Dunno whether it's gonna make any diff. though: :-/

#043  colorMode(HSB, 360, 100, 100);

#072  background(img3);

#084  float avg = map(fft.avgSize(), 0, fft.avgSize(), 0, num_of_circles);
#085  for (int i = 0; i < avg; ball[i].updateFFT(i++));

#088  for (FFTBall b : ball) {
#089    b.updateCoords();
#090    b.display();
#091  }
#092

#108

#148  else if (current_avg < old_avg)  //start going down

#156  if (pct < 1.0) {
#157    current_y = (stop_y - start_y)*pct + start_y;
#158    pct += 1.0/(60.0/1e3*time);  //time is capable of changing.
#159  }
#160
#161

#164

CodexVII

Thanks for the reply! I've implemented your fixes but it does seem to still go slower at some points :(

I thought your for loop was interesting though, I didn't realize you can call the method as a parameter in the for loop.

GoToLoop

A pity it wasn't enough! I've tried. Perhaps you call FFT methods too many times? :|
Nevertheless, some for () loop explanations: :-B

• Regular loop is for ( ; ; ), while "enhanced" or "foreach" loop is for ( : ).
• The latter extracts 1 element each iteration and stores it in the declared iterator.
• So in (FFTBall b : ball), 1 element is read from ball and stored in b each time.
• That's why b can be used as a FFTBall reference @ b.display(); for example.

CodexVII

I'll definitely try to start using the enhanced for loop whenever I can!

I'm starting to think that it's just having fps drops for me because I'm running it on a laptop. I'd love to test it on a higher specced machine but at the same time I don't want my program to have a high spec requirement. Thanks for the lesson though!