This is great! Thanks for putting it together. I had found separate code for playing a midi file and interpreting all messages as a long array – but your solution combining them is really helpful. Main modification I made so far, was additional output readings of the noteName (A#) and the octave. My plan is to combine it with minim so an oscillator can play the midi notes.. will post again once done.

Hello everyone,

I have a spreadsheet with the first 1 million numbers in PI. The goal is to have different visuals displayed along with music tones in accordance with the numbers in the chart (0-9). The sound works, however I cannot get the program to show any visuals at the same time. Eg. I want the program to draw an ellipse whenever "0" is read from the .CSV file.

Here is my code:

Table data;
int[] row; 

import ddf.minim.*;

Minim minim;

AudioPlayer DO;
AudioPlayer RE;
AudioPlayer MI;
AudioPlayer FA;
AudioPlayer SOL;
AudioPlayer LA;
AudioPlayer TI;
AudioPlayer DOO;

void setup() {
  size(500, 500);
  background(50);
  minim = new Minim(this);
  data = loadTable("pi.csv", "header");

  DO = minim.loadFile("do.wav");
  RE = minim.loadFile("re.wav");
  MI = minim.loadFile("mi.wav");
  FA = minim.loadFile("fa.wav");
  SOL = minim.loadFile("sol.wav");
  LA = minim.loadFile("la.wav");
  TI = minim.loadFile("ti-WAV.wav");
  DOO = minim.loadFile("do-octave.wav");
}

void draw() {
  fill(250);
  stroke(250);


  row = new int[ data.getRowCount()]; //create an array for row


  for (TableRow row : data.rows()) {
    for (int i=0; i < 10; i++) { // for loop with an increasing "i" allows program go get info through the rows
      int A = row.getInt(i);  //intA is equal to the number in the row

      delay(200);
      ellipse(mouseX, mouseY, 100, 50);

      if (A==0) {
        delay(500);
        ellipse(100, 100, 100, 50);
      }

      if (A==1) {
        RE.rewind();
        RE.play();
        ellipse(200, 200, 50, 50);
      }

      if (A==2) {
        MI.rewind();
        MI.play();
      }

      if (A==3) {
        FA.rewind();
        FA.play();
      }

      if (A==4) {
        SOL.rewind();
        SOL.play();
      }

      if (A==5) {
        LA.rewind();
        LA.play();
      }
      if (A==6) {
        TI.rewind();
        TI.play();
      }
      if (A==7) {
        DOO.rewind();
        DOO.play();
      }
      if (A==8) {
        delay(300);
      }

      if (A==9) {
        DO.rewind();
        DO.play();
      }
    }
  }
}

What can I do? Thank you!!

Assuming you mean e.g. the notes played by the keys of a piano, does this help?

https://en.m.wikipedia.org/wiki/Piano_key_frequencies

The numbers may surprise you -- rather than being 100, 200, 300 they are irregular in regular ways. If you dig into musical theory a bit more -- e.g. octaves -- there is a lot to learn about how these intervals work.

https://en.m.wikipedia.org/wiki/Octave

But for now, you could just transcribe notes using a lookup table.

I can't view the tutorial site from my phone, but there is a Scales entry that might be useful:

http://explodingart.com/soundcipher/tutorials.html

As an exercise, you could check what FFT output gives you zero or infinity and then save that specific buffer into a csv for example. Even better, if you save the buffer and the FFT and then have a look at the data and make some sense of it. If you provide the files, I could do some R or octave magic.

Kf

Hello. I am a processing noob and wanted to try out the Serial to MIDI processing sketch to work with my Arduino Uno. On running the code below, a TRIPLE_DOT error is generated at this line :

void MIDI_TX(unsigned char MESSAGE, unsigned char PITCH, unsigned char VELOCITY)

I've looked up all similar TRIPLE_DOT errors online but cannot seem to figure out what exactly might be going wrong with the code. Any advice would be much appreciated!

// BY: MARK DEMERS | May 2009
// DESCRIPTION: Demo sketch to play notes from middle C in the 4th octave up to B in the 5th octave and then back down.
// HOOK-UP:
// 1. Plug USB cable from Arduino into your computer.
// USAGE:
// 1. Install and Set-up Serial MIDI Converter from SpikenzieLabs
// 2. Open, compile, and upload this sketch into your Arduino.
// 3. Run Serial MIDI Converter in the background.
// 4. Launch your music software such as Garage Band or Ableton Live, choose a software instrument and listen to the music.

int note = 0;     

void setup() 
{
  Serial.begin(57600);                                       // Default speed of the Serial to MIDI Converter serial port
}

void loop() 
{

  for(int note=60; note<=83; note++)                        // Going Up
  {
    MIDI_TX(144,note,127);                                  // NOTE ON
    delay(100);

    MIDI_TX(128,note,127);                                  // NOTE OFF
    delay(100);
  }

  for(int note=82; note>=61; note--)                       // Coming Down
  {
    MIDI_TX(144,note,127);                                  // NOTE ON
    delay(250);

    MIDI_TX(128,note,127);                                  // NOTE OFF
    delay(250);
  }

}


void MIDI_TX(unsigned char MESSAGE, unsigned char PITCH, unsigned char VELOCITY) 
{
  Serial.print(MESSAGE);
  Serial.print(PITCH);
  Serial.print(VELOCITY);
}

Aha! This post answered my question:

https://forum.processing.org/two/discussion/comment/17292/#Comment_17292

My new code is:

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;
AudioPlayer song;
FFT fft;

// the number of bands per octave
int bandsPerOctave = 4;

// the spacing between bars
int barSpacing = 3;

void setup()
{
  size(512, 200);

  minim = new Minim(this);

  song = minim.loadFile("test.mp3", 1024);
  song.loop();

  fft = new FFT(song.bufferSize(), song.sampleRate());

  // calculate averages based on a miminum octave width of 22 Hz
  // split each octave into a number of bands
  fft.logAverages(22, bandsPerOctave);

  rectMode(CORNERS);
}

void draw()
{
  background(0);
  stroke(255);

  // perform a forward FFT on the samples in song's mix buffer
  fft.forward(song.mix);

  // avgWidth() returns the number of frequency bands each average represents
  // we'll use it as the width of our rectangles
  int w = int(width/fft.avgSize());

  for(int i = 0; i < fft.avgSize(); i++)
  {
    // get the amplitude of the frequency band
    float amplitude = fft.getAvg(i);

    // convert the amplitude to a DB value. 
    // this means values will range roughly from 0 for the loudest
    // bands to some negative value.
    float bandDB = 20 * log(2 * amplitude / fft.timeSize());
    // so then we want to map our DB value to the height of the window
    // given some reasonable range
    float bandHeight = map(bandDB, 0, -150, 0, height);

    // draw a rectangle for the band
    rect(i*w, height, i*w + w - barSpacing, bandHeight);
  }
}

Relevant to your post:

https://forum.processing.org/two/search?Search=octave

Kf

I want to create audio visuals like the bars visual that is commonly used in music videos. Minim has an FFT that gets the amplitude of each frequency band, but each band is evenly spaced. So the result is that the bars are mostly showing only the high frequencies, and the mid and low frequencies are bunched up on the left. I tried this, but I'm not good with logarithms and have no clue what the correct math to calculate this should be, and it didn't really work:

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;
AudioPlayer song;
FFT fft;

// the number of lines/bands to draw
int bands = 512;

float a = (log(20000 - 20) / log(bands));

void setup()
{
  size(512, 200);

  minim = new Minim(this);

  song = minim.loadFile("test.mp3", 2048);
  song.loop();

  fft = new FFT(song.bufferSize(), song.sampleRate());
}

void draw()
{
  background(0);
  stroke(255);

  fft.forward(song.mix);

  for(int i = 0; i < bands; i++)
  {
    // calculate the frequency for the current band on a logamorithmic scale
    float freq = pow(i, a) + 20;

    // get the amplitude at that frequency
    float amplitude = fft.getFreq(freq);

    // convert the amplitude to a DB value. 
    // this means values will range roughly from 0 for the loudest
    // bands to some negative value.
    float bandDB = 20 * log(2 * amplitude / fft.timeSize());
    // so then we want to map our DB value to the height of the window
    // given some reasonable range
    float bandHeight = map( bandDB, 0, -150, 0, height );

    // draw the band as a line
    line(i, height, i, bandHeight);
  }
}

I don't even know if it can be done using the FFT in Minim because there might not be enough bands calculated for the lower frequencies. If not, what can I use to accomplish this?

Just to clarify, processing is good at displaying data (sort of). When it comes to calculations of this type, you will need to create your own algorithms (or search them on the web). Since you will be doing matrix operation or solving equations of some sort (my guess, I am not familiar with RK4 calculations), you might be better off working with octave (free) or matlab software.

Kf

Hi I'm trying to make a musical keyboard with p5.sound and I'm running into trouble. I have some divs that make up the keyboard inside of a larger div called "keyboard". Each key div has an id corresponding to its note.

In the javascript file I add functionality for clicking each div and also playing keys on the keyboard.

Initially I had one octave and that was working reasonably well. When I added more divs for the second octave I started having notes playing when I didn't want them to. As soon as I load up the page, I get unwanted notes playing.

In my setup() function, I iterate over every div in "keyboard" and create a p5.Env object, set the ADSR, set the range, create a p5.Oscillator object, then set the amp to the envelop I created. Then I start the wave. And I keep track of each env in a global variable that maps div id to the env itself. So I can trigger attack and trigger release of each envelope in event listener functions like onmousedown etc.

Some potential problems are I have a function called keyPressed that might be mirroring a p5.js function. Also I have all of my event listener functions defined inside of setup().

Any idea what the problem is? Is there a better way to set this up?

I really appreciate any advice I can get, Thanks

Hi guys

I'm using version 1 kinect to make a blob interact with sound-reactive raining text. I was wondering how to make the text disappear once the blob intersects with one of the words? Please help! Thank you!

//Make text individual words disappear (change position off page) if blob position is equal word position.

import org.openkinect.freenect.*;
import org.openkinect.processing.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;
import ddf.minim.signals.*;
import ddf.minim.spi.*;
import ddf.minim.ugens.*;

Minim minim;
AudioInput in;
FFT fft;
KinectTracker tracker;
Kinect kinect;

String [] data = {"327-64-4367", "448-01-4857","553-15-8880","016-10-8387","222-07-1302","574-68-5578","104-42-0570","237-40-7110","212-21-5143","559-30-1997", "574-76-8818", "007-64-7860", "145-42-9696", "574-24-6501", "477-49-7241", "517-62-2683", "315-52-7169", "750-12-8784", "678-20-3681","5895 13th Street, Webster, NY 14580","7776 Grove Street, Romulus, MI 48174","2294 Brook Lane, Paramus, NJ 07652","2424 Route 64, Maineville, OH 45039","9645 Valley View Drive, Tewksbury, MA 01876","153 Cross Street, The Villages, FL 32162","920 Route 30, Loxahatchee, FL 33470","238 Lincoln Street, Jamestown, NY 14701","729 Meadow Street, Dubuque, IA 52001","133 Franklin Court, South Portland, ME 04106","994 Elm Avenue, Sun Prairie, WI 53590","392 Woodland Drive, Liverpool, NY 13090","901 Willow Lane, Bridgeport, CT 06606","92236","07731","45420","10701","08094","07030","19047","41017","17543","80302","80021","60134","32082","(494) 133-5459","(249) 186-5356","(818) 402-5177","(424) 926-1653","(201) 672-5146","(245) 199-2255","(147) 520-8998","(349) 179-5381","(758) 621-4845","(988) 984-0554","(786) 693-1798","(291) 731-8164","(345) 209-6039","(537) 175-2683","(918) 990-4772","(820) 906-9228","(916) 565-9637","(633) 293-3501","(881) 991-5264","(193) 382-1818","(329) 603-4359","(160) 347-3446","(870) 293-5829","5165 1130 7400 9060","6011 5436 3995 4943","3455 419447 60971","3486 888573 38316","6011 5910 4628 7894","6011 7865 3449 2309","4929 2788 4706 8154","5427 8194 1616 5774","5446 9823 4141 6972","6011 2940 9543 3242","4532 8600 5212 9117","6011 9832 4310 8231","3784 431486 81659","5520 6129 5998 3075","4485 5114 2769 9519","4716 7962 0326 5780","3418 877718 33322","4539 1378 7971 4063","3730 394465 12151","4539 4187 1183 6953","5295 2968 4751 7338","3755 105752 68078","157.95.65.166","107.165.28.12","75.62.96.132","249.88.95.147","187.244.83.79","134.131.159.96","49.206.109.182","95.33.230.69","211.162.26.78","186.58.2.250","PcCX5mQhf","s58@ubFUM","n1IPf4zJo","ShSUN0pPc","d-2eug@Qj","zy*@fYOaG","npujcoBVE","Kenzie Flores","Shamar Fox","Nadia Hatfield","Brodie Hood","Nathanial James","Leanna Woodard","Wilson Murillo","Lillie Sullivan","Lillianna Mcintyre","Edward Chan","Jay Chambers","Shirley Vaughn","Jaylah Daniel","Eden Gordon","Joanna Freeman","Jazmyn Hamilton","Molly Costa","Julie Snow","Kole Wu","Raul Santos","Jadiel Mercado","Anna Silva","Mckinley Wyatt","Lance Odom","Nylah Whitney","Melanie Berger","Colin Strong","Irvin Kent","Donna Levine","Tess Wilkinson","Micheal Shields","Helen Rocha","Destinee Bowers","Keyon Graves","Bradley Knight","Kaitlynn Santos","Natalya Hernandez","Humberto Knapp","Camryn Farrell","Pablo Aguilar"};  //40

String note;
color c;
int n;
int noteNumber;
int sampleRate= 44100;

float [] max= new float [sampleRate/2];
float maximum;
float frequency;
float hertz;
float midi;
float deg;
float brightnessThresh;

boolean ir = false;
boolean colorDepth = false;
boolean mirror = false;

Drop[] drops = new Drop[150];

void setup() {
  size(1280, 520); //size of kinect screen
  textSize(8);
  kinect = new Kinect(this);
  tracker = new KinectTracker();
  kinect.initDepth();
  kinect.initVideo();
  //kinect.enableIR(ir);
  kinect.enableColorDepth(colorDepth);

  brightnessThresh = 0;

  //mirror Image
  mirror = !mirror;
  kinect.enableMirror(mirror);

  deg = kinect.getTilt();
  // kinect.tilt(deg);

  minim = new Minim(this);
  minim.debugOn();
  in = minim.getLineIn(Minim.MONO, 4096, sampleRate);
  fft = new FFT(in.left.size(), sampleRate);

  textAlign(CENTER);
  for (int i = 0; i < drops.length; i++) {
    drops[i] = new Drop();
  }
}

void draw() {
  background(0);
  //smooth();
  findNote();
  image(kinect.getVideoImage(), 0, 0);
  image(kinect.getDepthImage(), 640, 0); //

  // Run the tracking analysis
  tracker.track();
  // Show the image
  tracker.display();

  int t = tracker.getThreshold();

  for (int i = 0; i < drops.length; i++) {
    drops[i].fall();
    drops[i].show();
  }

  fill(255);
  /*text(
    "Press 'i' to enable/disable between video image and IR image,  " +
    "Press 'c' to enable/disable between color depth and gray scale depth,  " +
    "Press 'm' to enable/diable mirror mode, "+
    "UP and DOWN to tilt camera   " +
    "Framerate: " + int(frameRate), 10, 515);
  text("threshold: " + t + "    " +  "framerate: " + int(frameRate) + "    " + 
    "UP increase threshold, DOWN decrease threshold", 10, 500); */

  }

void keyPressed() {
  int t = tracker.getThreshold();
  if (key == 'i') {
    //ir = !ir;
    //kinect.enableIR(ir);
  } else if (key == 'c') {
    colorDepth = !colorDepth;
    kinect.enableColorDepth(colorDepth);
  } else if (key == CODED) {
    if (keyCode == UP) {
      deg++;
    } else if (keyCode == DOWN) {
      deg--;
    } else if (key == CODED) {
      if (keyCode == RIGHT) {
        t+=5;
      tracker.setThreshold(t);
      }
    } else if (key == CODED) {
      if (keyCode == LEFT) {
        t-=5;
      tracker.setThreshold(t);
      }
    }
    deg = constrain(deg, 0, 30);
    kinect.setTilt(deg);
  }
}

class KinectTracker {
  int threshold = 1000; //depth threshold
  int[] depth;
  PImage display;

  KinectTracker() {
    kinect.initDepth();
    kinect.enableMirror(true);
    // Make a blank image
    display = createImage(kinect.width, kinect.height, RGB); 

  }

  void track() {
    depth = kinect.getRawDepth(); //Get raw depth as array of integers

    if (depth == null) return;

    float sumX = 0;
    float sumY = 0;
    float count = 0;

    for (int x = 0; x < kinect.width; x++) {
      for (int y = 0; y < kinect.height; y++) {

        int offset =  x + y*kinect.width;
        // Grabbing the raw depth
        int rawDepth = depth[offset];

        // Testing against threshold
        if (rawDepth < threshold) {
          sumX += x;
          sumY += y;
          count++;
        }
      }
    }
  }

  void display() {
    PImage img = kinect.getDepthImage();

    if (depth == null || img == null) return;

    // Going to rewrite the depth image to show which pixels are in threshold
    display.loadPixels();
    for (int x = 0; x < kinect.width; x++) {
      for (int y = 0; y < kinect.height; y++) {

        int offset = x + y * kinect.width;
        // Raw depth
        int rawDepth = depth[offset];
        int pix = x + y * display.width;
        if (rawDepth < threshold) {
          display.pixels[pix] = color(c, 150); //cream color 252,251,227
        } else {
          display.pixels[pix] = color(0);
        }
      }
    }
    display.updatePixels();


    // Draw blob image
    image(display, 0, 0);


  }

  int getThreshold() {
    return threshold;
  }
  void setThreshold(int t) {
    threshold =  t;
  }
}


//NOTES With Sounds
void findNote() {
  fft.forward(in.left);
  for (int f=0;f<sampleRate/2;f++) { //analyses the amplitude of each frequency analysed, between 0 and 22050 hertz
    max[f]=fft.getFreq(float(f)); //each index is correspondent to a frequency and contains the amplitude value 
  }
  maximum=max(max);//get the maximum value of the max array in order to find the peak of volume

  for (int i=0; i<max.length; i++) {
    if (max[i] == maximum) {
      frequency= i;
    }
  }

  midi= 69+12*(log((frequency-6)/440));// formula that transform frequency to midi numbers
  n= int (midi);

//the octave has 12 tones and semitones. 
if (n%12==9)
  {
    note = ("a");
    c = color (255, 99, 0);
  }

  if (n%12==10)
  {
    note = ("a#");
    c = color (255, 236, 0);
  }

  if (n%12==11)
  {
    note = ("b");
    c = color (153, 255, 0);
  }

  if (n%12==0)
  {
    note = ("c");
    c = color (40, 255, 0);
  }

  if (n%12==1)
  {
    note = ("c#");
    c = color (0, 255, 232);
  }

  if (n%12==2)
  {
    note = ("d");
    c = color (0, 124, 255);
  }

  if (n%12==3)
  {
    note = ("d#");
    c = color (5, 0, 255);
  }

  if (n%12==4)
  {
    note = ("e");
    c = color (69, 0, 234);
  }

  if (n%12==5)
  {
    note = ("f");
    c = color (85, 0, 79);
  }

  if (n%12==6)
  {
    note = ("f#");
    c = color (116, 0, 0);
  }

  if (n%12==7)
  {
    note = ("g");
    c = color (179, 0, 0);
  }

  if (n%12==8)
  {
    note = ("g#");
    c = color (238, 0, 0);
  }
}

void stop()
{
  in.close();
  minim.stop();

  super.stop();
}

class Drop {
  float x;
  float y;
  float z;
  float len;
  float yspeed;
  String textHolder = "text";
  float word;

  Drop() {
    x  = random(width); //rain drops at random width on x-axis
    y  = random(0, 700); //sections of rain
    z  = random(0, 1); //general speed
    len = map(z, 0, 20, 10, 20);
    yspeed  = map(z, 0, 20, 5, 2); //Speeds of raindrops, 4 is variation

    textHolder = data[int(random(data.length))];

  }

  void fall() { //speed of rain
    y = y + yspeed;
    float grav = map(z, 0, 20, 0, 0.2); //(z, 0, 20, 0, 0.2)
    yspeed = yspeed - grav; //+ grav;


    if (y > height) {
      y = random(-500, 40);
      yspeed = map(z, 0, 20, 4, 20);  //(z, 0, 20, 4, 1000);
    }
  }

  void show() {
    float thick = map(z, 0, 20, 1, 3);

    fill(c);
    text(textHolder, x, y, y+len); 
  }
}

Hi This modified example works for me. You can find it in the available example listing under libraries>>Sound>>Soundfile >>sample

/*
This is a sound file player. 
*/


import processing.sound.*;

SoundFile soundfile;
LowPass lowPass;
float freq=500;

void setup() {
    size(640,360);
    background(255);

    //Load a soundfile
    soundfile = new SoundFile(this, "vibraphon.aiff");

    lowPass = new LowPass(this);

    // These methods return useful infos about the file
    println("SFSampleRate= " + soundfile.sampleRate() + " Hz");
    println("SFSamples= " + soundfile.frames() + " samples");
    println("SFDuration= " + soundfile.duration() + " seconds");

    // Play the file in a loop
    soundfile.loop();
    lowPass.process(soundfile);
    lowPass.freq(freq);
}      


void draw() {
  // Map mouseX from 0.25 to 4.0 for playback rate. 1 equals original playback 
  // speed 2 is an octave up 0.5 is an octave down.
  soundfile.rate(map(mouseX, 0, width, 0.25, 4.0)); 

  // Map mouseY from 0.2 to 1.0 for amplitude  
  soundfile.amp(map(mouseY, 0, width, 0.2, 1.0)); 

  // Map mouseY from -1.0 to 1.0 for left to right 
  soundfile.pan(map(mouseY, 0, width, -1.0, 1.0));  
}

Regarding your error you describe in the second post:

there are no more free IDs

I believe it is coming from ReACTIVision. Not much experience working with this module.

Kf

Hey

I have windows 8.1 64 bit, processing 3.1.1 64 bit and the following code runs fine from the sound library>>Soundfile>sample>>sample.pde, :

/*
This is a sound file player. 
*/


import processing.sound.*;

SoundFile soundfile;

void setup() {
    size(640,360);
    background(255);

    //Load a soundfile
    soundfile = new SoundFile(this, "vibraphon.aiff");

    // These methods return useful infos about the file
    println("SFSampleRate= " + soundfile.sampleRate() + " Hz");
    println("SFSamples= " + soundfile.frames() + " samples");
    println("SFDuration= " + soundfile.duration() + " seconds");

    // Play the file in a loop
    soundfile.loop();
}      


void draw() {
  // Map mouseX from 0.25 to 4.0 for playback rate. 1 equals original playback 
  // speed 2 is an octave up 0.5 is an octave down.
  soundfile.rate(map(mouseX, 0, width, 0.25, 4.0)); 

  // Map mouseY from 0.2 to 1.0 for amplitude  
  soundfile.amp(map(mouseY, 0, width, 0.2, 1.0)); 

  // Map mouseY from -1.0 to 1.0 for left to right 
  soundfile.pan(map(mouseY, 0, width, -1.0, 1.0));  
}

Kf

@Chrisir I didn't know that. Thanks!
@jules512 framerate of 1000?? How??
Anyways, the code(Note that it uses ArrayList ):

import processing.video.*;
import ddf.minim.*;
import ddf.minim.ugens.*;

static final int xoffset = 30;//adjust to suit you needs
static final int yoffset = 30;//adjust to suit you needs

Minim minim;
AudioOutput out;

Oscil      wave;

Capture cam;

ArrayList<Circle> cirs;//An ArrayList to keep track of circles
PGraphics pg;//to draw rectangles
int pointillize = 30;
int Lx[] = {0, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280, 300};
int Ly[] = {0, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220};


void setup() {
  size(720, 600);
  frameRate(30);//not exactly required, but could come of use

  //initialize
  pg = createGraphics(width, height);
  cirs = new ArrayList<Circle>();
  pg.beginDraw();
  pg.background(0);
  pg.rectMode(CENTER);
  pg.endDraw();

  cam = new Capture(this, 320, 240, 30);
  cam.start();
  background(0);
  smooth(4);
   // initialize the minim and out objects
  minim = new Minim(this);
  out   = minim.getLineOut();

  wave = new Oscil( 440, 0.6f, Waves.SINE );

  wave.patch( out );
}

void draw() {

  cam.read();

  int x = Lx[ (int) random(Lx.length) ];
  int y = Ly[ (int) random(Ly.length) ];
  int loc = x + y*cam.width;

  // Look up the RGB color in the source image
  cam.loadPixels();
  float r = red(cam.pixels[loc]);
  float g = green(cam.pixels[loc]);
  float b = blue(cam.pixels[loc]);
  noStroke();

  if (frameCount%24 == 0){

      cirs.add(new Circle(((x+xoffset)*2),((y+yoffset)*2),pointillize+20, color(r,g,b, 200)));//two choices here - either use the colour of the pixel, or use a single bright colour, but that is not my concern

      //getting the wavelengh = lwave
      color c = color(r,g,b);
      float lwave = ((hue(c))*400/360)+380;

      //getting the related frequency= swave
      //octave 350_718Hz
      float swave = 300000000/lwave*1000000000/1099511627776L;


      //play sound of it
      wave.setFrequency( swave );


  } else {
    pg.beginDraw();
    pg.fill(r,g,b);
    pg.rect(((x+xoffset)*2),((y+yoffset)*2),pointillize,pointillize);
    pg.endDraw();
  }
  image(pg, 0, 0);

  for(int i = cirs.size() - 1; i >= 0; i--){
    Circle cir = cirs.get(i);
    cir.draw();
    if(cir.isDead())cirs.remove(i);
  }
  // bpm
  delay(5);
} 

Code for class Circle:

class Circle{
  int framesLeft = 60;
  float posX, posY, size;
  int colour;

  Circle(float posX_, float posY_, float size_, int colour_){
    posX = posX_;
    posY = posY_;
    size = size_;
    colour = colour_;
  }

  void draw(){
    pushStyle();
    fill(colour);
    ellipse(posX, posY, size * norm(framesLeft, 0, 60), size * norm(framesLeft, 0, 60));
    framesLeft--;
    popStyle();
  }

  boolean isDead(){
    return (framesLeft <= 0);
  }
}

I admit it is probably a bit of an overkill, but tell me if that is what you wanted.

Hello processing community.

My name is jules, design student, and I'm working on a code to translate live video from the webcam into sounds.

I'm taking out one pixel every 24 pixels that is then displayed bigger (and round) and is picked to be translated into a sine wave. I want this one selected pixel to appear bigger(let's say twice the scale), and then decrease its size slowly until it gets the same size as all the other squares. I'm doing this in order to display clearly where the sound is coming from/ highlight this part of the image for the moment it's played. The problem I face here is that the animation of the change of scale would take place while the normal loop of the draw is still running. And I don't know how to deal with that.

Thanks for reading

import processing.video.*;
import ddf.minim.*;
import ddf.minim.ugens.*;

Minim minim;
AudioOutput out;

Oscil      wave;

Capture cam;
int pointillize = 30;
int Lx[] = {1, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, 240, 260, 280, 300};
int Ly[] = {1, 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220};
int i  = 0;


void setup() {
  size(1200, 1200);

  cam = new Capture(this, 320, 240, 30);
  cam.start();
  background(0);
  rectMode(CENTER);
  smooth();

   // initialize the minim and out objects
  minim = new Minim(this);
  out   = minim.getLineOut();

  wave = new Oscil( 440, 0.6f, Waves.SINE );

  wave.patch( out );
}

void draw() {

    cam.read();

   int x = Lx[ (int) random(Lx.length) ];
  int y = Ly[ (int) random(Ly.length) ];
  int loc = x + y*cam.width;

  // Look up the RGB color in the source image
  loadPixels();
  float r = red(cam.pixels[loc]);
  float g = green(cam.pixels[loc]);
  float b = blue(cam.pixels[loc]);
  noStroke();

  // Draw an ellipse at that location with that color
   i = i + 1;
   fill(r,g,b,255);

  if (i == 24){

        ellipse(((x+150)*2),((y+150)*2),pointillize+20,pointillize+20);
        i = 0;

      //getting the wavelengh = lwave
      color c = color(r,g,b);
      float lwave = ((hue(c))*400/360)+380;

      //getting the related frequency= swave
      //octave 350_718Hz
      float swave = 300000000/lwave*1000000000/1099511627776L;


      //play sound of it
      wave.setFrequency( swave );


  } else {

      rect(((x+150)*2),((y+150)*2),pointillize,pointillize);
  }


  // bpm
  delay(5);
} 

Also, another question : The code runs quite slow, I putted the delay at (1) but still, I would like to have it faster. Does anyone knows why ?

Hello, I draw Lines of notes from an XML file. But the lines are not correct!?

Why they draw up and down? They would be more represented in waveform?

You can see 2 images: The notes in orange and the reaction with Processing.

Here the code:

**The code: **

XML xml;
int stepWert = 10;
float summe;
String stepNote;
String octaveWert;
String durationWert;
String instrumentWert;
String c = "C";
String d = "D";
String e = "E";
String f = "F";
String g = "G";
String a = "A";
String h = "H";
String b = "B";
float x = 0;

import processing.pdf.*;

void setup() {
  beginRecord (PDF, "vektoren.pdf");
  size (80500, 700);
  background (0);
  noFill();
  stroke(250, 187, 0);
  strokeWeight(1);

  //Laden des XML Dokuments
  xml = loadXML("score-partwise.xml");

  //Filtern der für den Sketch wichtigen Daten
  // aus dem geladenen XML Dokument
  XML[] instrument = xml.getChildren("part");
  XML[] step = xml.getChildren("part/measure/note/pitch/step");
  XML[] octave = xml.getChildren("part/measure/note/pitch/octave");
  XML[] duration = xml.getChildren("part/measure/note/duration");

  for (int n = 0; n < 1; n++) {
    instrumentWert = instrument[n].getString("id");
  }

  //für jeden Datensatz im XML Dokument
beginShape();
for (int n = 0; n < step.length; n++) {
  stepNote = step[n].getContent();
    curveVertex (x, height-summe);
    stepNote = step[n].getContent ();
    octaveWert = octave[n].getContent ();
    durationWert = duration[n].getContent ();
    println ("step" + stepNote + "octave" + octaveWert + "duration" + durationWert 
    + "instrument" + instrumentWert);

    if (stepNote.equals(c)) stepWert = 1;
    if (stepNote.equals(d)) stepWert = 2;
    if (stepNote.equals(e)) stepWert = 3;
    if (stepNote.equals(f)) stepWert = 4;
    if (stepNote.equals(g)) stepWert = 5;
    if (stepNote.equals(a)) stepWert = 6;
    if (stepNote.equals(h)) stepWert = 7;
    if (stepNote.equals(b)) stepWert = 8;

    println ("stepWert" + stepWert);

    summe = stepWert*40 + int(octaveWert)*40;
    println ("Summe" + summe);

    x+=7;
}
endShape();
endRecord();   
}

Here's more code:

import ddf.minim.analysis.*;
import ddf.minim.*;
import controlP5.*;
import toxi.geom.*;

Minim minim;
AudioInput in;
FFT fft;
BeatDetect beat;
Filter filter;
FreqAndNote freqAndNote;
FreqAndColour freqAndColour;

ControlP5 cp5;
DropdownList dropDownList;

ArrayList noteCollection = new ArrayList();

int runMode = 0;
int menu = 0;

int bands = 20;
int arrayRows = 2;

int flagNoteFrequency = 0;
int flagNoteCollection = 0;

int bass = 300;
int mids = 1000;
int treble = 4000;

float [][] bassArray;
float [][] bassDominantFreq;
float [][] midsArray;
float [][] midsDominantFreq;
float [][] trebleArray;
float [][] trebleDominantFreq;

float [][] dominantFreq = new float [arrayRows][bands];

int limitFreq = 15000;  

int backgroundMode = 0;
String [] arrayNotes = { "Do C", "Do# Reb C# Bb", "Re D", "Re# Mib D# Eb", "Mi E", "Fa F", "Fa# Solb F# Gb", "Sol G", "Sol# Lab G# Ab", "La A", "La# Sib A# Bb", "Si B" };;

int numColors = 12;
int wheelDiameter = 600;
float colorSpacing = 1;
int wheelThickness = 120;


void setup() {

  size(1920, 1080, P3D);
  //size(1280, 720, P3D);
  smooth();  
  background(0);

  minim = new Minim(this);
  in = minim.getLineIn(Minim.STEREO, 512);
  fft = new FFT(in.bufferSize(), in.sampleRate());
  //fft.logAverages(60, 7);
  fft.linAverages(80);

  filter = new Filter();
  freqAndNote = new FreqAndNote();
  freqAndColour = new FreqAndColour();


  cp5 = new ControlP5(this);
  // create a DropdownList
  dropDownList = cp5.addDropdownList("myList").setPosition(20, 40);

  customize(dropDownList); // customize the list

}

void draw() {  

  colorMode(RGB, 255);


  fft.forward(in.mix);

  filter.run(fft, runMode, bands);


  if ((runMode == 5)) {

    create(bands); 

    pushMatrix();
    translate(width/2, height/2);
    background(0); 

    //for (int i = 0; i < noteCollection.size (); i++) {
    //Had to change for loop because we can't use remove() for a List unless it's backwards  
    for (int i = noteCollection.size (); i-- !=0; ) {

      Note note = (Note) noteCollection.get(i);
      note.run();

      if (note.timer == 0) {
        //println("Note removed");
        noteCollection.remove(note);
      }
    }
    popMatrix();
  }

  if ((runMode == 7)) {

    colorMode(HSB, 360, 100, 100);
    background(0);

    strokeWeight(wheelThickness);
    noFill();
    strokeCap(SQUARE);

    int colorAngle = 360/numColors;

    pushMatrix();
    translate(width/2, height/2);

    //arrayNotes = { "Do C", "Do# Reb C# Bb", "Re D", "Re# Mib D# Eb", "Mi E", "Fa F", "Fa# Solb F# Gb", 
                   //"Sol G", "Sol# Lab G# Ab", "La A", "La# Sib A# Bb", "Si B" };

    for (int i = 0; i < numColors; i++) {

      float startAngle = i * colorAngle + colorSpacing;
      float stopAngle = (i + 1) * colorAngle - colorSpacing;

      //Colour
      stroke(startAngle, 80, 90);
      arc(0, 0, wheelDiameter, wheelDiameter, radians(startAngle), radians(stopAngle));

    }
    for (int i = 0; i < numColors; i++) {
      textSize(28);

      text(arrayNotes[i], (wheelDiameter/2 + wheelThickness/2) * cos(i*PI/6 + PI/12), (wheelDiameter/2 + wheelThickness/2)* sin(i*PI/6 + PI/12));
    }  

    popMatrix();

  }  

}

void create(int totalBands) {
  pushMatrix();
  translate(width/2, height/2);

  for (int i = 0; i < totalBands; i++) {
    float[][] dominantFreq = filter.dominantFreq(filter.filter((limitFreq/totalBands) * (i), (limitFreq/totalBands) * (i+1), filter.fftArray));
    //println("Int i = " + i);
    //If amplitude is big enough:
    if (dominantFreq[1][0] > 0.5  ) {

      //Create a new  note
      int ellipseXMin = -width/2 + i * (width/totalBands);
      int ellipseXMax = -width/2 + (i+1) * (width/totalBands);
      int ellipseX = int(random(ellipseXMin, ellipseXMax));
      Vec3D origin = new Vec3D(ellipseX, 0, 0);

      //println("Note frequency " + dominantFreq[0][i] + " and note amplitude " +  dominantFreq[1][i]);
      //println("Note origin " + origin);

      Note note = new Note(origin, dominantFreq[0][0], dominantFreq[1][0], 10000);
      noteCollection.add(note);
    }
  }  

  popMatrix();
}

Some auxiliary classes:

import java.util.regex.Matcher;
import java.util.regex.Pattern;

class FreqAndColour {

  //Parameters
  String NOTE_SYMBOL[] = {
    "C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", 
    "A#", "B"
  };

  //Constructor
  FreqAndColour() {
  }

  void noteToColour (String string) {

    Pattern p = Pattern.compile("([a-zA-Z]{1}[#]?)([\\d]?)");
    Matcher m = p.matcher(string);
    if (m.find()) {
      //Note
      String note = m.group(1);
      //println(note);
      //Octave
      int octave = int(m.group(2));
      //println(octave);

      int noteIndex = 0;

      for (int i = 0; i < NOTE_SYMBOL.length; i++) {
        if ( note.equals(NOTE_SYMBOL[i]) ) {
          noteIndex = i;
        }
      }

      colorMode(HSB, 100); 
      fill((noteIndex * (100/12)), 50, octave * (100/6));
    }
  }
}

class FreqAndNote {

  double PITCH_OF_A4 = 57D;
  double FACTOR = 12D / Math.log(2D);
  String NOTE_SYMBOL[] = {
    "C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", 
    "A#", "B"
  };

  float frequencyOfA4 = 440F;
  float frequency;

  //Constructor:
  FreqAndNote() {
  }


  double getPitch (double d) {
    return 57D + FACTOR * Math.log(d / (double)frequencyOfA4);
  }

  float getFrequency(double d) {
    return (float)(Math.exp((d - 57D) / FACTOR) * (double)frequencyOfA4);
  }

  String makeNoteSymbol(double d) {
    int i = (int)(d + 120.5D);
    StringBuffer stringbuffer = new StringBuffer(NOTE_SYMBOL[i % 12]);
    stringbuffer.append(Integer.toString(i / 12 - 10));
    return new String(stringbuffer);
  }

  float valueOf(String s) {
    return (new Float(s)).floatValue();
  }

  int parseNoteSymbol(String s) {
    s = s.trim().toUpperCase();
    for (int i = NOTE_SYMBOL.length - 1; i >= 0; i--)
    {
      if (!s.startsWith(NOTE_SYMBOL[i])) {
        continue;
      }
      return i + 12 * Integer.parseInt(s.substring(NOTE_SYMBOL[i].length()).trim());
    }
    return 0;
  }

  String fromFreqToNote(float frequency) {
    return makeNoteSymbol(getPitch(frequency));
  }

  float fromNoteToFreq (String string) {
    //return getFrequency(parseNoteSymbol(string));
    float f = parseNoteSymbol(string);

    println(f);
    return getFrequency(f);
  }
}

I would really appreciated if someone would answer my question: if the width and height are specified in pixels and there is a noStroke() before drawing, the size should be consistent. Right?

Hi there, I have some strange behavior in my code.

  void display() { 
    noStroke();
    String note = freqAndNote.fromFreqToNote(frequency);
    freqAndColour.noteToColour(note);
    ellipse(loc.x, loc.y, 20, 20);
  }

freqAndColour.noteToColour(note) returns the colour of the ellipse: fill((noteIndex * (100/12)), 50, octave * (100/6));

However, the size of the ellipses change when I cycle through different types of visualizations in my sketch. If the width and height are specified in pixels and there is a noStroke() before drawing, the size should be consistent. Right?

Any ideas?

Hi,

I'm working on a music education sketch and I can't seem to get this part to work. I need it to display one of these twelve words depending on the random output, but I'm not getting anything. What am I doing wrong?

float interval = 60+random(12);

void setup()
{
  size(400,400);
}

void draw()
{
  background(255);
  if (interval == 60)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Unison", 10, 30); 
  }
  if (interval == 61)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Minor 2nd", 10, 30); 
  }
  if (interval == 62)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Major 2nd", 10, 30); 
  }
  if (interval == 63)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Minor 3rd", 10, 30); 
  }
  if (interval == 64)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Major 3rd", 10, 30); 
  }
  if (interval == 65)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Perfect 4th", 10, 30); 
  }
  if (interval == 66)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Tritone", 10, 30); 
  }
  if (interval == 67)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Perfect 5th", 10, 30); 
  }
  if (interval == 68)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Minor 6th", 10, 30); 
  }
  if (interval == 69)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Major 6th", 10, 30); 
  }
  if (interval == 70)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Minor 7th", 10, 30); 
  }
  if (interval == 71)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Major 7th", 10, 30); 
  }
  if (interval == 72)
  {
    textSize(32);
    fill(0, 102, 153);
    text("Octave", 10, 30); 
  }
}

Hi GoToLoop, From your description of the issues I'm pretty sure we've covered everything doable on the Processing coding side of the problem. I can't help wondering if there might be something closer to the hardware that would answer the need. Just so you can see where all this was heading here's the sketch I wanted to improve with the question. I wanted to improve the resolution of the lower pitched sounds without slowing the response to the higher pitched sounds. It works best for visualizing simple music, but can also show frequency components of voice. Thanks Again!!!

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim       minim;
FFT         fft2;
AudioInput in1;
int bufMult = 2;

float points[] = new float[1024*bufMult];

int[][] spect;
int divs = 8;
int[] vColor;
float[] sigAmp;
int[] vxColor;

PImage img1;
PImage img2;

SampleCollector waveform;
boolean          listening;

class SampleCollector implements AudioListener
{
  private float[] left;
  private float[] right;
  private float[] half;
  private float[] bHalf;

  SampleCollector()
  {
    left = null; 
    right = null;
  }

  synchronized void samples(float[] samp)
  {
    left = samp;
    half = samp;
    bHalf = concat(samp, samp); //to give double length
  }

  synchronized void samples(float[] sampL, float[] sampR)
  { 
   // if (left!=null && half !=null) 
 //   {bHalf = concat(sampL, sampR);}
    left = sampL;
    right = sampR;
    half = left;
  }

  synchronized float[] aaa()
  {  
   if (left != null)
   {
    bHalf = concat(left, half); 
    for (int i=0; i<half.length; i++)
    {
     // half[i] = 0.09;//(bHalf[2*i] + bHalf[2*i+1])/2; //to give same array length over double the time
    }
   } 
    return left;
  }  
}

void setup()
{
  waveform = new SampleCollector();
  size(1000, 768);

  spect = new int[768][3];
  background(0);
  for (int i=0; i<256; i++)
  {  
     int j=255-i;
     stroke(j, i, 0);
     line(width/2, i, width, i);
     spect[i][0] = j;
     spect[i][1] = i;
     spect[i][2] = 0;

     stroke(0, j, i);
     line(width/2, i+256, width, i+256);
     spect[i+256][0] = 0;
     spect[i+256][1] = j;
     spect[i+256][2] = i;

     stroke(i, 0, j);
     line(width/2, i+512, width, i+512);     
     spect[i+512][0] = i;
     spect[i+512][1] = 0;
     spect[i+512][2] = j;     
     int divs=8;
  }
  divs = 12;
  vxColor = new int[height];
  vColor = new int[height];
  for (int k=0; k<divs; k++)
  for (int j=0; j<height; j+=divs)
  { 
    int col = j*768/height;
    if (col>767) {col=767;} //spect undefined past 767
    stroke(spect[col][0], spect[col][1], spect[col][2], 255);
    int vert = j/divs+k*height/divs;
    line(50, vert, 70, vert); 
    vColor[vert] = color(spect[col][0], spect[col][1], spect[col][2], 255);

  }

  minim = new Minim(this);

  in1 = minim.getLineIn(Minim.STEREO, 1024*bufMult); 
  fft2 = new FFT( 1024*bufMult, 44100 );
  fft2.logAverages(16,64);//(minbandwidth Hz for an octave, bands per octave)

  sigAmp = new float[4000];

  img1 = createImage(width, height, RGB);
  img2 = createImage(5, height, RGB);

//  frameRate(30);
  print("buffer size: ", in1.bufferSize(), "sample rate: ", in1.sampleRate(), "fft2.avgSize: ", fft2.avgSize() );
  print("  ",img2.pixels.length);
  in1.addListener( waveform );
}

void draw()
{
  background(0);
  stroke(255);

  int startTrace = 0;

   while ((points[startTrace]>0 || points[startTrace+1]<0) && startTrace<200 )
    {
      startTrace++;
    }

  if (startTrace>200) {startTrace=200;}
  if ( waveform.aaa() != null) 
  for(int i = 0; i<800; i++) //i<waveform.aaa().length-startTrace-1; i++) 
  {
    line( i+100, height*5/6-waveform.aaa()[i+startTrace]*1000, i+101, height*5/6 - waveform.aaa()[i+1+startTrace]*1000 );
  } 
  if (waveform.aaa() != null)
  fft2.forward( waveform.aaa() ); //points captured from in1 below
  for(int i = 0; i < fft2.avgSize(); i++)
  {
    stroke(vColor[i]);
    sigAmp[i] = fft2.getAvg(i)*i*i/10000;  
  }

  smooth();//without this inclusion of the alpha value = 4th term in stroke causes simple black
  for (int i=0; i<height; i++)
  {     
    stroke(vColor[i], sigAmp[i]);
    line(width-5, height-i, width, height-i); 
  }

  for(int i = 0; i<in1.bufferSize()-1; i++)
  {
    points[i]=in1.mix.get(i);
  }


  img2 = get(width-2, 0, 2, height);
  img1.copy(img1, 10, 0, width-2, height, 0, 0, width-2, height-1);
  img1.copy(img2, 0, 0, 2, height, width-10, 0, width, height);

  image(img1, 0, 0);

}