hi all

i want to do some simple lip syncing animation with processing, just volumes of samples in realtime controlling the size of a shape(mouth). I have the code below which analyses the level of one frequency band and varies the height of a rect.

the problem is that it updates the value too quickly and I don't want to slow down the framerate of the sketch. short of monitoring the system clock to decide when to redraw the shape is there perhaps a convenient way to set the resolution of this info in ess? i don't want to change the sample rate as i want the sound to remain intact

any advice greatly appreciated.

////////////////////////////////////////////

import krister.Ess.*;
Channel voice1;

void setup() {
 // start up Ess:
 Ess.start(this);
 
 // set up sound:
 voice1=new Channel("sample.wav");
 voice1.normalize();
 voice1.setupFFT(2);//1 frequency band is average amplitude

}

void draw() {
 background(255);
 drawMouth();
}

void drawMouth() {
 // get amplitude:
 voice1.loadSpectrum();
 float temp=voice1.spectrum[0]*10;
 // draw amplitude:
 noStroke();
 fill(0);
 rect(25,50,50,temp+.5);
 
}

// click plays once:
void mousePressed() {
 if (voice1.state!=Ess.PLAYING) {
   voice1.play(1);
 }
}

// clean up Ess before exiting:
public void stop() {
 Ess.stop();
 super.stop();
}

thanks watz, works perfectly except that i can't get it to work with just one frequency band, which is all i need

if i initialise both ess (or sonia) and soniahelper with one freq band i still get an array out of bounds exception at
ffthelper.update(myChannel.spectrum);

is there a way around this apart from something silly like:
float[] array = new float[256];
 array[0] = myChannel.spectrum[0];
 ffthelper.update(array);
//then just use ffthelper.spectrum[0]

Few comments and questions about the single band issue...

What's the reason for you using only a single frequency band Saying a "single" band is pretty meaningless without telling us any desired width of this band Even if you're thinking of analyzing only the range of formants of human voices you can work with ONE frequency band, but please don't mix this up with the spectrum bins returned by the FFT... In most cases speaking about a band of frequencies will refer to a whole collection of bins. Escpecially if you want to target/analyze the behaviour of a specific frequency bin, you'll need as many bins as possible to start with. In theory a spectrum of only one bin would cover the whole frequency range 0 Hz < f < 1/2 the sampling freq...

Since the FFT is attempting to break down the signal into a series of sinoids, at least 2 samples are required in order to work out the contributing frequencies of this signal (It's mathematically impossible to do it with one only).

Running danger of becoming too mathematical, the number of frequency bins required directly relates to the number of samples analyzed. In other words, the fewer bins you want in your spectrum the shorter (in time) the chunk of signal analyzed and so the poorer the quality/precision of the computed spectrum as well. For most applications a spectrum of anything under 128 bins will be pretty unusable since the related time window is too short (<3ms) and you might be better off just working with the average volume of the samples by using Ess's equivalent of Sonia's LiveInput.getLevel() method.

Thank you for the clarification, Toxi, I have to admit I am a bit hazy on the science myself. I took the liberty of blogging your post over on code & form, hope you don't mind.

Krushxy, not sure what you are trying to do, posting code might help. Toxi's FFT code does work if you pass it some appropriate data.

Well, the person who made that video was me, and the code I used was the one from Toxi: http://www.toxi.co.uk/p5/fftDebug/. I used the javax.sound.sampled.* API to get sound samples from an AIFF sound file and feed it through the FFT code.

It's not too hard if you know what you're doing, but I think it would be easier to use Ess and load MP3s for analysis.