Loading...
Logo
Processing Forum
OK, I'm in need of a coding friend, I am very much a newbie to this deal BUT IT LOOKS AWESOME!
Is anyone able to help me out with my uni project - I essentially want to be able to visualize speech, but I need code that can do the translation from microphone to visuals on screen. I will design the visuals which will be applied to the code. SO letters or phonemes (the sounds) will translate to a pattern/image (that layers on top), and if its possible pitch will depict the colour of the patterns?
Is anyone able to do this, help me, or point me in a good direction?

Thank you so much!!

Replies(6)

I'd start by getting the audio input working first.
Is the speech a pre-recorded thing, or something you'll be getting live from, say, a microphone?
Hey,
Thanks, I will look at audio input.
I will be getting it from a live sound feed from a microphone, so it will be interactive.

You made a third thread on the same topic. Why multiply them instead of updating this thread? I deleted the third one...
Sorry! I am new to this! I will keep it to this thread from now on :)
Is anyone able to help me with an outcome like this?

http://www.blakecarrington.com/014.php
Ohk, so I have a piece of code that generates a spectrogram, is there a way to warp the image, to get a more abstract visual? And even incorporate colour?


/**
* LiveSpectrogram
* Takes successive FFTs and renders them onto the screen as grayscale, scrolling left.
*
*/

import ddf.minim.analysis.*;
import ddf.minim.*;

Minim minim;
AudioInput in;
FFT fft;
int colmax = 500;
int rowmax = 256;
int[][] sgram = new int[rowmax][colmax];
int col;
int leftedge;


void setup()
{
 size(512, 256, P3D);
 textMode(SCREEN);

 minim = new Minim(this);
 
 in = minim.getLineIn(Minim.STEREO,2048);

 fft = new FFT(in.bufferSize(), in.sampleRate());
 fft.window(FFT.HAMMING);
}

void draw()
{
 background(0);
 stroke(255);
 // perform a forward FFT on the samples in the input buffer
 fft.forward(in.mix);

 for(int i = 0; i < rowmax /* fft.specSize() */; i++)
 {
   // fill in the new column of spectral values
   sgram[i][col] = (int)Math.round(Math.max(0,2*20*Math.log10(1000*fft.getBand(i))));
 }
 // next time will be the next column
 col = col + 1;
 // wrap back to the first column when we get to the end
 if (col == colmax) { col = 0; }
 
 // Draw points.  
 // leftedge is the column in the ring-filled array that is drawn at the extreme left
 // start from there, and draw to the end of the array
 for (int i = 0; i < colmax-leftedge; i++) {
   for (int j = 0; j < rowmax; j++) {
     stroke(sgram[j][i+leftedge]);
     point(i,height-j);
   }
 }
 // Draw the rest of the image as the beginning of the array (up to leftedge)
 for (int i = 0; i < leftedge; i++) {
   for (int j = 0; j < rowmax; j++) {
     stroke(sgram[j][i]);
     point(i+colmax-leftedge,height-j);
   }
 }
 // Next time around, we move the left edge over by one, to have the whole thing
 // scroll left
 leftedge = leftedge + 1;
 // Make sure it wraps around
 if (leftedge == colmax) { leftedge = 0; }
}


void stop()
{
 // always close Minim audio classes when you finish with them
 in.close();
 minim.stop();

 super.stop();
}

Incorporating colour is simple: you have drawing instructions (stroke(), point()). Just change the value in stroke() call (currently just a grey level) to something more colorful (eg. using the color() function), and you are done.

Warping the image is a bit harder, given the drawing method being used.