Processing, Minim Audio, Serial, and Linux
in
Integration and Hardware
•
1 year ago
I have a hardware project I've been working on for a while which is a modified LED fan for my computer. I replaced all of the LED's with RGB and connected it all to an ATMega168 chip, then built three of these. I created a multi-node bus using a modified serial line and a MAX232. Anyways, the idea is that all the LED's are controllable (256 level PWM on each LED channel) from the computer.
I wrote a Processing script that listens to the audio input using Minim, does an FFT, then uses the FFT values as PWM levels for the LED's. This has a cool effect of flashing the LED's to the music and works very well on my Windows PC's (tried on my laptop, Core 2 Duo T9300/2.5GHz/2GB RAM/nVidia 8600M/Win7 64 Pro and on my desktop, Core i7 930/4.0GHz/12GB RAM/Dual 5870's/Win7 64 Pro) and it works smoothly at 60FPS on both. The laptop I used a USB to serial adapter and tested one of the fan controllers on the bench, the desktop has three installed to a PCIe serial card and it runs smoothly as well. I tried both an old (1.1) version of Processing as well as the latest 2.X and both worked nicely.
So, I go to try it on my Linux boxes and it's a whole different world. I'm not sure why, either. The first is my old, slow Sempron 1.6GHz single-core, 2GB DDR RAM, Ubuntu 12.04 box. I managed to run the script using the latest Processing 2.X and eventually got maybe 10-15FPS out of it with the fan connected. I was attributing the slowness to the CPU being old, but another problem is that Minim only wanted to see the Mic In (in Windows I had used the loopback). I figured that sound card didn't have a loopback so whatever.
Now, my TV media PC is an Ubuntu 12.04 build with AMD A8-3870K Llano at 3.4GHz, 8GB DDR3 and it should be plenty capable of running 60FPS on this Processing script. I didn't have any fan device connected but it does have a serial port and the script only outputs, so I was able to run it and view the output on the screen (see the script below). It only got around 1 FPS or less with the serial enabled, then I commented out the serial and it still performed terribly. I know this computer has the loopback hardware (selecting one of the Mic inputs caused the Ubuntu sound menu level meter to match the playing music) but no matter what I set in the menu, Processing 2.X wanted to pull sound from my webcam's microphone. I tried disabling the webcam mic and it still used it. I have a feeling it's bypassing PulseAudio and using ALSA directly, then running into some sort of problem.
I have OpenJDK 6 on this machine which I use for Minecraft. I could try updating to 7 if that will help.
Has anyone else experienced extreme lag using either Minim or Serial on Linux? To rule out the machines being the problem, an old test I did on my old HP AthlonXP single-core showed that Windows had much better performance than Linux on the same box, though this was a while ago (2009 I think).
Here is my script, as long as you have a serial port you should be able to test, it doesn't check or hold for the port so it will just blindly send data.
I wrote a Processing script that listens to the audio input using Minim, does an FFT, then uses the FFT values as PWM levels for the LED's. This has a cool effect of flashing the LED's to the music and works very well on my Windows PC's (tried on my laptop, Core 2 Duo T9300/2.5GHz/2GB RAM/nVidia 8600M/Win7 64 Pro and on my desktop, Core i7 930/4.0GHz/12GB RAM/Dual 5870's/Win7 64 Pro) and it works smoothly at 60FPS on both. The laptop I used a USB to serial adapter and tested one of the fan controllers on the bench, the desktop has three installed to a PCIe serial card and it runs smoothly as well. I tried both an old (1.1) version of Processing as well as the latest 2.X and both worked nicely.
So, I go to try it on my Linux boxes and it's a whole different world. I'm not sure why, either. The first is my old, slow Sempron 1.6GHz single-core, 2GB DDR RAM, Ubuntu 12.04 box. I managed to run the script using the latest Processing 2.X and eventually got maybe 10-15FPS out of it with the fan connected. I was attributing the slowness to the CPU being old, but another problem is that Minim only wanted to see the Mic In (in Windows I had used the loopback). I figured that sound card didn't have a loopback so whatever.
Now, my TV media PC is an Ubuntu 12.04 build with AMD A8-3870K Llano at 3.4GHz, 8GB DDR3 and it should be plenty capable of running 60FPS on this Processing script. I didn't have any fan device connected but it does have a serial port and the script only outputs, so I was able to run it and view the output on the screen (see the script below). It only got around 1 FPS or less with the serial enabled, then I commented out the serial and it still performed terribly. I know this computer has the loopback hardware (selecting one of the Mic inputs caused the Ubuntu sound menu level meter to match the playing music) but no matter what I set in the menu, Processing 2.X wanted to pull sound from my webcam's microphone. I tried disabling the webcam mic and it still used it. I have a feeling it's bypassing PulseAudio and using ALSA directly, then running into some sort of problem.
I have OpenJDK 6 on this machine which I use for Minecraft. I could try updating to 7 if that will help.
Has anyone else experienced extreme lag using either Minim or Serial on Linux? To rule out the machines being the problem, an old test I did on my old HP AthlonXP single-core showed that Windows had much better performance than Linux on the same box, though this was a while ago (2009 I think).
Here is my script, as long as you have a serial port you should be able to test, it doesn't check or hold for the port so it will just blindly send data.
- import ddf.minim.analysis.*;
import ddf.minim.*;
import processing.serial.*;
Serial myPort;
Minim minim;
AudioInput input;
FFT fft;
String windowName;
char val2;
char val5;
char val8;
void setup()
{
frameRate(60);
size(512, 200, P3D);
textMode(SCREEN);
myPort = new Serial(this, Serial.list()[1], 38400);
minim = new Minim(this);
input = minim.getLineIn(minim.MONO, 2048);
// create an FFT object that has a time-domain buffer
// the same size as jingle's sample buffer
// note that this needs to be a power of two
// and that it means the size of the spectrum
// will be 512. see the online tutorial for more info.
fft = new FFT(input.bufferSize(), input.sampleRate());
textFont(createFont("Arial", 16));
windowName = "None";
}
void set_fan_led(int addr, int led, char val)
{
char serial_cmd[] = {0x00,(char)led,(char)addr,val,0xFF};
for(int i = 0; i < 5; i++)
{
myPort.write(serial_cmd[i]);
}
}
void fan_update_leds(int addr)
{
final char serial_cmd[] = {0x00,0x0C,(char)addr,0x01,0xFF};
for(int i = 0; i < 5; i++)
{
myPort.write(serial_cmd[i]);
}
}
void draw()
{
background(0);
stroke(255);
// perform a forward FFT on the samples in jingle's left buffer
// note that if jingle were a MONO file,
// this would be the same as using jingle.right or jingle.left
fft.forward(input.mix);
val2 = (char)(exp(fft.getBand(150))-1);
val5 = (char)(exp(fft.getBand(160))-1);
val8 = (char)(exp(fft.getBand(170))-1);
set_fan_led(0x10, 0x10, val2);
set_fan_led(0x10, 0x11, val5);
set_fan_led(0x10, 0x12, val8);
set_fan_led(0x10, 0x13, val2);
set_fan_led(0x10, 0x14, val5);
set_fan_led(0x10, 0x15, val8);
set_fan_led(0x10, 0x16, val2);
set_fan_led(0x10, 0x17, val5);
set_fan_led(0x10, 0x18, val8);
set_fan_led(0x10, 0x19, val2);
set_fan_led(0x10, 0x1A, val5);
set_fan_led(0x10, 0x1B, val8);
set_fan_led(0x11, 0x10, val2);
set_fan_led(0x11, 0x11, val5);
set_fan_led(0x11, 0x12, val8);
set_fan_led(0x11, 0x13, val2);
set_fan_led(0x11, 0x14, val5);
set_fan_led(0x11, 0x15, val8);
set_fan_led(0x11, 0x16, val2);
set_fan_led(0x11, 0x17, val5);
set_fan_led(0x11, 0x18, val8);
set_fan_led(0x11, 0x19, val2);
set_fan_led(0x11, 0x1A, val5);
set_fan_led(0x11, 0x1B, val8);
set_fan_led(0x12, 0x10, val2);
set_fan_led(0x12, 0x11, val5);
set_fan_led(0x12, 0x12, val8);
set_fan_led(0x12, 0x13, val2);
set_fan_led(0x12, 0x14, val5);
set_fan_led(0x12, 0x15, val8);
set_fan_led(0x12, 0x16, val2);
set_fan_led(0x12, 0x17, val5);
set_fan_led(0x12, 0x18, val8);
set_fan_led(0x12, 0x19, val2);
set_fan_led(0x12, 0x1A, val5);
set_fan_led(0x12, 0x1B, val8);
fan_update_leds(0x10);
fan_update_leds(0x11);
fan_update_leds(0x12);
for(int i = 0; i < fft.specSize(); i++)
{
//draw the line for frequency band i, scaling it by 4 so we can see it a bit better
line(i, height, i, height - fft.getBand(i)*4);
}
fill(255);
// keep us informed about the window being used
//text("The window being used is: " + windowName, 5, 20);
}
void keyReleased()
{
if ( key == 'w' )
{
// a Hamming window can be used to shape the sample buffer that is passed to the FFT
// this can reduce the amount of noise in the spectrum
fft.window(FFT.HAMMING);
windowName = "Hamming";
}
if ( key == 'e' )
{
fft.window(FFT.NONE);
windowName = "None";
}
}
void stop()
{
// always close Minim audio classes when you finish with them
input.close();
minim.stop();
super.stop();
}
1