For a test, I rigged up some RGB LED's on a breadboard powered by an ATTiny2313 microcontroller.  Using the Minim library, I compute an FFT on the audio input and use that to flash the LED's to music.  I made this in Windows and it works quite well, getting usually around 30+ FPS on the display and LED's (the LED's take 4 bytes per command, 4 RGB LED's with 3 elements a piece, so sending 48 bytes per frame).  Anyways, it isn't a problem because it worked very well, so it isn't my serial port, it isn't my microcontroller, and it isn't my Processing code.

I dual-boot Win7 and Ubuntu on this PC, and I installed Processing in Ubuntu.  I copied in my code and, without the serial port code, the FFT graph works very well at a smooth frame rate.  However, once I add the LED code it diminishes the FPS to an abysmal rate, probably around 3-5 FPS.  I'm assuming there's something with Linux that is keeping my serial port from sending data at a fast rate (baud is 19200 so it should be able to send a lot of data and fast).