Newbie with an EEG neuro-feedback based app

Share your Work

evoenc

Newbie with an EEG neuro-feedback based app

in Share your Work • 1 year ago

Hi all! I am a longtime lurker of this forum, and I finally joined. I have gotten a lot of good info from the folks here.

I just created my first distribution app. I feel like I have passed a milestone. It is a game made for the Neurosky EEG headset (I built it against my MindWave). It is called Zen-Fire Meditation Toys, and as the name implies, it has a few modes of play that might help a meditator, or maybe it might just be fun. The hardest part to code was the Zen-Fire and the Freeform modes, which feature pretty graphical glowing orbs dancing around based on your EEG input values from the MindWave.

There are a lot of settings for the toys, so a player can personalize it. I have a few ideas as to how to improve the game, to make it a bit more fun. But, for the moment, I got it stable (in my testing) and it is in alpha on sourceforge. I even made some very basic documentation for it. It is GPL, but the code is a mess, and I would like to scrub it well before releasing it.

If anyone here has a MindWave or MindSet and would like to give it a whirl, I'd appreciate any feedback. I don't have a linux version posted because I don't have a linux machine to test it on, but I can make one on request.

https://sourceforge.net/projects/zenfire/files/

Replies(19)

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Probably nobody on this forum has downloaded the app. It requires a Neurosky headset, after all.

But, I have made some changes that corrected a bug (I think it is corrected!)

If anyone has one or two minutes to kill, could you please download a copy of the app at the link, just to see if it launches on your computer? It would help a lot!

http://sourceforge.net/projects/zenfire/files/

Thanks!

Jesse Sco..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

It launches fine for me... obviously I don't have a EEG ... yet!

OSX 10.6.8

Java SE 1.6

~ J

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Thanks Jesse, I appreciate it.

I'd love to hear from Win 32 and Win 64 bit users! Thanks!

And, Jesse... the Neurosky toy is only about US$120. Neat little piece of equipment.

Jesse Sco..

Re: Re: Newbie with an EEG neuro-feedback based app

1 year ago

Ok, just tried both on my Windows partition...

32-bit opens fine for me, but the 64-bit doesn't... just won't open.

Haven't looked at the Java Console at all...

I'm on a 64-bit install, but I don't know offhand what my Java VM is set to...

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Jesse, thank you for that observation. One of the users noted something similar in a prior trial. They have Win 64 also, but it was the 32 bit version which actually opened for them.

Reading about it, it is related to which version of Explorer the user is running, not just the OS. Win 64 comes with both bit versions of Explorer, and which you use determines which JVM you install and run. So, even on a Win XP or 7 64-bit installation, you can still be running the 32 bit JVM. This can be irritating and confusing. I will have to update the installation docs and the manual to help with this.... basically, telling users to try both versions of the application after installing the JRE.

ref: http://www.java.com/en/download/faq/java_win64bit.xml

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Hi evoenc,

I am currently experimenting with a Mindwave Mobile and I would love to try Zen-Fire on either Linux or maybe even port it to Android. What would I have to do to get it running on Linux and where can I find the sources to try porting it to Android?

Martin

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Hi Martin, at this point, I am still de-bugging code. So, although I released it at one point, I pulled it for an update, and I am slowly slogging through making some repairs and neatening it up. But, I can certainly try working up a Linux version of the app.

I cannot say (without looking it up) if Neurosky has ThinkGear Connector for Linux. If it does, then you could use Zen-Fire on Linux. Be aware that I haven't been able to test on Linux at all. Later today I can make a version and put it on sourceforge for you.

On Android... The game is not really designed to be on a phone or tablet, so I can't say how it would work out. It would be interesting to find out how it goes. I am not familiar with porting to the platform, however, so I wouldn't be able to help with that. Due to the fact that I haven't had many takers on the software, I have not been putting a lot of effort into cleaning up the code in a hurry. Hope you can wait a bit for the code, I'd really like to refactor it before releasing it again.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

I haven't yet tried out Processing on Android, I just saw that there is a page on the Processing wiki about it. It might well be that a phone's processor isn't able to perform well on this task.

On Linux: There is a third-party library that can read Mindset data. I haven't tried if it's compatible with the Mindwave Mobile though. Take your time for the refactoring, my calendar is quite stuffed anyway, so I don't know when I will find a big enough block of time to make real progress in porting.

My final goal is to implement the Alpha-Theta Training protocol on my mobile phone in some way or the other. I am currently working on a branch of Binaural-Beats for Android, but that project does not use EEG data per se and it would be advantageous to join forces on a more specialized project.

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Ok, my week is going to be busy as well. I am away from the home office for a few days, so unfortunately I can't run off your Linux version just yet. Sorry for the delay.

I can understand the desire for portable Alpha-Theta. I built my app to support A/T for my own use. It also includes a binaural "Theta driver" for the special game that starts at SMR for 5 minutes, goes to A/T for as many minutes as you want, then does SMR again for 5 mins.

I am also adding a new feature, music-as-reward, which is good for A/T. I will be using this code in your Linux release. My suggestion: in the meantime, see if you can get Neurosky's ThinkGear connector and some of the Neurosky out of the box games running on your Linux system. That will get a lot of troubleshooting out of the way.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

I tried Zen-Fire on Windows and it connects to my Mindwave Mobile correctly using the Mindwave Mobile specific Thinkgear Connector. Unfortunately I can't hear any sounds other than the binaural beat from the Theta driver and the music from the folder I specified. I tried all programs and setting instruments for all channels in the Free form program, but none worked... Are the sound files maybe missing from the bundle or do the sounds get generated in Java?

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Nice! I got the Mindwave Mobile running under Linux using the newest version of Puzzlebox Synapse (Zip download at the bottom of the page). After installing some Python packages

sudo apt-get install python-pyside, python-bluez and python-qt4

and creating a rfcomm device via

sudo rfcomm connect rfcomm0 MY:BL:UE:TO:OT:H1 (insert the address of your Mindwave Mobile here)

I was able to run

/usr/bin/env python synapse-gui.py

in the directory with the unzipped Synapse files and select /dev/rfcomm0 as Mindwave device.

Ready to move forward on Linux :)

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Great that you got the Linux NS software up and running. Promising start.

Ok, with Windows, you have hit a problem that I have not been able to hunt down just yet. But, what you wrote gives a major hint as to how to address it. Most of the sounds are handled via the Processing version of Java Midi library, and the Theta Driver is handled via a separate audio library. The Midi library is the hitch here, obviously. (An older version used sound files, but it got quite weighty. And I couldn't provide the variety of sounds using files that the library could. I wanted a smaller, more capable app!)

Can you supply what version of Windows you're using, including 32/64 bit, and what version of the Java JRE you have running? I might need this info to ask for help from the midi library developer.

If you're running an old Java version, would you be willing to install a newer version and try again? And -- once I get the chance to build a version Zen-Fire to help troubleshoot this -- would you be willing to help track this down?

**Edit:

The Processing Midi library that I am using is SoundCipher, and on its site, it states:

Troubleshooting

For Windows users who get no sound when playing a note. Possibly the JavaSound soundbank.gm file is not installed. Some versions of the JRE (Java Runtime Environment) do not include the soundbank in order to save space. Try installing a full (development) version of Java which will include the sound bank.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

I'm on a 32 bit Windows 7 and have a 32 bit JRE7 and JDK7 installed. As mentioned on the SoundCipher page and some Stackoverflow pages I tried putting the soundbank.gm into all folders I could think of, to no avail.

I then installed Processing itself together with the SoundCipher library and tried the examples there and they work (and sound quite good :)

Could you maybe post the sources somewhere, preferrably Github? I'm new to Processing but I have a lot of experience with Java, so I think I should be able to understand the code and maybe track down the issue.

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Hi Martin, I have placed a Linux build on sourceforge, it contains the source... but beware, it is a mess! I used Processing as a rapid prototype environment -- with emphasis on "rapid" ;) As mentioned, it needs refactoring and much of the code needs to be properly placed into classes. It is not clean code that I am proud to share at this point, but in the interest of hunting this bug, I hand it over to you.

My suspicion about the sound issue is now this: I am using the Minim Library sound controls for Gain and Volume. My understanding of these controls is that an implementation of the JRE either uses Gain or Volume, never both. I have not sat in front of a PC that uses Volume, only Gain. So, I have only been able to code accurately against Gain, and there might be problem areas for implementations that use Volume. In the code, look for the variable "gainControl".

Let me know what you find, I will make any corrections as soon as I can. I hope the Linux version works for you.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Hi evoenc,

unfortunately, I can't get my Mindwave Mobile to work under Linux right now, some strange thing with permission denied for the /dev/rfcomm0 device. But Puzzlebox Synapse (the Linux alternative to the Thinkgear Connector) has a simulation mode where it emits random data. When running the Linux version with the simulated data I also get no sound except for the Theta Driver.

On Windows I have tried to track down the problem and I have this minimal example:

import arb.soundcipher.*;

// Comment out the line below and everything works

import ddf.minim.*;

SoundCipher sc = new SoundCipher(this);

int counter = 0;

void draw() {

counter += 1;

if(counter % 100 == 0) {

println("bing");

sc.playNote(60, 100, 2.0);

}

If I run that code as is I get a

Exception in thread "Thread-3" java.lang.ArrayIndexOutOfBoundsException: 15

at javazoom.jl.decoder.LayerIDecoder$SubbandLayer1Stereo.read_allocation(Unknown Source)

and I don't hear any sound (probably the sound engine crashes all together). I've googled for that exception and it doesn't seem to be Processing specific. There are a lot of search results where the problem is an MP3 file with tags which should not be the case here.

If I comment out the minim import then everything works and I hear a piano note every second. Can you reproduce this? BTW, I'm using Processing 1.5.1 and tried it with the shipped version of Minim and the 2.1.0 beta.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Correcting myself: The problem indeed does seem to be MP3 related. In the full stacktrace

Exception in thread "Thread-3" java.lang.ArrayIndexOutOfBoundsException: 15

at javazoom.jl.decoder.LayerIDecoder$SubbandLayer1Stereo.read_allocation(Unknown Source)

at javazoom.jl.decoder.LayerIDecoder.readAllocation(Unknown Source)

at javazoom.jl.decoder.LayerIDecoder.decodeFrame(Unknown Source)

at javazoom.jl.decoder.Decoder.decodeFrame(Unknown Source)

at javazoom.spi.mpeg.sampled.convert.DecodedMpegAudioInputStream.execute(Unknown Source)

at org.tritonus.share.TCircularBuffer.read(TCircularBuffer.java:134)

at org.tritonus.share.sampled.convert.TAsynchronousFilteredAudioInputStream.read(TAsynchronousFilteredAudioInputStream.java:189)

at com.sun.media.sound.AudioFloatInputStream$DirectAudioFloatInputStream.read(AudioFloatInputStream.java:178)

at com.sun.media.sound.SoftAbstractResampler$ModelAbstractResamplerStream.nextBuffer(SoftAbstractResampler.java:208)

at com.sun.media.sound.SoftAbstractResampler$ModelAbstractResamplerStream.open(SoftAbstractResampler.java:135)

at com.sun.media.sound.SoftVoice.processControlLogic(SoftVoice.java:614)

at com.sun.media.sound.SoftMainMixer.processAudioBuffers(SoftMainMixer.java:505)

at com.sun.media.sound.SoftMainMixer$2.fillBuffer(SoftMainMixer.java:880)

at com.sun.media.sound.SoftMainMixer$2.read(SoftMainMixer.java:893)

at javax.sound.sampled.AudioInputStream.read(AudioInputStream.java:292)

at com.sun.media.sound.SoftSynthesizer$WeakAudioStream.read(SoftSynthesizer.java:107)

at javax.sound.sampled.AudioInputStream.read(AudioInputStream.java:292)

at com.sun.media.sound.SoftJitterCorrector$JitterStream$1.run(SoftJitterCorrector.java:186)

at java.lang.Thread.run(Thread.java:722)

there is a line with DecodedMpegAudioInputStream.execute

It looks like the MIDI engine uses an MP3 file to play notes?

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Okay, so the culprit is mp3spi.jar located in the lib folder. This is a SPI (service provider implementation) for playing audio, and it seems to have problems with some MP3 files. If I remove this jar I get all MIDI sounds in both 0.89 and 0.88. Unfortunately, music playback does not work anymore (no surprise), but having real neuro-feedback is much more important anyway.

evoenc

Re: Newbie with an EEG neuro-feedback based app

1 year ago

Hi Martin, thanks for taking the time to hunt this down, I appreciate it. I will have to direct this issue over to the author of the Minim library, and maybe others, so it would be helpful if I had all the info down from your installation:

- Windows version/Linux version

- JRE version

Not all Windows users experience this issue. So, this is somewhat exceptional. I will pose the problem for minim troubleshooting and see if I can get to the core of the issue and fix it.

Otherwise, to your other general questions, the SoundCipher library does not use mp3 play functions, to the best of my knowledge. And, certainly not from my app. I use Minim's mp3 play functions in the app to play either background music (which can be enjoyable), and to use mp3 music as the reward itself... which I find exceedingly helpful for closed-eyes practices, including A/T. I would like this feature to work for you, and I will follow up the best that I can. If you have music in wav format, it might be able to play it without that mp3 jar -- try it out.

Please continue to let me know if you have any further questions. (And send me your Windows & Linux versions and their JRE versions) In the meantime, I hope you enjoy the other features of the game.

martin.ma..

Re: Newbie with an EEG neuro-feedback based app

1 year ago

I'm on a 32 bit Windows 7 and have a 32 bit JRE7 b6 beta and JDK7 b6 beta installed. Before that, I was on JDK7 b5. It seems that Java 7 has a new default MIDI sequencer (Gervill) implementation. That might well be a problem here.

I'll be on vacation for 3 weeks starting tomorrow