Screenshot so far:



I'm trying to reverse-engineer this view:


Goal: Display an audio file so that the user can "mark" downbeats manually up to MS precision, and then save the list of downbeats to a file. Spectral View in Cool Edit Pro is the closest thing I've ever found to this, but it's pay-to-use, and I want to make it so the online community can benefit for free.

What I'm doing:

Use Minim to loadFile() a .mp3 or .wav file. Get the dataLine of that player (I hacked the minim distro just to add the getter and setter) and mute the line. Attach an audioListener to that player. As you get new samples (that won't be played to speakers), fill a LinkedList of FFTNodes of fftdata to be displayed.

Use Minim to LoadFile() the media again, and use this when the user wants to play / pause the media from the cursor position.

Problems:

Sometimes the dataLine of the AudioPlayer doesn't have Mute control (like, my soundcard is performing differently from run to run).

Sometimes when you play the audioPlayer (Note: fairly rarely, but when it happens you have to restart the program) you'll hear TSHHHHHHHH like the tv noise sound. Scares the heck out of me.

There is a noticeable latency when you hit play, which prevents you from definitively saying "I am on a downbeat" because the sound doesn't play until a split second after you hit space. This basically cripples the utility.

Minim can't handle .ogg files, which are 5/6 of all music simfiles online. Like, this utility won't be useful to anyone until it can support .oggs (because people are too lazy to convert music)

In Minim, if I set the bufferSize to ANYTHING other than 1024 (512, 128, 64 all failed) I get "ArrayIndexOutOfBoundsException(65)"... hopefully this will be resolved soon.

Joal would solve some problems with latency, but I can't get it to play ... anything! Also, I doubt I would ever be able to get millisecond precision data on where the playing line is in the file. (Minim does this)

Notes:

I believe Minim will get .ogg support quite soon.

JavaSound will eventually get optimized, and then plays will be faster.

I think if I just play around with the gradient, I might get a bit closer to cool edit's spectral view,

If I can make the FFT buffer size smaller (which should increase the resolution of the peaks) I could also get a more useful spectral view.

I'm going to look into Sonia for the front line, as the back line is more or less performing acceptably.

Yes, I'm just feeling "constricted" by today's audio programming resources, and would like to ask the simple question: am I missing something

Well, it may be helpful for some other audio programmers to partake in some of my discoveries:

1.) AudioSystem.getAudioInputStream( on any PCM file) returns a stream with no mark support! They're basically useless!

2.) Using javaZoom's vorbis library, property "ogg.position.byte" (the only way to get song position) is horribly low granularity! you get udpates ~ every 30 ms!

3.) Of course, one can get around #1 by just always using .mp3 and .ogg files, which is nice and great! But, when you have the problem #2 with oggs, you need to get from ogg > mp3! and you can't do that without going through PCM! and pcm has no mark support so you can't do ANYTHING with it!

4.) To go just a little bit backwards in a file, you have to rewind to the beginning and then rescan the whole file!!! WHY!?!?!?!?

EDIT: oh, well, ok, figured out how mark and reset actually work.

So, I misdiagnosed my condition.

Output using jsresource's Mp3Encoder class
Code:

Input format: VORBISENC 44100.0 Hz, unknown bits per sample, stereo, unknown frame size, unknown frame rate, 
Intermediate: PCM_SIGNED 44100.0 Hz, 16 bit, stereo, 4 bytes/frame, little-endian
Got converted AudioInputStream: org.tritonus.sampled.convert.lame.Mp3LameFormatConversionProvider$EncodedMpegAudioInputStream
Output format: MPEG1L3 44100.0 Hz, unknown bits per sample, stereo, unknown frame size, unknown frame rate,  false
Writing C:\Users\Ben Braun\Desktop\Pad simfiles\Banzaigtv's ITG Simfiles\Are You Serious\Are You Serious.mp3...
 

But the .mp3 file comes out empty!

HOWEVER: if we try to save a .au file to the .mp3, it succeeds! I think it's because the .ogg file is not sending the correct header, and therefore AudioSystem.write can't write anything. (in fact, it does write a few bytes, but not enough to actually hear anything)

and, if I try to write my own loop that does something similar, I get that the "converted stream" doesn't have any available bytes even at the beginning!

(help....)