We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.
IndexProgramming Questions & HelpSound,  Music Libraries › Loading .ogg > java+processing visualizationize
Page Index Toggle Pages: 1
Loading .ogg > java+processing visualizationize (Read 2134 times)
Loading .ogg > java+processing visualizationize
Jan 21st, 2007, 10:21pm
 
This would be my dream. I only suggest it because .ogg's read a lot faster then mp3's and because I saw this plugin:

http://www.tritonus.org/plugins.html

Could it be used to stream stereo audio through ess, and then analyze the fft's and then display the letters "S M G" bumping to the beats?

or shall my dream be just another wisp of unverified thoughts...

reply naow.

edit: my experience with java: I know java + have learned data structures, and use tree maps whenever I can. (I love em). Therefore, processing is not for me (?) i havent found another plugin that could save the age of the applet from the brinks other than this. processing is amazing.
Re: Loading .ogg > java+processing visualizatio
Reply #1 - Jan 22nd, 2007, 10:49am
 
ESS only supports mono currently I think, but since it's built to work with the tritonus library ( http://www.tree-axis.com/Ess/mp3_import.html ), it should be able to load .ogg files as well as mp3.

As for working out the beat from the FFT, I'm not entirely sure how to do that, but I'm sure there's people with more experience of audio-reactive systems who may be able to help.
Re: Loading .ogg > java+processing visualizatio
Reply #2 - Jan 22nd, 2007, 1:52pm
 
I managed to load the .ogg & .mp3 audio through a jsresources.org's scripts, will I be able to still analyze the audio? *is listening to my music through it atm*

I'm fairly sure I could hook it up... *will do more research*
Re: Loading .ogg > java+processing visualizatio
Reply #3 - Jan 23rd, 2007, 12:27am
 
K, yes, I think my next path of action should be to find how "live input" works, and somehow route it to the audio stream I now have running, and use it for data.

Anyone else make any discoveries on this front?

edit: I made one:

write before the "line.write(abdata, 0, nbytesRead)" line in jsresources's audio reader, I added:

System.out.println(abdata[0])

^ abdata is an array of info sent to the mixer! cool! If I had a class that could do something with tihs array, I could draw pretty pictures!

abdata[0] btw returns seemingly random numbers, so it does have to do something with the audio.

edit#2: *researching how to have multiple threads: I need the audio to be loaded by one thread, and analyzed by another. (draw is being executed like... once a second XD)
Re: Loading .ogg > java+processing visualizatio
Reply #4 - Jan 23rd, 2007, 2:56am
 
Found this on jsresource:
Quote:
Java Sound is an API concerned with basic sound input and output. It does not contain digital signal processing algorithms. Nevertheless, you can do this with Java; you just have to code it on your own. Craig Lindley's book (see Q: 1) contains some DSP algorithm. Also, it is often easy to transform C or C++ code found on the net to Java.

You may want to have a look at the comp.dsp FAQ

For code that does fft, have a look at the Peruna Project (original website is offline, view it at the Internet Archive). (Matthias)


So, yes, I need this. atm it just draws lines on my applet of the raw bits of data. The "peruna project" is not even on the internet archive anymore (it's not! i searched for an hour.)

Does anyone have some examples that were on this peruna project?

once again I have access to the bits of the audio being written to the speakers from my .ogg file... I just need to get the values into something that makes sense for the audio coming out ("no sound" seems to be just as loud as "some sound".)
Re: Loading .ogg > java+processing visualizatio
Reply #5 - Jan 23rd, 2007, 10:57am
 
The data you're getting is the raw data sent to the soundcard, what yuo need is to do an FFT (Fast Fourier Transform) on it. If you search, you should be able to find some Java implementations, that will allow you to convert the data to something useful.
Re: Loading .ogg > java+processing visualizatio
Reply #6 - Jan 23rd, 2007, 2:36pm
 
K. I did it and came up with a java wrapper for

http://www.fftw.org/

(I imported it to eclipse.)

err now what. the "entire thing" looks like fft operations, what specifically do I want to go from byte[] > useful audio data?

the package seems to hvae a "wisdom" and "path" directory...

I'm very confused, but this seems to be the "final stretch" before I can get my visualizationizer working.

And in other news, I saw a really cool processing generated splatter.mov from david brawes or something... so my hope was rekindled.

SO yes: my current question, what do I do with these fft packages? there are subroutines like "ulaw > alaw" and pcm backwards and pcm forwards, and no javadocs Sad

and especially google searching the site for
audio site:www.fftw.org
returns nothing... is this not the "normal" application for getting audio data back?
Re: Loading .ogg > java+processing visualizatio
Reply #7 - Jan 23rd, 2007, 3:21pm
 
I strongly reccomend looking for a real Java one rather than just wrapping a C one, bonus points for finding one that's designed for audio.

In general terms the FFT will take your audio data, and return a specturm analysis of different frequencies, the sort of thing you see in winamp and iTunes and the like (an array of values, each representing the "volume" of different frequency ranges)
Re: Loading .ogg > java+processing visualizatio
Reply #8 - Jan 23rd, 2007, 7:46pm
 
You do know that there is already a Processing library called Ess that will handle live input for you and also has an FFT class, right? I would recommend using that over trying to work with the lower level JavaSound API directly and trying to get it to play nice with a bound FFT library.
Re: Loading .ogg > java+processing visualizatio
Reply #9 - Jan 23rd, 2007, 11:31pm
 
  • I want stereo.
  • Unless theres a way to get ESS to watch the bitstream I currently have going. I tried ESS, and I couldnt get it to load the file. The fact is that it has to read from a file, and ess isnt doing that for me.


  • So... Write a tutorial Cheesy!

    *anyone have any other fft libraries I can look at? there has to be SOMEONE out there who has researched audio streams before <_<

    edit: Heyyyy I just realized: can I use ess to "Watch" the current audio stream i have going? then I could use JSound to get stereo audio, but only look at one of the channels for ess (I mean, the visualization doesnt have to be channel responsive... Its just a shame if the audio has to be mono-ized...)

    Found this: http://skruntskrunt.abez.ca/software.html#enveloper
    Re: Loading .ogg > java+processing visualizatio
    Reply #10 - Jan 24th, 2007, 1:37am
     
    This didn't work:

    Code:
    public void draw(){
    background(0,0,0);
    setSize(200,200);
    stroke(255,255,255);
    if (!isrunning&&authread.line!=null){
    if (authread.line.isOpen()){
    isrunning = true;
    }
    }
    if (isrunning){
    //see jsresource's audio player to discover what "abdata" is.
    byte[] data = authread.abData;
    float[] data2 = new float[data.length];
    for (int a = 0; a < data.length; a++){
    data2[a] = new Byte(data[a]).floatValue();
    }
    data2 = fftMag(data2);
    for (int a = 0; a < data2.length; a++)
    ellipse(a,data2[a]+100,10,10);
    }
    }


    Code:
    private int n, nu;
    boolean IsPowerOfTwo (int value)
    {
    return (value & -value) == value;
    }
    public float[] fftMag(float[] x) {
    // assume n is a power of 2
    n = x.length;
    while (!IsPowerOfTwo(n)){
    --n;
    }
    nu = (int)(Math.log(n)/Math.log(2));
    int n2 = n/2;
    int nu1 = nu - 1;
    float[] xre = new float[n];
    float[] xim = new float[n];
    float[] mag = new float[n2];
    float tr, ti, p, arg, c, s;
    for (int i = 0; i < n; i++) {
    xre[i] = x[i];
    xim[i] = 0.0f;
    }
    int k = 0;

    for (int l = 1; l <= nu; l++) {
    while (k < n) {
    for (int i = 1; i <= n2; i++) {
    p = bitrev (k >> nu1);
    arg = 2 * (float) Math.PI * p / n;
    c = (float) Math.cos (arg);
    s = (float) Math.sin (arg);
    tr = xre[k+n2]*c + xim[k+n2]*s;
    ti = xim[k+n2]*c - xre[k+n2]*s;
    xre[k+n2] = xre[k] - tr;
    xim[k+n2] = xim[k] - ti;
    xre[k] += tr;
    xim[k] += ti;
    k++;
    }
    k += n2;
    }
    k = 0;
    nu1--;
    n2 = n2/2;
    }
    k = 0;
    int r;
    while (k < n) {
    r = bitrev (k);
    if (r > k) {
    tr = xre[k];
    ti = xim[k];
    xre[k] = xre[r];
    xim[k] = xim[r];
    xre[r] = tr;
    xim[r] = ti;
    }
    k++;
    }

    mag[0] = (float) (Math.sqrt(xre[0]*xre[0] + xim[0]*xim[0]))/n;
    for (int i = 1; i < n/2; i++)
    mag[i]= 2 * (float) (Math.sqrt(xre[i]*xre[i] + xim[i]*xim[i]))/n;
    return mag;
    }
    private int bitrev(int j) {

    int j2;
    int j1 = j;
    int k = 0;
    for (int i = 1; i <= nu; i++) {
    j2 = j1/2;
    k = 2*k + j1 - 2*j2;
    j1 = j2;
    }
    return k;
    }


    Am I on the right track?
    Re: Loading .ogg > java+processing visualizatio
    Reply #11 - Jan 24th, 2007, 4:04am
     
    Current status:

    http://www.smgroove.com/downloads/XMAS/MOV00351.MPG

    That's without any FFT or anything. I just decided to make the "viewing area" smaller.

    wooooo so I'm back to my question: how do I leave just that "middle line"? and how can I use it to get to "specific pitches" in a piece of audio?

    I swear i've done extensive research but havent gotten blunt enough answers to these questions....

    Edit: Btw, in this vid its reading a .ogg from my usb connected external harddrive. I have another one where it reads a .mp3 from our website, and one where it loads a .wav from my HD.
    Re: Loading .ogg > java+processing visualizatio
    Reply #12 - Jan 25th, 2007, 4:27am
     
    java says this:

    Quote:
    If you're processing incoming sound, you can read the bytes from a TargetDataLine and then manipulate them. An algorithmically trivial example that can yield sonically intriguing results is to play a sound backwards by arranging its frames in reverse order. This trivial example may not be of much use for your program, but there are numerous sophisticated digital signal processing (DSP) techniques that might be more appropriate. Some examples are equalization, dynamic-range compression, peak limiting, and time stretching or compression, as well as special effects such as delay, chorus, flanging, distortion, and so on.


    K. so its the only way XD

    I believe this action to be the fundamental coding behind any processing sound project. Coders to action! Hear my rallying call and come forth!

    we need a method of taking those bits and getting useful data <_<
    Re: Loading .ogg > java+processing visualizatio
    Reply #13 - Jan 25th, 2007, 11:11am
     
    The problem is that you've gone completely outsdie the realm of Processing and are pretty much doing normal Java programming, so you're much more likely to be able to get help on a normal Java forum than here.
    Re: Loading .ogg > java+processing visualizatio
    Reply #14 - Jan 25th, 2007, 2:31pm
     
    Will do, but I will report back here to post my findings
    Page Index Toggle Pages: 1