how to make "soundboard" work on phone

rhythmiccycle

A couple of months ago I made the following post, about how to get sound to play.

https://forum.processing.org/two/discussion/comment/53262

@akenaton was extremely helpful, and helped me get sound to play on my phone. Now I'm ready to take the program to the next level. Eventually I want to make an app to help me learn Cantonese. Right now I'm just trying to make a program that will play 6 different sound files, depending on what button is pressed. Each file is a mp3 and is only one word, so its about 1 second long.

the code below runs fine in java mode:

Button[] b;
import ddf.minim.*;
Minim minim;


void setup() {
  //  size(1080,1920);//phone res
  size(360, 640);// 1/3 of phone res

  minim = new Minim(this);

  rectMode(RADIUS);  
  b= new Button[6];
  b[0] = new Button(width/6, height-width/6, "sin1.mp3");
  b[1] = new Button(3*width/6, height-width/6, "sin2.mp3");
  b[2] = new Button(5*width/6, height-width/6, "sin3.mp3");
  b[3] = new Button(width/6, height-3*width/6, "sin4.mp3");
  b[4] = new Button(3*width/6, height-3*width/6, "sin5.mp3");
  b[5] = new Button(5*width/6, height-3*width/6, "sin6.mp3");
//there are 6, 1 second, mp3 files in the folder with the project. 

}
void draw() {
  background(255);
  for (int n = 0; n < 6; n++) {
    b[n].update();
  }
}

//---------------------------------------------------------
//---------------------------------------------------------
//---------------------------------------------------------

class Button {
  float x, y, r;
  color myColor;
  boolean initialPress;
  String mp3;
  AudioPlayer player;


  Button( float myX, float myY, String myMp3) {
    x=myX;
    y=myY;
    r=width/7;
    myColor=color(200, 200, 10);
    initialPress=true;
    mp3=myMp3;
  }


  void update() {
    fill(myColor);
    rect(x, y, r, r);

    if (mousePressed && over() && initialPress ) {//when the button is pressed (initially) , this code runs
      initialPress=false;
      player = minim.loadFile(mp3);
      player.play();
    } else if ( !mousePressed || !over()) {//when the button is NOT being pressed, this code runs
      initialPress=true;
    }
  }
 
  boolean over() {
    float d = max(abs(x-mouseX), abs(y-mouseY)); // this is a square
  return d <= r ;
   }
 
 }
 </code>

I want this to run on my phone. However I learned before that Minim doesn't seem to work in android mode, and I have to use android.media.MediaPlayer instead. I'm not really sure how to even begin doing this. I tried some things before, but when I had multiple sound files in a folder the android media player would just play all files one by one. Therefor I don't even what to show you may attempt to get the android media player to work, because I know I did it all wrong.

I want to get the above code to run on my phone. PLEASE HELP

akenaton

@rhythmiccycle===

a) of course you cannot use Minim for that (though i have seen, i believe, that there is an attempt by di fede to port minim to android) and you must use either mediaplayer or soundPool (which is for very short sounds)

b) you write "the code below runs fine in java mode"

as it is your code cannot run even in java mode: your over() method is supposed to return a boolean and is returning a float (d)

c) i dont see what is the difference between playing 1 or 6 sounds and what do you mean by "the android media player just play files one by one"

d) mousePressed (or events of this kind) cannot work with android; you have to use the motionEvent class and, in your case, the Action_Down or Action_Up events.

e) there is a Button class in android...

f) though i can find workarounds for adding buttons and so on, i think that what you want could be easily done with eclipse or Android studio and is (against the general aim for P5...) tricky with P3XX android mode...in this moment! - think to P3XX android mode like a kind of graphics lib, not much.

rhythmiccycle

to address (b)

the end of the code was not displaying, when i went to edit post it was there, but it didn't show up in the actual post.

I edited the code above. It works now.

the end of the code is

boolean over() { float d = max(abs(x-mouseX), abs(y-mouseY)); // this is a square return d <= r ; } }

so it returns "true" is the mouse is over the square.

(i'll address your other points soon)

rhythmiccycle

@akenaton to address (a) found this, and it is very similar to what i'm trying to do. http://www.101apps.co.za/index.php/articles/using-android-s-soundpool-class-a-tutorial.html

However this tutorial is for java, using eclipse. I found some code on message boards for soundpool for processing, but I can't get any of them to run. For example this page: https://forum.processing.org/one/topic/help-with-soundpool-and-general-audio-for-android-using-the-emulator.html

when I try to run that code, I get the error " the class SoundPool does not exist" among other errors.

to address (d) I have successfully used "mousePressed" in another program the ran fine on my phone. I'm sure motionEvent is better, but I'll worry about that later.

...(e) I like building my buttons from scratch.

...(f) If it can be done in processing I'd prefer that, If I need to switch to java I guess I will, but I'll have to go back to just getting a "hello world" to run on my phone.

to address (c) I'll try to run it with mediaplayer Like I did before, and I'll post that code maybe later today or tomorrow.

akenaton

@rhytmiccycle===

i usually work with eclipse+android native but more by curiosity than necessity it happens that i try to do something with P3XX android mode; i have tried successfully with sound pool which is the better class to play very short sounds. An i am sure that you can import this class!

as for motionEvent, depends:: if you want only to capture the click it is not mandatory but if you want to get all the other events (swipe, longPress,multitouch you have to use it. Trying to rewrite your code i have done that.

as for android buttons vs your rects, yes, your code can work as it is; yet it s easy in a for loop to create a lot of android buttons, setting backgrounds, adding onClickListener() and so on.

your code (rewritten) + mediaPlayer can work, i have tested it.

As for using eclipse, if you want, later, to really work for phones you ll see that it is the best way (or Android Studio); i add that it is simpler than rying to find workarounds with P5!

rhythmiccycle

....part (c)....

below is my attempt to get it to work with media player. It runs on the phone, but when I push any button it just plays all the mp3's in the data folder one by one, and all the buttons do the same thing. Maybe I need to add some type of stop at the end of file, but i'm not sure how to fix this.

I tried to just use one button, but it still plays all the files in the data

import java.io.File;
import java.io.IOException;
import android.media.MediaPlayer;
import android.media.MediaPlayer.OnPreparedListener;
import android.content.res.AssetFileDescriptor;
import android.content.res.Resources;
import android.content.res.AssetManager;
import android.content.Context;
import android.app.Activity;
import android.os.Bundle;
import android.app.Application;

/////////////////////////////////////////////////////////

Button[] b;
//import ddf.minim.*;
//Minim minim;//audio context
/////////////////////////////////////////////////////

Context context; 
Activity act;

////////////////////////////////////////////////////
void setup() {
  // size(1080,1920);//phone res
  orientation(PORTRAIT);
  //size(360, 640);// 1/3 of phone res

  //minim = new Minim(this);

  act = this.getActivity();
  context = act.getApplicationContext();


  rectMode(RADIUS);  
  b= new Button[6];
  b[0] = new Button(width/6, height-width/6, "sin1.mp3");
  b[1] = new Button(3*width/6, height-width/6, "sin2.mp3");
  b[2] = new Button(5*width/6, height-width/6, "sin3.mp3");
  b[3] = new Button(width/6, height-3*width/6, "sin4.mp3");
  b[4] = new Button(3*width/6, height-3*width/6, "sin5.mp3");
  b[5] = new Button(5*width/6, height-3*width/6, "sin6.mp3");
}
void draw() {
  background(255);
  for (int n = 0; n < 6; n++) {
    b[n].update();
  }
}

//---------------------------------------------------------
//---------------------------------------------------------
//---------------------------------------------------------

class Button {
  float x, y, r;
  color myColor;
  boolean initialPress;
  String mp3;
  //AudioPlayer player;
  MediaPlayer mp;
  AssetFileDescriptor afd;

  Button( float myX, float myY, String myMp3) {
    x=myX;
    y=myY;
    r=width/7;
    myColor=color(200, 200, 10);
    initialPress=true;
    mp3=myMp3;
  }

boolean over() {
    float d = max(abs(x-mouseX), abs(y-mouseY)); // this is a square
    return d < = r;
  }


  void update() {
    fill(myColor);
    rect(x, y, r, r);

    if (mousePressed && over() && initialPress ) {
      initialPress=false;
      // player = minim.loadFile(mp3);
      //player.play();

      try {
        mp = new MediaPlayer();

        afd = context.getAssets().openFd(mp3);//which is in the data folder
        mp.setDataSource(afd.getFileDescriptor());
        mp.prepare();
      } 
      catch(IOException e) {
        println("j'arrive pas à préparer");
      }
      mp.setLooping(false);
      mp.start();
    } else if ( !mousePressed || !over()) {
      initialPress=true;
    }
  }

  
}

akenaton

@rythmiccycle=== of course! you have to create an array with your players: six players!!! not only one!!! (and also in order to avoid illegal state error add a condition: if(players[i].isPlaying()== false)....launch the sound () else (stop it, i dont know!)

and you have also to create methods on pause && on stop to release the players (like in minim)

rhythmiccycle

I thought the player array was:

b[0].mp

b[1].mp

b[2].mp

Etc...

I guess it doesn't work like that. I'll create the players is the setup, rathet than in the class, and try again later today.

rhythmiccycle

I got it working with sound pool. I couldn't get media player to work. I found the sample code in the line I posted at the start of this thread. I didn't realize it was there.

I don't really understand whats going on with the "hash map." Specifically the lines: soundPoolMap = new HashMap<Object, Object>(2); soundPoolMap.put(1, soundPool.load(afd1, 1));

Below is the whole program:

Button[] b;

import android.app.Activity;
import android.content.res.AssetFileDescriptor;
import android.content.Context;
import android.media.MediaPlayer;
import android.media.SoundPool;
import android.media.AudioManager;


SoundPool soundPool;
HashMap soundPoolMap;
Activity act;
Context cont;
AssetFileDescriptor afd1, afd2, afd3, afd4, afd5, afd6;

void setup() {
  //  size(1080,1920);//phone res
  // size(360, 640);// 1/3 of phone res
  orientation(PORTRAIT);

  rectMode(RADIUS);  
  b= new Button[6];
  b[0] = new Button(width/6, height-width/6, 1);
  b[1] = new Button(3*width/6, height-width/6, 2);
  b[2] = new Button(5*width/6, height-width/6, 3);
  b[3] = new Button(width/6, height-3*width/6, 4);
  b[4] = new Button(3*width/6, height-3*width/6, 5);
  b[5] = new Button(5*width/6, height-3*width/6, 6);

  act = this.getActivity();
  cont = act.getApplicationContext();

  // load up the files
  try {
    afd1 = cont.getAssets().openFd("sin1.mp3");
    afd2 = cont.getAssets().openFd("sin2.mp3");
    afd3 = cont.getAssets().openFd("sin3.mp3");
    afd4 = cont.getAssets().openFd("sin4.mp3");
    afd5 = cont.getAssets().openFd("sin5.mp3");
    afd6 = cont.getAssets().openFd("sin6.mp3");
  } 
  catch(IOException e) {
    println("error loading files:" + e);
  }
  // create the soundPool HashMap - apparently this constructor is now depracated?
  soundPool = new SoundPool(12, AudioManager.STREAM_MUSIC, 0);
  soundPoolMap = new HashMap(2);
  soundPoolMap.put(1, soundPool.load(afd1, 1));
  soundPoolMap.put(2, soundPool.load(afd2, 1));
  soundPoolMap.put(3, soundPool.load(afd3, 1));
  soundPoolMap.put(4, soundPool.load(afd4, 1));
  soundPoolMap.put(5, soundPool.load(afd5, 1));
  soundPoolMap.put(6, soundPool.load(afd6, 1));
}
void draw() {
  background(255);
  for (int n = 0; n < 6; n++) {
    b[n].update();
  }
}

///-----------------------------------------------------------
//--------------------------------------------------------------
///----------------------------------------------------------

class Button {
  float x, y, r;
  color myColor;
  boolean initialPress;
  int mp3;

  Button( float myX, float myY, int myMp3) {
    x=myX;
    y=myY;
    r=width/7;
    myColor=color(200, 200, 10);
    initialPress=true;
    mp3=myMp3;
  }


  void update() {
    fill(myColor);
    rect(x, y, r, r);

    if (mousePressed && over() && initialPress ) {
      initialPress=false;
      soundPool.play(mp3, 1.0, 1.0, 1, 0, 1f);
    } else if ( !mousePressed || !over()) {
      initialPress=true;
    }
  }

  boolean over() {
    float d = max(abs(x-mouseX), abs(y-mouseY)); // this is a square
    return d < = r;
  }
  
}

akenaton

@rythmiccycle===

a) ---- your code has to work (i have not got any time to test it) with short sounds. Sound Pool uses mediaPlayer to play but instead of prepare when called it "prepares" in Ram when you load the sounds, so it is more quick but cannot exceed some sound duration (1 or 2"); these sounds are stocked in a collection (hashmap= key+value), which can stores a lot of kind of values and keys, not only strings, but also integers or...and that is why you have <Object,Object> (yet, i think that in your case you can use other constructors for a hashMap, does not matter!!!) -

b) As for mediaPlayer (without soundPool) you can do exactly the same thing (it will be less quick with short sounds) and the other code you provide could work if instead of using the same mediaPlayer you have an array of 6 mediaPlayers instead, and that is not your button array!!!! which you create in your setup::

        MediaPlayer[]players;

        public void setup(){
        .....

        rectMode(RADIUS);  
          b= new Boutton[6];
          b[0] = new Boutton(width/6, height-width/6, "sin1.mp3");
          b[1] = new Boutton(3*width/6, height-width/6, "sin2.mp3");
          b[2] = new Boutton(5*width/6, height-width/6, "sin3.mp3");
          b[3] = new Boutton(width/6, height-3*width/6, "sin4.mp3");
          b[4] = new Boutton(3*width/6, height-3*width/6, "sin5.mp3");
          b[5] = new Boutton(5*width/6, height-3*width/6, "sin6.mp3");

          //there are 6, 1 second, mp3 files in the folder with the project.

          players = new MediaPlayer[6];
          for (int i=0;i<players.length; i++){
            players[i] = new MediaPlayer();
          }
        }

/// then in your class or elsewhere (touchEvent) you have::

                                     if(players[nbre].isPlaying() == false){// error if the player is already playing!!!
                                      try {

                                        afd = context.getAssets().openFd(mp3);//which is in the data folder
                                        players[nbre].setDataSource(afd.getFileDescriptor());

                                        players[nbre].prepare();
                                      } 
                                      catch(IOException e) {
                                        println("j'arrive pas à préparer" + e.getMessage());
                                      }

                                      players[nbre].start();//nbre = your button number from 0 to 6...
                                    .....
                                    }

// in this case you CAN have multiple instances (30 or less) playing at the same time... With your code that is not possible, you can only create a new instance AFTER having killed which one was running....

rhythmiccycle

I tried the "MediaPlayer[]players;" thing, and it runs, but it still just plays all the files in the data folder when any button is pressed.

I got it working with sound pool, and that is good enough for me. I give up on mediaPlayer (without soundPool).

Thank you so much @akenaton again!!!! I don't know how I could have ever gotten this to work without your help. You are the Best!

Now I have the tools I need to start building an app to help me learn to speak Cantonese.