Video in Android Mode

sgrigg

Hello I am trying to make a project and the first hurdle is making it sense when a viewer is present and play the video/pause video when viewer leaves. I have searched extensively through the forums and other parts of the net and I can't find any code new enough that it is working for me. I am running Processing 3 and I am currently building onto a Samsung Galaxy Tab 3 running 4.4.2. My instincts say that this will be too slow and I will need to figure out how to do it in Java/Eclipse for speed sake, but I wanted to try this first.

Does anyone have any code for efficient video playback that is working for them in Processing 3 Android mode?

Thanks!

akenaton

@sgrigg:: basically you can use the android videoView widget for that + MediaController && MediaPlayer; You create an instance of videoView, add mediaController and preparedListener to that, and set the uri for the video; then "on prepared" you start the video. I have done that with kitkat and gingerbread, its not at all slow!!!

sgrigg

hi again @akenaton! I tried messing around to do what you said but I am pretty new to Java. Am I on the right track?

    import android.media.MediaPlayer;
    import android.content.Context;
    import android.app.Activity;
    import android.widget.VideoView;
    import android.widget.MediaController;
    
    MediaPlayer player;
    Context context;
    Activity act;
    VideoView video;
    MediaController control;
    
    void setup() {
      act = this.getActivity();
      context = act.getApplicationContext();
      try {
        control = new MediaController(context);
        player = new MediaPlayer();
        video = new VideoView(context);
        video.setVideoPath("test.mp4");
        video.setMediaController(control);
        video.setOnPreparedListener(new OnPreparedListener() {
          onPrepared();}
          );
      } 
      catch(IOException e) {
        println("file did not load");
      }
    };
    
    void onPrepared(){
      video.start();
    }

Thanks!

akenaton

@sgrigg:: YES!!!! (i am not sure about 2 things in your code::

a) videoPath, which can be a little more complicated to do, but see what i answered to the user who wanted to play sound...)(or perhaps parse uri) b) perhaps you have to make this code within a thread

i dont know if it works; what i know is that i have already made that and that i have code::: but i am happy to see someone TRYING, so i don t post it, until you are tired out!

sgrigg

@akenaton yes I do always want to try. I am glad that people on the internet are willing to help and I don't want to be the entitled type that expects everyone to just do the coding for me.

that said I am super grateful if you have anymore advice on how I could proceed. I followed your advice and checked out different ways of loading the file. I tried to integrate your solution from the other thread on playing sound. It is still throwing errors though. I am also not so sure that I understand how I should be using "setOnPreparedListener". I looked at the android developer page for it but I am still unsure.

here is where I am currently:

    import android.media.MediaPlayer;
    import android.content.Context;
    import android.content.res.AssetFileDescriptor;
    import android.app.Activity;
    import android.widget.VideoView;
    import android.widget.MediaController;
    
    MediaPlayer player;
    Context context;
    Activity act;
    VideoView video;
    MediaController control;
    AssetFileDescriptor afd;
    
    public void setup() {
      act = this.getActivity();
      context = act.getApplicationContext();
      try {
        control = new MediaController(context);
        player = new MediaPlayer();
        video = new VideoView(context);
        afd = context.getAssets().openFd("test.mp4");
        video.setVideoPath(afd.getFileDescriptor());
        video.setMediaController(control);
        video.setOnPreparedListener(new OnPreparedListener() {
          
        @Override
          public void onPrepared(){
            video.start();
              }
        });
      } 
      catch(IOException e) {
        println("file did not load");
      }
    };
    

    
     public void stop(){
            super.stop();
            if (player !=null){
              player.release();
              player = null;
            }
           };

and here is the error it is throwing

1. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android2300963045517878573sketch/src/processing/test/sketch_151018b/sketch_151018b.java (at line 48)
    video.setVideoPath(afd.getFileDescriptor());
          ^^^^^^^^^^^^
The method setVideoPath(String) in the type VideoView is not applicable for the arguments (FileDescriptor)
----------
2. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android2300963045517878573sketch/src/processing/test/sketch_151018b/sketch_151018b.java (at line 50)
    video.setOnPreparedListener(new OnPreparedListener() {
          ^^^^^^^^^^^^^^^^^^^^^
The method setOnPreparedListener(MediaPlayer.OnPreparedListener) in the type VideoView is not applicable for the arguments (new OnPreparedListener(){})
----------
3. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android2300963045517878573sketch/src/processing/test/sketch_151018b/sketch_151018b.java (at line 50)
    video.setOnPreparedListener(new OnPreparedListener() {
                                    ^^^^^^^^^^^^^^^^^^
OnPreparedListener cannot be resolved to a type
----------
4. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android2300963045517878573sketch/src/processing/test/sketch_151018b/sketch_151018b.java (at line 53)
    public void onPrepared(){
                ^^^^^^^^^^^^
The method onPrepared() of type new OnPreparedListener(){} must override a superclass method
----------
4 problems (4 errors)

As always thank you for your help!

akenaton

@sgrigg== 4 errors - 1 = 3

import android.media.MediaPlayer.OnPreparedListener;

3-1 = 2 if instead of passing afd as an argument you put here the path to your video: videoPath is waiting for an uri, afd gives you a file.

sgrigg

@akenaton thanks yet again!

I am down to one error but I still don't really understand how I should be using OnPreparedListeners.

import android.media.MediaPlayer;
import android.media.MediaPlayer.OnPreparedListener;
import android.content.Context;
import android.content.res.AssetFileDescriptor;
import android.app.Activity;
import android.widget.VideoView;
import android.widget.MediaController;
import android.net.Uri;

MediaPlayer player;
Context context;
Activity act;
MediaController control;
VideoView video;
OnPreparedListener onP;

public void setup(){
  act = this.getActivity();
  context = act.getApplicationContext();
  onP = new OnPreparedListener();
  try {
    Uri uri = Uri.parse("data/test.mp4");
    control = new MediaController(context);
    player = new MediaPlayer();
    video = new VideoView(context);
    video.setVideoURI(uri);
    video.setMediaController(control);
    video.setOnPreparedListener(onP );
  } 
  catch(IOException e) {
    println("file did not load");
  }
};

public void onPrepared(){
        video.requestFocus();
        video.start();
          };
 
 
 public void stop(){
        super.stop();
        if (player !=null){
          player.release();
          player = null;
        }
       };

and here is the one error

1. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android6803413764917291697sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 47)
    onP = new OnPreparedListener(onPrepared());
              ^^^^^^^^^^^^^^^^^^
Cannot instantiate the type MediaPlayer.OnPreparedListener
----------
1 problem (1 error)

I feel like I am learning but definitely still stumbling.

akenaton

@sgrigg:: learning is good! --- think to that: why is the use for my player in this code??? --- what is videoView??? ---what do you do in your onPrepared if you dont find see below....

          video.setOnPreparedListener(new
                MediaPlayer.OnPreparedListener()  {

              public void onPrepared(MediaPlayer mp) {

                System.out.println("ready!");


              }
            });

sgrigg

@akenaton so this finally looks right but now I am getting all of these errors because I don't use a lot of the processing files that get imported. I tried messing around but I am a little confused.

import android.media.MediaPlayer;
import android.media.MediaPlayer.OnPreparedListener;
import android.content.Context;
import android.app.Activity;
import android.widget.VideoView;
import android.widget.MediaController;
import android.net.Uri;

MediaPlayer player;
Context context;
Activity act;
MediaController control;
VideoView video;

public void setup(){
  act = this.getActivity();
  context = act.getApplicationContext();
  try {
    Uri uri = Uri.parse("data/test.mp4");
    control = new MediaController(context);
    player = new MediaPlayer();
    video = new VideoView(context);
    video.setVideoURI(uri);
    video.setMediaController(control);
    video.setOnPreparedListener( new MediaPlayer.OnPreparedListener(){
      public void onPrepared(MediaPlayer player) {
        System.out.println("ready!");
        video.requestFocus();
        video.start();
      }
    }); 
  }
  catch(IOException e) {
    println("file did not load");
  }
};


 
 public void stop(){
        super.stop();
        if (player !=null){
          player.release();
          player = null;
        }
       };

and here are the errors

----------
1. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 4)
    import processing.data.*; 
           ^^^^^^^^^^^^^^^
The import processing.data is never used
----------
2. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 5)
    import processing.event.*; 
           ^^^^^^^^^^^^^^^^
The import processing.event is never used
----------
3. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 6)
    import processing.opengl.*; 
           ^^^^^^^^^^^^^^^^^
The import processing.opengl is never used
----------
4. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 9)
    import android.media.MediaPlayer.OnPreparedListener; 
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The import android.media.MediaPlayer.OnPreparedListener is never used
----------
5. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 16)
    import java.util.HashMap; 
           ^^^^^^^^^^^^^^^^^
The import java.util.HashMap is never used
----------
6. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 17)
    import java.util.ArrayList; 
           ^^^^^^^^^^^^^^^^^^^
The import java.util.ArrayList is never used
----------
7. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 18)
    import java.io.File; 
           ^^^^^^^^^^^^
The import java.io.File is never used
----------
8. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 19)
    import java.io.BufferedReader; 
           ^^^^^^^^^^^^^^^^^^^^^^
The import java.io.BufferedReader is never used
----------
9. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 20)
    import java.io.PrintWriter; 
           ^^^^^^^^^^^^^^^^^^^
The import java.io.PrintWriter is never used
----------
10. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 21)
    import java.io.InputStream; 
           ^^^^^^^^^^^^^^^^^^^
The import java.io.InputStream is never used
----------
11. WARNING in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 22)
    import java.io.OutputStream; 
           ^^^^^^^^^^^^^^^^^^^^
The import java.io.OutputStream is never used
----------
12. ERROR in /private/var/folders/08/3m4jfj5s16q6fd7_qmd1xvbw0000gn/T/android288404626173998018sketch/src/processing/test/sketch_151019a/sketch_151019a.java (at line 59)
    catch(IOException e) {
          ^^^^^^^^^^^
Unreachable catch block for IOException. This exception is never thrown from the try statement body
----------
12 problems (1 error, 11 warnings)

thanks in advance for any and all help!

akenaton

@sgrigg:: dont worry about stupid warnings error is (not easy to solve!!!) your uri (video)

sgrigg

@akenaton I feel silly but I am still so stumped. I have tried this based on some research but I am still getting nothing

import android.media.MediaPlayer;
import android.media.MediaPlayer.OnPreparedListener;
import android.content.Context;
import android.app.Activity;
import android.widget.VideoView;
import android.widget.MediaController;
import android.net.Uri;
 
MediaPlayer player;
Context context;
Activity act;
MediaController control;
VideoView video;
 
public void setup(){
  act = this.getActivity();
  context = act.getApplicationContext();
  try {
    String vidPath ="android.resource://"+getPackageName(context)+"/data/test.mp4";
    Uri uri = Uri.parse(vidPath);
    control = new MediaController(context);
    player = new MediaPlayer();
    video = new VideoView(context);
    video.setVideoURI(uri);
    video.setMediaController(control);
    video.setOnPreparedListener( new MediaPlayer.OnPreparedListener(){
      public void onPrepared(MediaPlayer player) {
        System.out.println("ready!");
        video.requestFocus();
        video.start();
      }
    }); 
  }
  catch(IOException e) {
    println("file did not load");
  }
};
 
 
public String getPackageName(Context context) {
    return context.getPackageName();
}
 
  
 public void stop(){
        super.stop();
        if (player !=null){
          player.release();
          player = null;
        }
       };

akenaton

@sgrigg:: no error ??? no video?? put some println with a boolean in order to verify that prepared is ok try to verify if your file is found

sgrigg

@akenaton it just doesn't run. the println's never even get a chance to run. I think I am in a pretty strong dead end heh.

akenaton

@sgrigg:: have you any error??? if not that means probably that your file is not found && your player never in prepared state... put the code you are now using...

hudson_m4000

I thought I'd have a go too ..... :)

This code is very basic but is at least doing something.. you can hear the audio on the video but can't see it! I am guessing that Processing is placing it's graphics on top of the video, so I (we?) need a way to overcome this. I'm not sure how yet, but am assuming there may be someway to request focus?

The video must be on sdCard ... as yet I can't find a way to get the absolute path name for a video in the apps internal memory/assets folder

import android.media.MediaPlayer;
import android.os.Environment;
import android.app.Activity;
import android.content.Context;

MediaPlayer mp ;
Context context ;
Activity activity ;
String statusText = "VIDEO INITIALIZING";
String SDCard = new String(Environment.getExternalStorageDirectory().getAbsolutePath());

void setup() {
  activity = this.getActivity();
  context = activity.getApplicationContext();
  mp = new MediaPlayer();
  try {
    //test.mp4 is on SDCard in a folder called VideoTest
    mp.setDataSource(SDCard + "/VideoTest" + "/test.mp4");
    mp.prepare();
    mp.start();
    statusText = "VIDEO PLAYING";
  }
  catch (IOException e) {
    println("error loading test.mp4");
  }
  fill(255); 
  textAlign(CENTER, CENTER);
}

void draw() {
  background(0);
  text (statusText, 0, 0, width, height);
}


void mousePressed() {
  if (mp.isPlaying()) {
    statusText = "VIDEO PAUSED";
    mp.pause();
  } else {
    statusText = "VIDEO PLAYING";
    mp.start();
  }
}

akenaton

@hudson_m4000: with this code you cannot get the video, only the sound from it for video either you use video view or getHolder + mediaPlayer in order to get a display in the 2 cases, with P3 it's rather tricky whereas with 2 it's easy...

hudson_m4000

@akenaton A problem for another day/time I think. Video would be nice someday, but as you say.. not easy.. (for me!) Other things to do..... :)

akenaton

@hudson_m4000 @sgrigg i can do that with Eclipse+P3; tested; i have only now to make the same thing withP3 only and post the snippet.

hudson_m4000

@akenaton that would be very useful. I dont know for sure what the plan is but there is an android video library being developed ... i found info at github. Also android mode is still in beta so maybe we should see what happens over next few weeks or months?

sgrigg

@akenaton @hudson_m4000 sorry I dropped off the discussion. I ended up just taking a different route to my problem (java/eclipse). I was able to find more direct tutorials that way. It is funny to me that writing the code I needed ended up being easier in Java than in Processing. Hopefully P3 starts to work better with android mode in the future!

akenaton

@sgrigg== good idea!!! - and i am sure that in 1 hour you get your video working with videoView (without P3) or 3 days with Eclipse+P3 As for P3 only maybe 15 days?

akenaton

@hudson_m4000=== that s done!!! video in videoView with P3 alone....Hurrah!

hudson_m4000

@akenaton Hurrah indeed.... You have a lot of knowledge and patience!!!

dustyyeti

Hi everyone, great project, I'm trying to do this too! @akenaton, you said you did it in P3, video playback on Android??? I tried the last code posted by @hudson_m4000, and it complied and ran on my device, but did not find the video (which I placed in the SDCARD directory)

What was the trick for you? What did you do differently from hudson's code, and is the file reference correct?

this is from my setup, everything else is identical to hudson's: try { //test.mp4 is on SDCard in a folder called VideoTest mp.setDataSource(SDCard + "/transit.mp4"); mp.prepare(); mp.start(); statusText = "VIDEO PLAYING"; }
The error throws: "error loading transit.mp4"

akenaton

@dustyyeti===

• what is SDcard (i mean, in your code); is it some constant???

• as for me, in this case, i use getExternalStorage()

• is your mp4 in a folder called "VideoTest"??? (in this case of course it cannot be found if SDcard is a kind of constant)

• use try catch and file not found exception

• yet the error seems to be when loading; this could mean that there is some problem with your mp4; some of them do not work with android; try another one.

dustyyeti

@akenaton Hi thanks for your quick response. I left all the code identical to @hudson_m4000 last post; but I should've just posted the whole script here; sorry about that. Let's call this code we're discussing here "ANDROID_MEDIA".

I placed the video file in the sdcard directory. My file explorer on the android device shows the file in that folder, and I can in fact launch it on the stock video player on the device by tapping the file icon in the folder. It plays back without any issue, so I think the file is not at odds with the OS.

The transit video file is one of the standard example assets with the processing libraries. I converted the original transit.mov to .mp4 to match the codec type used in the hudson code, but have tried both mov and mp4. Both play on the device's native video player.

Executing the code on the device renders the default status string "Initializing Video"

Touching the screen changes status text to "VIDEO PLAYING"

Subsequent touches do not change the status text.

No video shows. There's a black stage, 400x400 with the status text overlayed. There's no audio in this file, so I can't comment on the audio playback mentioned in earlier comments.

Whether the file is being located or not, I DO NOT KNOW. I experimented with changing the name to an invalid name, and there's no different reaction; which is to say, I get no sign if the file is found or not found.

I've also experimented with using omerjerk's "processing-video-android" library https://github.com/omerjerk/processing-video-android

Let's call it "OMJ_VIDEO"

OMJ_VIDEO library has worked playing both the .mp4 and the .mov file.

With OMJ_VIDEO, the media files were placed in the sketch folder and packaged in the apk sent to the device; so again, the file sourcing is different in this second script.

ANDROID_MEDIA code

import android.media.MediaPlayer; import android.os.Environment; import android.app.Activity; import android.content.Context; MediaPlayer mp ; Context context ; Activity activity ; String statusText = "VIDEO INITIALIZING"; String SDCard = new String(Environment.getExternalStorageDirectory().getAbsolutePath()); void setup() { size(400,400); activity = this.getActivity(); context = activity.getApplicationContext(); mp = new MediaPlayer(); try { //test.mp4 is on SDCard in a folder called VideoTest mp.setDataSource(SDCard + "/transit.mp4"); mp.prepare(); mp.start(); statusText = "VIDEO PLAYING"; } catch (IOException e) { println("error loading transit.mp4"); } fill(255); textAlign(CENTER, CENTER); }

void draw() { background(0); text (statusText, 0, 0, width, height); }

void mousePressed() { if (mp.isPlaying()) { statusText = "VIDEO PAUSED"; mp.pause(); } else { statusText = "VIDEO PLAYING"; mp.start(); } }

OMJ_VIDEO code works without problem

import in.omerjerk.processing.video.android.*; Movie movie; void setup() { size(1280, 1280, P2D); movie = new Movie(this, "transit.mov"); movie.loop(); movie.play(); } void draw() { image(movie, 0, 0); } void movieEvent(Movie m) { m.read(); }

akenaton

@dustyyeti===

• as you want to use the file on sdcard the second solution is irrelevant, except that it shows that the .mp4 can be read

• i have written & tested the first code (of course with another .mp4) so i am sure that it works

• what does mean // test.mp4 is....(is your file in some folder or at the first level???)

• ASAP ill give a more detailed look to the code and tell you...

dustyyeti

@akenaton Oh, I just left that comment from the copied code. The error catching loop written in there looks like it's made to be sure that the file is located and doesn't through path errors.

And now that you've brought this back to my attention, I notice that the console IS throwing the error "error loading transit.mp4"

So this is probably a path/source issue. GUH. Not sure how my device wants to locate this directory path.

String SDCard = new String(Environment.getExternalStorageDirectory().getAbsolutePath());

I need to figure that out.... the Android directory structure is very odd to me. This device is not rooted, nor does it have external SD card; but as we know there's sdcard directory in the folder structure. I have files in there, how do I route the code to find it? What's the absolute path code string like? "/mnt/sdcard"...
?

dustyyeti

@akenaton instead of using the Environment procedure to find the path to the file location, I have attempted to pass a hard-coded string. I've been looking all over forums for how the android (Lollipop) file directory structure might refer to this.

mp.setDataSource("/storage/emulated/0/transit.mp4");

For example, this doesn't work. But that REALLY seems to be the path to the file, looking in explorer, terminal, everywhere....

akenaton

@dustyyeti===

yes, the path to env storage looks like that; but it is not the same for lollipop, mashmallow and so on... in some cases for example there is a /shell in the path... Anyway i dont think that you can "guess" it... What is your OS on the phone?

Stupid question: how have you put the file on the phone? Till now i thought that it was using fileIO but without any reason. Supposing that it was some video from yours (or downloaded) it could be placed (by android) in another folder (DCIM eg...) and in this case you have to find it with a filtered intent .

As for the storage structure it s not really odd: there are resources for an app ("inside" it, like res or assets), there are folders in the native env storage (your mnt/emulated...) and then others on an added sd card.

As i told you i ll try the code as soon as possible.

akenaton

@dustyyeti===

i got a look to your code; 2 main problems:

• first one could be about permissions (you need read/write external storage)

• Depending of your OS and targeted SDK you have also to get permissions from runtime

• or try targeting lollipop; or manually add permissions to your app

• second one is that you never create a video view; so, if getting permissions you access to your file you get only the audio (your code is for audio) :-h

steve_j

Hi @akenaton, how do you create the videoview in p3. hudson_m4000 code is fine for getting audio but how to we display the video? Thanks

Conrad

Unfortunately, omerjerk's "processing-video-android" library no longer works. It relied on a method that was always in danger of being deprecated in Android-mode, namely getSurfaceView(). Now there seems to be no good means of displaying video in Android sketches anymore.

akenaton

@steve_j, @Conrad

you can show video with android mediaplayer or android videoView (which is a kind of wrapper for mediaPlayer) following these steps (i choose mediaPlayer)::

imports:(some of them are useless)

    import android.media.MediaMetadataRetriever;
    import android.media.MediaPlayer;
    import android.os.Handler;
    import android.os.HandlerThread;
    import android.os.Looper;

    import android.app.Activity;
    import android.view.ViewGroup;
    import android.view.View;
    import android.view.SurfaceHolder;
    import android.view.SurfaceView;

in setup():

• Looper.prepare();

• You find your video with fileDescriptor (like for the sound); your video is in your data folder; choose a standard format: mp4, 3Gp..., not mkv! - This is the first step and here you have to be sure that your video is found using MediaMetadataRetriever metaRetriever and getting height or width: if it returns 0 or -1, there is a problem!

• you create a SurfaceView (mySurface) then you get the Holder from this surface; then you add callbacks to it (SurfaceCreated, surfaceChanged, surfaceDestroyed). In surfaceCreated you set your player.display to the surface: myPlayer.setDisplay(mySurface); the same for surfaceChanged (in case of config changes)

• you create some method to add your video to surface view; it has to be a runnable (this.getactivity().runOnUiThread(new runnable)...In its run() you call another method which prepare the player in a standard way using the afd you have created at the beginning. At this moment your player is ok; you can add your surfaceview to the mainActivity; finally you start the player (if player().isPlaying == false){myPlayer.start();}

• nothing to do in draw

• in void onPause() or onDestroy() dont forget to release the player

try to do that and you can seee your video... tested with processing 3XXX and android mode 3; not tested with am 4 SONYXPERIA Z running kitKat nexus 7 TABLET running mmallow (6.0)

Conrad

Many many thanks @akenaton! I'm stuck trying to create the SurfaceView. In the past, that came from papplet, but that doesn't work anymore. Is there an example you could point me to that might get me moving in the right direction?

akenaton

@Conrad===

no problem for that:

///////////////////

imports as i have already explained

////////////////////////////////////

    SurfaceView surface;
    Activity act;
SurfaceHolder mSurfaceholder;

    /////in setup 
    ...........

    surface = new SurfaceView(act);
    mSurfaceHolder = surface.getHolder();
    mSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    mSurfaceHolder.addCallback(new SurfaceHolder.Callback() {here your 3 callBack}

Conrad

Many thanks @akenaton! I’ll give that a try today. Really appreciate all your help.

Conrad

Thanks again @akenaton! It works now. A couple of things:

• I was building a simple sketch inside processing editor, and not in Android Studio. I hadn't implemented a mainActivity. Should I for my final application?
• the setType method for surfaceHolder is deprecated. Is there a better way of defining holder type?
• I don't have the layoutParameters worked out yet so that they comply with the processing default layout. The Processing application runs centered based onto size() method in setup, but the holder surface view is still the fullscreen. Is there a means of getting the layout parameters so that they stay contained within the size() of the sketch?

Here's my simple video player sketch, based on @akenaton 's excellent description. Thanks again! (Can't get the forum's preprocessor not to convert "@Override" into an HTML ref to a forum user, so I changed it to "@ Override")(Thanks! @GoToLoop)

import android.media.MediaMetadataRetriever;
import android.os.Handler;
import android.os.HandlerThread;
import android.os.Looper;
import android.app.Activity;
import android.view.ViewGroup;
import android.view.View;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.media.MediaMetadataRetriever;
import android.media.MediaPlayer;
import android.content.res.Resources;
import android.content.res.AssetFileDescriptor;
import android.content.res.AssetManager;
import android.content.Context;

AssetFileDescriptor afd;
Context context;
Activity act;
SurfaceView mySurface;
SurfaceHolder mSurfaceHolder;
MediaMetadataRetriever metaRetriever;
MediaPlayer mMediaPlayer;

void setup() {
  size(400, 400, P2D);
  act = this.getActivity();
  context = act.getApplicationContext();
  Looper.prepare();
  mMediaPlayer = new MediaPlayer();
  try {
    afd = context.getAssets().openFd("Title.m4v");
    MediaMetadataRetriever metaRetriever = new MediaMetadataRetriever();
    metaRetriever.setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength());
    String height = metaRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT); 
    if (int(height) < 2) {
      throw new IOException();
    }
  }
  catch (IllegalArgumentException e) {
    e.printStackTrace();
  }
  catch (IllegalStateException e) {
    e.printStackTrace();
  } 
  catch (IOException e) {
    e.printStackTrace();
  }

  mySurface = new SurfaceView(act);
  mSurfaceHolder = mySurface.getHolder();
  mSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  mSurfaceHolder.addCallback(new SurfaceHolder.Callback() {
    @Override
      public void surfaceCreated(SurfaceHolder surfaceHolder) {
      mMediaPlayer.setDisplay(surfaceHolder);
    }

    @Override
      public void surfaceChanged(SurfaceHolder surfaceHolder, int i, int i2, int i3) {
      mMediaPlayer.setDisplay(surfaceHolder);
    }

    @Override
      public void surfaceDestroyed(SurfaceHolder surfaceHolder) {
    }
  }
  );
  startVideo();
}

void startVideo() {
  act.runOnUiThread(new Runnable() {
    public void run() {
      try {
        mMediaPlayer.setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength());
        mSurfaceHolder = mySurface.getHolder();
        mSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
        mMediaPlayer.prepare();
//        act.addContentView(mySurface, new ViewGroup.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT,ViewGroup.LayoutParams.WRAP_CONTENT));
        act.addContentView(mySurface, new ViewGroup.LayoutParams(400,400));
        if (mMediaPlayer.isPlaying() == false) {
          mMediaPlayer.start();
        }
      }
      catch (IllegalArgumentException e) {
        e.printStackTrace();
      }
      catch (IllegalStateException e) {
        e.printStackTrace();
      } 
      catch (IOException e) {
        e.printStackTrace();
      }
    }
  }
  );
};

void draw() {
}

void onPause() {
  if (mMediaPlayer!=null) {
    mMediaPlayer.release();
    mMediaPlayer = null;
  }
  super.onPause() ;
}

void onStop() {
  if (mMediaPlayer!=null) {
    mMediaPlayer.release();
    mMediaPlayer = null;
  }
  super.onStop() ;
}

void onDestroy() {
  if (mMediaPlayer!=null) {
    mMediaPlayer.release();
    mMediaPlayer = null;
  }
  super.onDestroy() ;
}

void onResume() {
  super.onResume() ;
}

akenaton

@Conrad===

ok; happy that it uruns && put answered for others (there are a lot...and i have not so much time!) as for the other questions, yes, it is possible, i ll give you more details after christmas!

GoToLoop

(Can't get the forum's preprocessor not to convert "@Override" into an HTML ref to a forum user, so I changed it to "@ Override")

@ Override is already a good workaround. ;;)

Since you're using <pre></pre> to post your code, you can replace @ w/ @ or @ :ar!

akenaton

@Conrad===

• SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS is deprecated but it does not matter because now the type is set automatically: you can suppress this.

• Yes, the surface view is (default) full screen and located at 0,0; but as i told you you can change it if you want: firstly in your surfaceChanged() you get the video size, width and height writing:

int video_Width = player.getVideoWidth(); int video_Height = player.getVideoHeight();

then, in the same place you write=

LayoutParams layoutParams = surface.getLayoutParams(); layoutParams.height = video_Height; layoutParams.width = video_Width; surface.setLayoutParams(layoutParams);

Now the video will be at the choosen size (of course you can choosen to minimize it...) and at 0,0

• Supposing that you want that video is not at this place but centered you have only to add gravity param to CENTER (or BOTTOM or....)

akenaton

@Conrad===

precision:

• you can do exactly the same using TexturedView or VideoView; it is more simple that using SurfaceView. In this case you cannot use afd, you have to use standard fileIO.

Conrad

Thanks @akenaton. I need to troubleshoot the code I wrote above first. It works on the Android emulator just fine. But on the hardware I have (an Android 5.1 Watch) it’s just a white screen and no errors reporting in adb.

Conrad

@akenaton Following your advice, the surfaceChanged callback looks like this:

    @Override
      public void surfaceChanged(SurfaceHolder surfaceHolder, int i, int i2, int i3) {
      mMediaPlayer.setDisplay(surfaceHolder);
      int video_Width = mMediaPlayer.getVideoWidth();
      int video_Height = mMediaPlayer.getVideoHeight();
      android.view.ViewGroup.LayoutParams layoutParams = mySurface.getLayoutParams();
      layoutParams.height = video_Height;
      layoutParams.width = video_Width;
      mySurface.setLayoutParams(layoutParams);
      mySurface.setZOrderOnTop(true);
    }

But I can not figure out how to add gravity to layoutParams. I have tried creating a RelativeLayout.LayoutParams but I can only get this to work from my activity.runOnUiThread. It doesn't work from the surfaceChanged callback for some reason. BTW, I've found that I need the setZOrderOnTop call on actual hardware, but not on the emulator. Not sure why that is.

akenaton

@Conrad===

that is easy

import Gravity

then (surfaceChanged):

surface.setLayoutParams(new FrameLayout.LayoutParams( video_Width, video_Height, Gravity.CENTER));

As for the other problem i cannot see why you have to setZ = i have tested my code without it and with a real device...

Conrad

Ignore this. I figured out what I was doing wrong. I've since fixed it. I'll post the solution once I've cleaned up the code.

@akenaton that fixed it. Many thanks. On most devices it works without setting the Z. On the Android Watch for some reason I need it. Probably has to do with some system-level changes that were made to accommodate the lack of the three system-wide navigation buttons.

On a different issue that I see on all devices, I now need to clean up after the video finishes playing. I'm just trying to show a video, then get on with my sketch. But Processing never seems to call the completion handler. I added the following call right after mMediaPlayer.start(); in run() but I never get the log message and the Surface just sits on top of everything.

Aurelian24

Have you ever cleaned up the code? Wich is the easyest way to reproduce video in android using processing ?

That´s seems to be a lot of code.

do you put the video file in the data folder?

akenaton

@Aurelian24===

have you tried this code? Have you read the posts before? easier way could be to use ketai, if it works...

flashbacker

Hello, I have been reading this thread with a lot of attention, but i am still having problem with the location of the file, either on my phone or tablet (respectively Android 6 and 5) The environment procedure to find the path always return "/storage/emulated/0" wether i have an external storage or not. I tried to enter some manual values that i could find with some apps as mentionned by @Conrad but it didn't work as well. The code seems to work fine other than that, the app opens on my devices, but empty... Only message is that it couldnt locate the file... @akenaton would you have any suggestion?

Thanks

hudson_m4000

Try this ...

    File[] SDcards ; 
    String primarySD, externalSD;


    void setup() {
      //ANDROID ...  get files paths for SD cards(s) and convert to Strings
      if (Build.VERSION.SDK_INT >= 19) { 
        //from API19 (4.4 KitKat) get array of filepaths for all connected SD cards 
        //(and maybe other connected media?)
        //1st path returned is the emulated primary SD, 2nd path is real external SD, etc.
        SDcards = context.getExternalFilesDirs(null);
        primarySD = SDcards[0].getPath().split("/Android")[0];
        externalSD = SDcards[1].getPath().split("/Android")[0];
      } else if (Build.VERSION.SDK_INT >= 8) { // very old phones !
        //from API8 you can only get the file path of the primary SD card
        //(could be emulated or real dependent on device)
        SDcard = context.getExternalFilesDir(null);
        primarySD = SDcards[0].getPath().split("/Android")[0];
      }
    }

I use this and it took me ages to find it all out!! Hope its of some use and works for you too :)

Chrisir

There is a new forum now by the way