Multiple videos smoothness - threaded playback

grumo · June 2018

Hello! This script only requires a bunch of .mp4s in the data folder. You'll see a collage of 9 videos (or duplicates) in different positions and transparencies. The thing runs ok but even pre-loading the videos at the beginning every time there's a change/swap in one of the videos (around every 4 seconds) the whole thing freezes for a sec, sometimes (try to use mp4s of different sizes).

I tried to use the thread() command in the switching of the videos but nothing happened. I put a println to see the thread but alway show the main thread, I really don't know if I'm doing this ok..

Thanks a LOT for any help!

import processing.video.*;

File path; 
String[] files;
int numfiles = 0;
int timer = -5000;
int tcontrol= 2000;
int numVideos=9;

VideoM[] v = new VideoM[numVideos];
Movie[] vv;

void setup(){
  size(1920, 1080, P2D);
  frameRate(60);
  files = files(); 
  numfiles = files.length;
  if (numfiles>11) numfiles=11;
  loadvideos();

  for(int i = 0;i < numVideos;i++){
     v[i] = new VideoM();
  }
}


void draw() {

  background(0);
  for(int i = 0; i <numVideos; i++){
     v[i].display();
  }

  if ((millis() - timer) > tcontrol)  {
    thread("newvideo");   
    timer = millis();
    tcontrol= int(random(2,6))*1000; 
}
}


void loadvideos(){

 String video;
 vv = new Movie[numfiles];

 for (int i=0; i<numfiles; i++){
   video= files[int(random(numfiles))];
   vv[i] = new Movie(this, video);
   println ("Loading ", video);
 }
}

class VideoM {

 Movie m;
 String video;

 int x;
 int y;
 int w;
 int h;
 int alpha;
 int alphai;
 int fadeout=0;

 VideoM() {
  genera();
  println(numfiles);
  m = vv[int(random(numfiles)) ];
  m.loop();
  // m.volume(random(0.4,0.6));
  // m.speed(random(0.6,1.0));
  m.jump(random(m.duration()));  
  //m.play(); 
 }

 void genera(){
    x=int(random(50, width/2+width/4)); 
    y=int(random(50, height/2+height/4)); 
    w=int(random(280,820));
    h=int(w/1.88);
    alpha=int(random(100,255)); 
    alphai=0; 
 }

 void display(){
   tint(255, alphai);
   if (fadeout==0) {
     alphai++; if (alphai>alpha) alphai=alpha;
   } else {  alphai--; if (alphai<0) {alphai=0;fadeout=0;this.newvid();}
     }

   if (frameCount > 1) { image(m, x, y, w, h); }
 } 

 void cambiavid(){
   fadeout=1;
 }

void newvid() {  
   m=null;
   int j=int(random(numfiles));
   println("cambio: ", j);
   m = vv[j];
   m.loop();
   genera();
   m.jump(random(m.duration())); 
   println(Thread.currentThread());
 }
}  

void newset(){
  for(int i = 0;i < numVideos;i++){
    println(i);
    v[i].newvid();
  }
}

void newvideo(){
  int i = int(random(numVideos));
  //v[i].nuevovid(this);
  v[i].cambiavid();
}

void movieEvent(Movie m) {
  m.read();
}

boolean bStop,bColor=true;
boolean bSave=false,bVidO=false;
void keyPressed()
{
  int k = keyCode;

  // reset all videos
  if (key == 'n' || key == 'N') {
    newset();
    }

}  


// load files in data

String[] files(){
// The data path of the folder to look in (write your own)
java.io.File folder = new java.io.File(dataPath(""));

// let's set a filter (which returns true if file's extension is .jpg)
java.io.FilenameFilter pngFilter = new java.io.FilenameFilter() {
  public boolean accept(File dir, String name) {
    return name.toLowerCase().endsWith(".mp4");
  }
};

// list all the folders inside the main directory
String[] listFolders = folder.list(new java.io.FilenameFilter() {
  public boolean accept(File current, String name) {
    return new File(current, name).isDirectory();
  }
});

// list the files in the data folder, passing the filter as parameter
String[] filenames = folder.list(pngFilter);

return(filenames);
}

Chrisir · June 2018

There is a new forum

Please ask there too

kfrajer · June 2018

Not sure if doing this through a different thread will help. If you are loading the videos, I would say that assigning the videos, the way you do, should work, assuming your computer can handle loading these many videos.

I have to say I am not sure your approach works at the end. You are assigning VideoM objects from your video list but I don't think you are ensuring the same video is not picked. Why is this important? Well, because you are calling jump(). If two VideoM objects are handling the same video, you are calling jump on the same handle. it is very likely only the last jump is applied and both handles would display the same video.

I have to say your code comments are missing? It is hard to understand what you are trying to accomplish in certain parts of your code. Adding comments would help. Also, the name of your arrays are not the best. For this bit of code, the array functionality is lost between the lines. Using better names like masterVideolist would make your code easier to follow.

I have two questions:
1. When the timer ends, do you change one video or all the videos?
2. The display function manages some tinting. What does it do? I can see the alphai counter going up and down but I didn't understand what you want to do.

I have added some code below but it is not a solution. I just want to capture some changes I did and I will change it as soon as you answer my questions.

Kf

import processing.video.*;

final int NSHOWVIDEOS=5;

File path; 
String[] files;
int numfiles = 0;
int timer = -5000;
int tcontrol= 2000;

boolean bStop, bColor=true;
boolean bSave=false, bVidO=false;


VideoM[] v = new VideoM[NSHOWVIDEOS];
Movie[] videoStack;

void setup() {
  size(1920, 1080);
  frameRate(60);
  files = files(); 
  numfiles = files.length;
  if (numfiles>11) numfiles=11;

  loadvideos();  //Inits videoStack

  for (int i = 0; i < NSHOWVIDEOS; i++) {
    v[i] = new VideoM(i);
  }
}


void draw() {

  background(0);
  for (int i = 0; i <NSHOWVIDEOS; i++) {
    v[i].display();
  }

  if ((millis() - timer) > tcontrol) {
    thread("newvideo");   
    timer = millis();
    tcontrol= int(random(2, 6))*1000;
  }
}

void movieEvent(Movie m) {
  m.read();
}


void keyPressed()
{
  int k = keyCode;

  // reset all videos
  if (key == 'n' || key == 'N') {
    newset();
  }
}  




//=====================================================
void newset() {
  for (int i = 0; i < NSHOWVIDEOS; i++) {
    println(i);
    v[i].newvid();
  }
}

//=====================================================
void newvideo() {
  int i = int(random(NSHOWVIDEOS));
  //v[i].nuevovid(this);
  v[i].cambiavid();
}

//=====================================================
void loadvideos() {

  videoStack = new Movie[numfiles];

  for (int i=0; i<numfiles; i++) {
    String video= files[i]; //files[int(random(numfiles))];
    videoStack[i] = new Movie(this, video);
    videoStack[i].play();
    videoStack[i].pause();
    println ("Loading ", video);
  }
}


//=====================================================
// load files in data
String[] files() {
  // The data path of the folder to look in (write your own)
  java.io.File folder = new java.io.File(dataPath(""));

  // let's set a filter (which returns true if file's extension is .jpg)
  java.io.FilenameFilter pngFilter = new java.io.FilenameFilter() {
    public boolean accept(File dir, String name) {
      return name.toLowerCase().endsWith(".mp4");
    }
  };

  // list all the folders inside the main directory
  String[] listFolders = folder.list(new java.io.FilenameFilter() {
    public boolean accept(File current, String name) {
      return new File(current, name).isDirectory();
    }
  }
  );

  // list the files in the data folder, passing the filter as parameter
  String[] filenames = folder.list(pngFilter);

  return(filenames);
}


//=====================================================

class VideoM {

  Movie m;
  String video;

  int x, y, w, h;
  int alpha;
  int alphai;
  int fadeout=0;
  int idx;

  VideoM(int aidx) {

    idx=aidx;
    newvid();

  }

  void newvid() {  
    m=null;
    int j=idx;//int(random(numfiles));
    println("cambio: ", j);
    m = videoStack[j];
    m.loop();
    genera();
    m.jump(random(m.duration())); 
    println(Thread.currentThread());
  }

  void genera() {
    x=int(random(50, width/2+width/4)); 
    y=int(random(50, height/2+height/4)); 
    w=int(random(280, 820));
    h=int(w/1.88);
    alpha=int(random(100, 255)); 
    alphai=0;
  }

  void display() {
    tint(255, alphai);
    if (fadeout==0) {
      alphai++; 
      if (alphai>alpha) alphai=alpha;
    } else {  
      alphai--; 
      if (alphai<0) {
        alphai=0;
        fadeout=0;
        this.newvid();
      }
    }

    if (frameCount > 1) { 
      //image(m, x, y, w, h);
      image(m, x, y);
    }
  } 

  void cambiavid() {
    fadeout=1;
  }
}

grumo · June 2018

Thanks a lot @kfrajer

This changes one video, calling the function newvideo (in here I used the thread function but I don't think is doing anything)
The alphai is the alpha value of the object, so it goes up or down to create a fadein/fadeput effect every time a new video is called.

In general the script works well, is just that the moment the newvideo() function is called sometimes there's a freezing in the other videos --or even stop. It should be some memory issue but I limited the preloaded videos and the videos shown and the script works better. The thing is that the script running won't take more than 300 Mb in memory (an there's much more available), it should work better, isn't?

kfrajer · June 2018

In general the script works well, is just that the moment the newvideo() function is called sometimes there's a freezing in the other videos

That is good to know bc it wasn't running on my machine so it made me thing accessing the same reference was the issue. Can you make a quick test and only load one video and displaying 9 of them? Does it work for you?

One suggestion. When you load the videos, you could do something like, as soon as each gets created, to call their play() followed by a pause() called to get the videos ready to stream. When you change the video and before you drop the video handle, you could leave it ready to play when the video gets picked again in the next reading operation. This is the concept:

Video[] videoStack; ///All your available videos

//Adding videos in setup
videoStack[i] = new Movie(this, video);
videoStack[i].play();
videoStack[i].pause();

//Assigning a new video to the VideoM object
void newvid() { 
    m.pause(); //New
    m=null;
    m=....

This is an idea and not totally sure if it will work. I'll see if I can do some testing later today.

Kf

jeremydouglass · June 2018

@grumo -- if this is an ongoing project I recommend continuing to post for advice about it on the new forum:

https://discourse.processing.org/

grumo · June 2018

Thanks @jeremydouglass ! I tried but my account is in hold (user: prandam) since a week ago or more.. dunno what happened.

jeremydouglass · June 2018

@grumo -- I've reactivated your account on the new forum.

Howdy, Stranger!

Categories

In this Discussion

Multiple videos smoothness - threaded playback

Answers