Frame Accurate Interactive Video Playback - a solution for 1.5.x but not working in 3.x

chrios1001

After much hacking I have made a sketch that does frame accurate interactive video playback. I based this on tips and code from the older processing forums. BUT - it only works in processing 1.5.1 I had to use various workarounds, and I don't know why they work. It is something to do with the conversion from time in seconds to an accurate frame number. A limitation of the jump() command.

I'm posting this code in the hope that it might help others and maybe to find a solution that works in Processing 3.x

here's the code with lots of comments and link to a reliable test video:

/*
 An example of frame accurate interactive video playback
 Tested working for frame accurate interactive viewing of videos at 25fps and 30fps - any length.
 This only works in processing 1.5.1
 I cannot get it working in newer versions of Processing
 This is probably due to the switch to the gstreamer library

 Easiest way to check it's working is to use a movie file with the
 correct frame number printed on each frame so you can spot skipping frames
 as you use the keys: -/+ to navigate.

 The suitable video for this sketch (with it's numbered frames) can be d/ld here:
 http://33dd.com/processing/100_Numbered_frames_30fps.mov
 */

import processing.video.*; 
Movie myMovie; 
float movieFps=30.00;//fps of the movie - please set to match your movie file
float advanceLength = 1/(movieFps-0.01);//get length of one frame in seconds - and subtract 0.01 (why -0.01 I don't know??)
//this movie is 30fps - but sketch is only accurate if we declare fps as 29.99fps - why??

void setup()
{ 
  size(640, 320, P2D);//P2D needed for processing 1.5.--
  //size(640, 320);//use this method in processing 2.++ - no more P2D
  background(0); 

  //load a movie with visible framenumbers - makes it easier to check accuracy
  myMovie = new Movie(this, "100_Numbered_frames_30fps.mov"); //see link at top for video source
  myMovie.loop();//play the movie
  myMovie.speed(0);//best way to stop the movie - pause() halts the thread - so we must use speed(0)
} 

void movieEvent(Movie myMovie)
{ 
  myMovie.read();
} 

void draw()
{ 
  image(myMovie, 0, 0, width, height);
} 

void keyPressed()
{ //use - / +  keys to go back and forward one frame
  if (key == '-')
  {
    if (myMovie.time() < advanceLength)//if movie time < 1 frame jump to start of movie (another hack needed to make it work)
    {
      myMovie.jump((myMovie.time() - advanceLength) % 0);
    } else  
    {//for some reason this only works accurately if we jump backwards then forward one frame
      myMovie.jump((myMovie.time() - advanceLength) % myMovie.duration()); 
      myMovie.jump((myMovie.time() + advanceLength) % myMovie.duration());
    }
  }  
  if (key == '=')
  {
    //jump forward by one frame 
    myMovie.jump((myMovie.time() + advanceLength) % myMovie.duration());
  }
}

chrios1001

Here's my attempt in processing 3.x - this adds the changes to be compatible with the gstreamer video library - since you cannot set speed(0)

/*
 Attempt at frame accurate interactive video playback
 not working in Processing 3.x
 */

import processing.video.*; 
Movie myMovie; 
float movieFps=30.00;//fps of the movie - please set to match your movie file
float advanceLength = 1/(movieFps-0.01);//get length of one frame in seconds - and subtract 0.01 (why -0.01 I don't know??)
//this movie is 30fps - but sketch is only accurate if we declare fps as 29.99fps - why??
void setup()
{ 
  size(640, 320, P2D);//P2D needed for processing 1.5.--
  //size(640, 320);//use this method in processing 2.++ - no more P2D;
  background(0); 

  //load a movie with visible framenumbers - makes it easier to check accuracy
  myMovie = new Movie(this, "100_Numbered_frames_30fps.mov"); //see link at top for video source
  myMovie.loop();//play the movie
  //myMovie.speed(0);//does not work in Processing 3.x
  myMovie.pause();//pause the movie (instead of speed(0);
} 

void movieEvent(Movie myMovie)
{   
  myMovie.read();
} 

void draw()
{ 
  image(myMovie, 0, 0, width, height);
} 

void keyPressed()
{ 
  if (key == '-')//use - / +  keys to go back and forward one frame
  {
    myMovie.play();//start the movie
    if (myMovie.time() < advanceLength)//if movie time < 1 frame jump to start of movie (another hack needed to make it work)
    {
      myMovie.jump((myMovie.time() - advanceLength) % 0);
    } else  
    {//for some reason this only works accurately if we jump backwards then forward one frame
      myMovie.jump((myMovie.time() - advanceLength) % myMovie.duration()); 
      myMovie.jump((myMovie.time() + advanceLength) % myMovie.duration());
    }
    myMovie.pause();//stop the movie
  }

  if (key == '=') 
  {   
    myMovie.play();//start the movie
    myMovie.jump((myMovie.time() + advanceLength) % myMovie.duration()); //jump forward by one frame 
    myMovie.pause();//stop the movie
  }
}