Real time video export from Processing

edited January 2015 in Share Your Work

This morning I started thinking about how to help someone who had trouble exporting video from Processing. One hour later I had this proof of concept working:

https://github.com/hamoid/Fun-Programming/tree/master/processing/ideas/2015/01/streamToFFMPEG

It allows you to build a video by adding frames one by one. No need to save all frames first and then merge them into a video. It uses ffmpeg and pipes. In this demo, you can hold the mouse down to record. You don't need to record everything as one long segment: you can observe and when you see something interesting happening, hold down record (ok, I have to admit you won't see much exciting happening in this demo :-).

It will have to be adapted to Windows and OS X, and it should be configurable (bitrate, video format, etc).

But maybe it can be useful to someone already.

Tagged:

Comments

  • Thank you !

  • Works for me (also on Archlinux) on a system with mplayer installed. Which I'm guessing installed the right libraries.

  • Great! Thanks for testing it @monkstone.

    It would be great to have this also working on Windows and Mac.

    If someone knows how to...

    1. download ffmpeg
    2. make ffmpeg accessible from the Processing program by doing ONE of these:

      a. putting the ffmpeg executable in the sketch folder

      b. adjusting the PATH variable so ffmpeg is found

      c. or setting the absolute route to ffmpeg in the Processing program

    ...please give it a try.

    Any suggestions about turning this into a tool or a plugin? How could this be made easier to use?

    At least it should I should describe how to make it work in different systems.

  • I think the easiest way to make it work on Mac is to

    1. Install homebrew (a small package manager)

    2. Type "brew install ffmpeg" in the terminal (without quotes).

    3. Run the program I linked above to see how to render a sketch to mp4.

    Homebrew gives you access to thousands of useful command line programs on mac.

  • Sounds cool!

  • You accept a int as frameRate, could this be a float? I ask this cause 29.97 for example is a common used frame rate. (processing supports float frameRate as well).

    In the constructor of ProcessBuilder you have:

    sketchPath + "/" + filename);

    sketchPath should be path.

    Also in saveFrame you have:

    pg.loadPixels();

    It would be nice to have a way to prevent this, this is for when I already loaded the pixels myself. For example, I use a webcam and I scan for blobs, therefor I used loadPixels, no blobs where found but I still want to save the webcam image to the video. There is no need to loadPixels again.

    Here you can see that loading adds up:

      PGraphics pg = createGraphics(600, 600);
    
      int start = millis();
      for (int i = 0; i < 10000; i++) {
        pg.loadPixels();
      }
      println(millis()-start);
    
  • Very useful tips, thanks.

    Fps is now float, sketchPath is now path, loadPixels() can be switched off.

  • edited January 2015

    cool! You should make a library out of this :)

  • Yes, it would be great as a library. Though I've never seen a library with a dependency on an external binary. I wonder what would people think about that.

    Did you try it? If yes, did it work? On what platform?

  • I have seen libraries with dependencies. It should not hold you back. And I will try it tonight :)

  • Yeah I got it to work (MAC OSX Yosemithe or hwatever it's called). I have set the path to "/usr/local/bin/ffmpeg". I think that will work most of the times but it's worth looking into if you make a lib.

    It works really great!

  • Great to hear that it works :)

    As a test I created this video without ever stopping my sketch. It did not take much time, only a little more than the duration of the full video:

  • Ok, this is my first attempt at creating a library: https://github.com/hamoid/video_export_processing

    Here's a download (the green button).

    The ffmpeg detection could be nicer... I accept pull requests :P

  • Can you include the resources folder on github.

  • edited January 2015

    Ah I found this now:

    I just had not used any template. I wrote everything from scratch. Ok, time to read.

  • haha :) Yeah the template is quite useful.

  • I think now it looks like a proper library :)

  • edited February 2015

    great idea hamoid!

    i've tested it on windows, and it throws

    Input #0, rawvideo, from 'pipe:':
      Duration: N/A, start: 0.000000, bitrate: 259200 kb/s
        Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 600x600, 259200 kb/s, 30 tbr, 30 tbn, 30 tbc
    [NULL @ 0000000002d6f0e0] Unable to find a suitable output format for 'using'
    using: Invalid argument
    

    It has to be with ffmpegMetadataComment string. I took a look at ffmpeg documentation and I didn't find "-comment" option, so maybe it's deprecated. This stays in your code at line 172 but I might be missing something.

  • I would say it has to do with quotes while executing a command line program on Windows. It's not complaining about "comment", but about "using", which should not be read by itself, but as part of a string.

    I'll find out how to fix that. Thanks!

  • Hello hamoid, i want try you video-export-library, but i'm not nurd-geek-mac user and install "ffmpeg" it's a nightmare. There is easier way to use your library or install ffmpeg on mac ? like just drag and drop :) ????? thanks and congratulation for the library.

  • Thanks. Is it not possible to download a binary from http://evermeet.cx/ffmpeg/ and uncompress the 7z file somewhere on your hard drive? When I meet a Mac user I'll write instructions about how to install ffmpeg. I wish that was automatic!

  • BTW. The issue above on Windows should be solved with the current version.

  • @Stanlepunk

    Read the 5th post in this topic.

  • ok thk for the reply, sorry for my late answer but I don't receive the notification.

  • O yeah hamoid, one more thing. You have all values private at the moment. Personally I never use private and I encourage other people to do neither unless there is a very good reason (like security).

    To explain further, let's say I want to build on top of your class

    class SomeFancyVideoOutput extends VideoOutput {
        // damnit, you made all private!
    } 
    
  • Why not having an instance of VideoOutput instead of extending it? (favor composition over inheritance)? Doesn't having private variables and methods lead to decoupling, which is a good thing?

  • edited June 2015
    • Due to the tweakish nature of our community, IMO 3rd-party libraries should avoid private.
    • At most use protected. This way we can consciously "hack" a library by inheriting it.
    • That is, we access protected members of some class by will rather than by "accident".
    • private & "package default" access levels are more like:
      "We prohibit in order to protect yourself from yourself" way of thinking. :(|)
    • Of course composition is generally more preferred than inheritance! ;)
  • Ok, I'll make them protected with the next release. Would that be enough, @clankill3r ?

  • Marvelous @hamoid! I believe a simple replace all "private " w/ "protected " would suffice! *-:)

  • Cool :) Yeah one more thing. The video's won't play in quicktime or vlc. Only in quick preview. (mac osx) This is kind of a big deal.

  • I wonder why. I will have to do some testing. The videos produced in my computer work fine in VLC. Here another user having issues, this time on Windows: http://forum.processing.org/two/discussion/comment/44048

    Does this file play ok? http://filebin.ca/23rkQE33zQqH/basic.mp4 I produced that one on my computer (Arch Linux) with ffmpeg 2.6.3.

  • I released version 0.0.6 that I hope fixes the playback issues in Mac and Windows.

  • yeah that video works fine.

  • You right, I tried again!. It works with the VLC player, but not the media player

  • Which one works with VLC? The one I uploaded or your own? or both?

  • edited June 2015
    • VLC already comes w/ built-in codecs.
    • Windows Media Player and most other players demand we got the requested codec for the video we're trying to play.
    • A good codec collection for Windows is K-Lite Standard Codec Pack:
      http://www.codecguide.com/download_kl.htm
  • @GoToLoop that's true, but I was missing "-pix_fmt yuv420p", which can cause trouble in most video players. I added that in 0.0.6. I hope it improves compatibility.

  • hamoid the one you uploaded for me worked with VLC.

  • I don't use libraries, but this is cool. I was just thinking, if you changed the frame rate, could you make slow motion video? (Of your program)

  • Reducing the frame rate would not look smooth, except if you use some kind of tricky interpolation like http://slowmovideo.granjow.net/

  • like, run at 90fps and play at 20fps

  • I doubt you could run a sketch a 90 fps while saving the video file. But to create a slow motion effect, you just need to reduce the motion increments. Instead of moving 5 pixels per frame, move 1 pixel instead. Or 0.1. You can always control the speed of your animation, even interactively (with a knob, or your mouse).

  • Would it possible to have sound as well?

  • @clankill3r Possibility to add sound is planned: https://github.com/hamoid/video_export_processing/issues/11 but it won't happen earlier than next week.

Sign In or Register to comment.