Real time video export from Processing

hamoid · January 2015

This morning I started thinking about how to help someone who had trouble exporting video from Processing. One hour later I had this proof of concept working:

https://github.com/hamoid/Fun-Programming/tree/master/processing/ideas/2015/01/streamToFFMPEG

It allows you to build a video by adding frames one by one. No need to save all frames first and then merge them into a video. It uses ffmpeg and pipes. In this demo, you can hold the mouse down to record. You don't need to record everything as one long segment: you can observe and when you see something interesting happening, hold down record (ok, I have to admit you won't see much exciting happening in this demo :-).

It will have to be adapted to Windows and OS X, and it should be configurable (bitrate, video format, etc).

But maybe it can be useful to someone already.

tlecoz · January 2015

Thank you !

monkstone · January 2015

Works for me (also on Archlinux) on a system with mplayer installed. Which I'm guessing installed the right libraries.

hamoid · January 2015

Great! Thanks for testing it @monkstone.

It would be great to have this also working on Windows and Mac.

If someone knows how to...

download ffmpeg
make ffmpeg accessible from the Processing program by doing ONE of these:

a. putting the ffmpeg executable in the sketch folder

b. adjusting the PATH variable so ffmpeg is found

c. or setting the absolute route to ffmpeg in the Processing program

...please give it a try.

Any suggestions about turning this into a tool or a plugin? How could this be made easier to use?

At least it should I should describe how to make it work in different systems.

hamoid · January 2015

I think the easiest way to make it work on Mac is to

Install homebrew (a small package manager)
Type "brew install ffmpeg" in the terminal (without quotes).
Run the program I linked above to see how to render a sketch to mp4.

Homebrew gives you access to thousands of useful command line programs on mac.

clankill3r · January 2015

Sounds cool!

clankill3r · January 2015

You accept a int as frameRate, could this be a float? I ask this cause 29.97 for example is a common used frame rate. (processing supports float frameRate as well).

In the constructor of ProcessBuilder you have:

sketchPath + "/" + filename);

sketchPath should be path.

Also in saveFrame you have:

pg.loadPixels();

It would be nice to have a way to prevent this, this is for when I already loaded the pixels myself. For example, I use a webcam and I scan for blobs, therefor I used loadPixels, no blobs where found but I still want to save the webcam image to the video. There is no need to loadPixels again.

Here you can see that loading adds up:

  PGraphics pg = createGraphics(600, 600);

  int start = millis();
  for (int i = 0; i < 10000; i++) {
    pg.loadPixels();
  }
  println(millis()-start);

hamoid · January 2015

Very useful tips, thanks.

Fps is now float, sketchPath is now path, loadPixels() can be switched off.

clankill3r · January 2015

cool! You should make a library out of this :)

hamoid · January 2015

Yes, it would be great as a library. Though I've never seen a library with a dependency on an external binary. I wonder what would people think about that.

Did you try it? If yes, did it work? On what platform?

clankill3r · January 2015

I have seen libraries with dependencies. It should not hold you back. And I will try it tonight :)

clankill3r · January 2015

Yeah I got it to work (MAC OSX Yosemithe or hwatever it's called). I have set the path to "/usr/local/bin/ffmpeg". I think that will work most of the times but it's worth looking into if you make a lib.

It works really great!

hamoid · January 2015

Great to hear that it works :)

As a test I created this video without ever stopping my sketch. It did not take much time, only a little more than the duration of the full video:

hamoid · January 2015

Ok, this is my first attempt at creating a library: https://github.com/hamoid/video_export_processing

Here's a download (the green button).

The ffmpeg detection could be nicer... I accept pull requests :P

clankill3r · January 2015

Can you include the resources folder on github.

hamoid · January 2015

The word "resources" is only mentioned briefly twice in these pages:

What should that folder contain?

hamoid · January 2015

Ah I found this now:

https://github.com/processing/processing-library-template

I just had not used any template. I wrote everything from scratch. Ok, time to read.

clankill3r · January 2015

haha :) Yeah the template is quite useful.

hamoid · January 2015

I think now it looks like a proper library :)

redraw · February 2015

great idea hamoid!

i've tested it on windows, and it throws

Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 259200 kb/s
    Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 600x600, 259200 kb/s, 30 tbr, 30 tbn, 30 tbc
[NULL @ 0000000002d6f0e0] Unable to find a suitable output format for 'using'
using: Invalid argument

It has to be with ffmpegMetadataComment string. I took a look at ffmpeg documentation and I didn't find "-comment" option, so maybe it's deprecated. This stays in your code at line 172 but I might be missing something.

hamoid · February 2015

I would say it has to do with quotes while executing a command line program on Windows. It's not complaining about "comment", but about "using", which should not be read by itself, but as part of a string.

I'll find out how to fix that. Thanks!

Stanlepunk · February 2015

Hello hamoid, i want try you video-export-library, but i'm not nurd-geek-mac user and install "ffmpeg" it's a nightmare. There is easier way to use your library or install ffmpeg on mac ? like just drag and drop :) ????? thanks and congratulation for the library.

hamoid · February 2015

Thanks. Is it not possible to download a binary from http://evermeet.cx/ffmpeg/ and uncompress the 7z file somewhere on your hard drive? When I meet a Mac user I'll write instructions about how to install ffmpeg. I wish that was automatic!

hamoid · February 2015

BTW. The issue above on Windows should be solved with the current version.

clankill3r · February 2015

@Stanlepunk

Read the 5th post in this topic.

Stanlepunk · March 2015

ok thk for the reply, sorry for my late answer but I don't receive the notification.

clankill3r · April 2015

O yeah hamoid, one more thing. You have all values private at the moment. Personally I never use private and I encourage other people to do neither unless there is a very good reason (like security).

To explain further, let's say I want to build on top of your class

class SomeFancyVideoOutput extends VideoOutput {
    // damnit, you made all private!
}

hamoid · June 2015

Why not having an instance of VideoOutput instead of extending it? (favor composition over inheritance)? Doesn't having private variables and methods lead to decoupling, which is a good thing?

GoToLoop · June 2015

Due to the tweakish nature of our community, IMO 3rd-party libraries should avoid private.
At most use protected. This way we can consciously "hack" a library by inheriting it.
That is, we access protected members of some class by will rather than by "accident".
private & "package default" access levels are more like:
"We prohibit in order to protect yourself from yourself" way of thinking. :(|)
Of course composition is generally more preferred than inheritance! ;)

hamoid · June 2015

Ok, I'll make them protected with the next release. Would that be enough, @clankill3r ?

GoToLoop · June 2015

Marvelous @hamoid! I believe a simple replace all "private " w/ "protected " would suffice! *-:)

clankill3r · June 2015

Cool :) Yeah one more thing. The video's won't play in quicktime or vlc. Only in quick preview. (mac osx) This is kind of a big deal.

hamoid · June 2015

I wonder why. I will have to do some testing. The videos produced in my computer work fine in VLC. Here another user having issues, this time on Windows: http://forum.processing.org/two/discussion/comment/44048

Does this file play ok? http://filebin.ca/23rkQE33zQqH/basic.mp4 I produced that one on my computer (Arch Linux) with ffmpeg 2.6.3.

hamoid · June 2015

I released version 0.0.6 that I hope fixes the playback issues in Mac and Windows.

clankill3r · June 2015

yeah that video works fine.

Ria · June 2015

You right, I tried again!. It works with the VLC player, but not the media player

hamoid · June 2015

Which one works with VLC? The one I uploaded or your own? or both?

GoToLoop · June 2015

VLC already comes w/ built-in codecs.
Windows Media Player and most other players demand we got the requested codec for the video we're trying to play.
A good codec collection for Windows is K-Lite Standard Codec Pack:
http://www.codecguide.com/download_kl.htm

hamoid · June 2015

@GoToLoop that's true, but I was missing "-pix_fmt yuv420p", which can cause trouble in most video players. I added that in 0.0.6. I hope it improves compatibility.

Ria · June 2015

hamoid the one you uploaded for me worked with VLC.

Repet · July 2015

I don't use libraries, but this is cool. I was just thinking, if you changed the frame rate, could you make slow motion video? (Of your program)

hamoid · August 2015

Reducing the frame rate would not look smooth, except if you use some kind of tricky interpolation like http://slowmovideo.granjow.net/

Repet · August 2015

like, run at 90fps and play at 20fps

hamoid · September 2015

I doubt you could run a sketch a 90 fps while saving the video file. But to create a slow motion effect, you just need to reduce the motion increments. Instead of moving 5 pixels per frame, move 1 pixel instead. Or 0.1. You can always control the speed of your animation, even interactively (with a knob, or your mouse).

clankill3r · October 2016

Would it possible to have sound as well?

hamoid · October 2016

@clankill3r Possibility to add sound is planned: https://github.com/hamoid/video_export_processing/issues/11 but it won't happen earlier than next week.

Howdy, Stranger!

Categories

In this Discussion

Real time video export from Processing

Comments