Video Export library

Lord_of_the_Galaxy

@GoToLoop Nah, don't bother about that.I used both of the things, and there was no difference.
I use System.exit(0) because exit()calls<imported library based class>.dispose() I wanted that not to happen, to see if that was the problem, but I failed. I just didn't bother to change my code back.

Lord_of_the_Galaxy

@hamoid I hope you can find a solution to the problem.
Thanks for trying to make the cool library work on Windows 8.1 too.

hamoid

@Lord_of_the_Galaxy I just tried on a Windows 7 machine with Processing 3.1.1, latest version of VLC and ffmpeg.exe from a few months ago. The library worked fine as long as I stopped the sketch pressing the stop button on the Processing IDE. Pressing ESC did nothing (quite strange) and closing the window clicking the X produced a broken video.

The first time I run the program it asked for the ffmpeg location. I chose ffmpeg.exe, stopped the program, started it again and then it worked fine.

When I had the broken video on VLC, if I chose "properties", it did not show any properties (coded, duration, etc). With a working video it does show those details. I've read that the most important properties of video files are often at the end of the file. That's why it won't play if some bytes are missing.

Could you maybe try on a different computer to see if there's any difference?

Lord_of_the_Galaxy

@hamoid A different computer? That could take a long time, but I'll try it on any computer I can. I am presently researching exactly what the stop button does to see if I could possibly shed some light on the problem. Thanks for your effort, we should probably be able to solve the problem eventually.
As for the ESC problem, do look into it and see if the mistake was in your code (post the code here if you haven't got the time) or in Processing itself (it should be reported then).

hamoid

The code of the library is very simple: https://github.com/hamoid/video_export_processing/blob/master/src/com/hamoid/VideoExport.java

Pressing ESC on my Linux quits the program fine. In the Windows machine it did not when using JAVA2D. It did quit with P2D (if I remember right).

Lord_of_the_Galaxy

I'm afraid you misunderstood me. I asked you to post the code of the test application you used, not of the library. I've already seen the code for the library, and found that it should technically work.

hamoid

Ah, I see. The code was just the "basic" example that comes with the Video Export library. Can't get much more minimal than that :)

Lord_of_the_Galaxy

Then I guess that is some kind of funny, new error. It closes on ESC when I use it on my Win8.1 system. Could it be that the window was inactive when you pressed ESC?

hamoid

I tried multiple times and I first clicked on the window to focus on it. The behavior was different on Java2D and P2D. But it was somehow related to the library, because with an empty sketch (no code) ESC did work fine.

Lord_of_the_Galaxy

Alright then. I'll try to see if that is a processing error or library error. Keep working on the library whenever you can and keep me updated on any updates to the library through this forum thread. I'll inform you if I can test it on other computers.

Slugnic

Hello, I am also trying to figure out how to use your library in an application. I'm on windows 8.1 with Processing 3.1 It seems the only thing that works for me is using the Stop button in processing. using exit(), closing the app by pressing the red x in the upper corner of the window, or pressing esc results in a broken movie file...

The strangest this is that if i use P2D and begin and end a recording and then close the window by clicking on the red x in the upper corner produces a working video...using esc or exit() still do not work...

ok...I just did some more tests and it seems to compile fine as .exe

Slugnic

Either way, thanks for the Library!

hamoid

Thanks for your detailed report @Slugnic ! There's now an issue open in the Processing issues about this: https://github.com/processing/processing/issues/4445

Slugnic

cool, i had one other question...if I move the ffmpeg folder to a new location, how do i get your library to forget ffmpegs current location and set it to the new location? Thanks!

hamoid

See forgetFfmpegPath() at http://funprogramming.org/VideoExport-for-Processing/reference/com/hamoid/VideoExport.html I should make this more obvious.

Slugnic

fantastic, works well!

Lord_of_the_Galaxy

@Slugnic Thanks to you I found way to make it work too!
JAVA2D, P2D, P3D still don't work in any way except for the IDE stop button(even after exporting as application). However, the new (& at least for me, rather slow) FX2D renderer seems to work fine at all times.

Slugnic

@Lord_of_the_Galaxy Hmm, I can't seem to get FX2D to work... are you calling exit()?

theliquu69

I am writing a program for making frame-by-frame animations, take a look at this source code: http://pastebin.com/ZybVvs3C I used pastebin to post the code because it is very long, also it will expire in 1 week because I don't want to spoil the app. So, as I read, P2D works fine with the red X button. But it seems like I can't use it in my code. What have I done wrong?

I am using Windows 7, 64-bit.

Slugnic

I so wish i could use this library with the new fullscreen() processing function...but i would need to be able to use esc or call some kind of dispose method. would you recomend reverting back to processing 2?

Slugnic

@theliquu69 I tried your code, and as long as the dispose and exit are not called and I use P2D seems to work fine as long as I close the Papplet window with the red X

Lord_of_the_Galaxy

@Slugnic Nope, I don't call exit().(Although it works even if I do).

Lord_of_the_Galaxy

@Slugnic Nope, I don't call exit().(Although it works even if I do).

Futzi47

Hi hamoid. Thanks a lot for this very useful library.

I think I'm having similar issues as Lord_of_the_Galaxy and Slignic, but running on OS X el capitan. The way my sketch works is a little more complex. I also use a lot of classes I wrote myself so i'll try to narrow down the way I use VideoExport to the essential:

VideoExport video_export;
int max_num_clips = 1000; //export 1000 videoclips...
int num_clips = 0;
int max_clip_duration = 2;  //...each being 2 seconds long.
int clip_duration = 0;
int frame_rate = 30;

void reset(PApplet parent, String video_out)
{
    video_export = new VideoExport(parent, video_out);
    video_export.setFrameRate(30);
    //also reset all the other stuff in the sketch that has no influence on 
    //VideoExport
}

void setup()
{
    size(3840, 1080, OPENGL);
    smooth(8);
    video_export = new VideoExport(this, "/some/file/path/my_clip_####");
    video_export.setFrameRate(30);
}

void draw()
{
    //do some artsy stuff
    video_export.saveFrame();

    if(clip_duration>=max_clip_duration)
    {
        video_export.dispose();
        /*after this, I call a function that starts a different application
            (SuperCollider), which generates a soundfile that uses data beeing generated
            with this processing sketch. once it is done, it will run ffmpeg to
            concatenate the previously disposed videofile and the generated
            soundfile. */
        if(num_clips >= max_num_clips)
        {
            exit();
        }
        else
        {
            reset(this, "/some/file/path/my_clip_####");    
        }
    }

    clip_duration += 1/(float)frame_rate;
    num_clips ++;
}

most of the time this works perfectly, but some times I get the same error message:

fmpeg version 3.0.2 Copyright (c) 2000-2016 the FFmpeg developers built with Apple LLVM version 7.3.0 (clang-703.0.29) configuration: --prefix=/usr/local/Cellar/ffmpeg/3.0.2 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libxvid --enable-vda libavutil 55. 17.103 / 55. 17.103 libavcodec 57. 24.102 / 57. 24.102 libavformat 57. 25.100 / 57. 25.100 libavdevice 57. 0.101 / 57. 0.101 libavfilter 6. 31.100 / 6. 31.100 libavresample 3. 0. 0 / 3. 0. 0 libswscale 4. 0.100 / 4. 0.100 libswresample 2. 0.101 / 2. 0.101 libpostproc 54. 0.100 / 54. 0.100 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe562808600] moov atom not found /Users/tristan/Documents/Processing/aggregate/MF_Studies_2/_video/MF_Studies_2_2016_6_11_11_42_18.mov: Invalid data found when processing input

so out of a thousand clips my computer generates, around 900 get concatenated succesfully, while the others don't because the exported videofile is corrupted. Also, the last clip, at which the processing sketch finishes via the exit(); function is allways corrupted.

I have run a few tests and I'm having some ideas: Most of the times this doesn't work is when I have other applications running in the background (safari, mail). SuperCollider also starts almost imidiately after I call the dispose() method; My assumption is, that whatever is beeing executed when calling the dispose method() is not finished when SuperCollider runs ffmpeg to concatenate video and audio. I've tried adding a delay() with up to 5 seconds after dispose() and before I run SuperCollider, getting some minor improvement, but not fully reliable, yet and I can't say for sure whether this actually causes the issue or not. Maybe if there was a way to check if VideoExport is done disposing or not?

I'd be really glad if you could look into this. Maybe add a method that returns true when VideoExport is done disposing?

hamoid

Interesting usage @Futzi47 ! I just discovered that ProcessBuilder has a waitFor() method, which might help. I uploaded a version which uses it to http://funprogramming.org/VideoExport-for-Processing/download/VideoExport-11.zip Could you let me know if that solves your issue? If it's of any use, you can see the source code of the library at https://github.com/hamoid/video_export_processing

Futzi47

Hi @hamoid, Thanks so much! so far this seems to do the trick! I've had 2 corrupted files at first, but then I did a performance comparison of both videoExport versions while running CPU intensive applications in the background. the old version failed to export any file correctly, the new version worked perfectly. So this definitely seems to be the problem in some way. I'm still wondering though why I got 2 corrupted files in the beginning and now I don't get any. I'll run a few more tests to be sure and let you know.

Futzi47

Hey @hamoid. So i've run a few more tests and managed to bring to light the following:

the new videoExport library doesn't produce any more corrupted files! ... unless the method dontSaveDebugInfo(); is called. then it produces faulty files. I tried to look it up in the source code but couldn't find the usage of the boolean set in the method

public void dontSaveDebugInfo() {
        saveDebugInfo = false;
    }

another thing that comes up after around 230 exported clips is the sketch hangs itself up and i get the following outOfMemory error message.:

java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at processing.opengl.PSurfaceJOGL$2.run(PSurfaceJOGL.java:451) at java.lang.Thread.run(Thread.java:745)Caused by: java.lang.OutOfMemoryError: Java heap space at com.hamoid.VideoExport.<init>(Unknown Source) at com.hamoid.VideoExport.<init>(Unknown Source) at aggregate$Export.init(aggregate.java:536) at aggregate$Export.<init>(aggregate.java:524) at aggregate.init_1(aggregate.java:1692) at aggregate.draw(aggregate.java:68) at processing.core.PApplet.handleDraw(PApplet.java:2403) at processing.opengl.PSurfaceJOGL$DrawListener.display(PSurfaceJOGL.java:861) at jogamp.opengl.GLDrawableHelper.displayImpl(GLDrawableHelper.java:692) at jogamp.opengl.GLDrawableHelper.display(GLDrawableHelper.java:674) at jogamp.opengl.GLAutoDrawableBase$2.run(GLAutoDrawableBase.java:443) at jogamp.opengl.GLDrawableHelper.invokeGLImpl(GLDrawableHelper.java:1293) at jogamp.opengl.GLDrawableHelper.invokeGL(GLDrawableHelper.java:1147) at com.jogamp.newt.opengl.GLWindow.display(GLWindow.java:759) at com.jogamp.opengl.util.AWTAnimatorImpl.display(AWTAnimatorImpl.java:81) at com.jogamp.opengl.util.AnimatorBase.display(AnimatorBase.java:452) at com.jogamp.opengl.util.FPSAnimator$MainTask.run(FPSAnimator.java:178) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505)

I guess this has something to do with the used memory not being freed perhaps but this is pretty much where my knowledge of what goes on under the surface ends. I've tracked my memory usage while running the sketch and shortly before crashing java hits around 4 GB of memory usage.

i've tried doing the following:

this.videoExport.saveFrame();
this.videoExport.dispose();
this.videoExport = null;
System.gc();
//then
video_export = new VideoExport(this, "/some/file/path/my_clip_####");

getting no significant improvement.

I admit I'm using the libary in a rather extreme way and the way it works right now is absolutely sufficient. But if you'd find a fix for this, i'd be really glad. Thanks again for your work!

hamoid

Thanks for being such an extreme tester @Futzi47 ;)

The first issue: that variable is used in line 337: https://github.com/hamoid/video_export_processing/blob/master/src/com/hamoid/VideoExport.java#L337

Maybe I'll have to redirectOutput() to some kind of temporary file when no debug info is desired.

About the second issue, It would be good to avoid calling new VideoExport() more than once. To make that work in your case, I should add a new method like videoExport.setVideoFile("/some/file/path/my_clip_2752.mp4").

I'll look into it when I have no urgent tasks pending. Thanks again! I'll add these as issues in GitHub.

Lord_of_the_Galaxy

@Futzi47 Out of Memory errors cannot be solved easily without seeing the whole code.
@hamoid So Processes were the problem. I think now it will work.

Futzi47

@Lord_of_the_Galaxy You're right, but my code is a pretty big pile of stuff, way above 1000 lines of code. wouldn't wan't anybody to have to dig through that mess ;). Also I've managed to circle the problem area to videoExport since the problem doesn't occur When I don't use videoExport. Other than that I'm not using it any differently than in the example above.

Lord_of_the_Galaxy

@Futzi47 I think I realise the problem - your code saves so many images to VideoExport that it runs out of memory. There is, however , a way to set that but it would mean a major overhaul of the library. :D
To check if that is right, measure the memory used by one clip and multiply it by the number of clips. That would help to give a good estimate of the memory used by the lib(pls post the value here).

Futzi47

@Lord_of_the_Galaxy nope that's not the problem. While saveFrame() is called I'm keeping track of the memory and it always mounts up to a certain amount of MB depending on the length of the clip and once the dispose() method is called, it drops back to normal PLUS 10-20 MB, which stacks up to a few GB after around 200 clips. The Images get dumped via the dispose() method. there's gotta be something else...

Lord_of_the_Galaxy

@Futzi47 I'll try myself and check to find the problem.Thanks!
BTW I hope you know that System.gc () doesn't always call the Garbage Collector.(Google it for more info)

Lord_of_the_Galaxy

@hamoid Finally, that latest version of the library is working flawlessly. Thanks a lot for this wonderful library. What change did you do that made this difference?

theliquu69

Hello, it's me again. I've been working on a video game, and thought it would be cool if I implemented a function to record video. The problem is, apparently the latest version of FFmpeg doesn't work with Video Export. This is my ffmpeg.txt:

ffmpeg version N-87191-gca7dc3e Copyright (c) 2000-2017 the FFmpeg developers built with gcc 4.9.2 (Raspbian 4.9.2-10) configuration: libavutil 55. 74.100 / 55. 74.100 libavcodec 57.104.101 / 57.104.101 libavformat 57. 81.100 / 57. 81.100 libavdevice 57. 8.100 / 57. 8.100 libavfilter 6.101.100 / 6.101.100 libswscale 4. 7.103 / 4. 7.103 libswresample 2. 8.100 / 2. 8.100 Unrecognized option 'crf'. Error splitting the argument list: Option not found

EDIT: @hamoid After some quick research, I have found this: http://ffmpeg.gusari.org/viewtopic.php?f=11&t=2529 This might help you solve the issue in the library.

jeremydouglass

@theliquu69 -- by:

This might help you solve the issue in the library.

...do you mean @hamoid ?

hamoid

Hi, I mention this in the github page: https://github.com/hamoid/video_export_processing

Could it be that your copy ffmpeg does not have libx264 enabled?

It works fine for me with ffmpeg 3.3.3 (current version).

hamoid

BTW. I think you are using a raspberry pi, right? If that's the case, is it fast enough to run a game and export a video at the same time? I believe the raspberry pi and specially the sdcard are probably not very fast. But it's worth trying for sure! :)

theliquu69

@jeremydouglass Yeah, forgot about that. Thanks, I edited my comment.

@hamoid Yeah, I use a Raspberry Pi, it's pretty tough (about 20 fps with no tiles on the screen, 15 fps with all tiles being rendered on a single layer (there is 3 layers) with 2D shadows), in fact my current profile pic is a screenshot of an earlier version of this game. But hey, it works, I will try and record it on my Windows machine and report the results. By the way, I have tried installing the package libx264 but apt-get reports that this package doesn't exist.

theliquu69

@hamoid Also I am using FFmpeg built from source code, because the Raspbian repositories don't have the binary for Raspbian Jessie.

According to the SD Card speed, I have a class 10 high-speed Samsung SD card (sadly not UHS, didn't have as much money to buy one :P and idk if a Raspberry Pi even supports these) so it souldn't be slowing my sketch down too much although I am not saying it will not slow it down. Taking a screenshot with save() creates a lag spike, idk how efficient your method is though.

hamoid

This may help: https://trac.ffmpeg.org/wiki/CompilationGuide/RaspberryPi Find "264" in that page. Let us know how much it affects performance, once it works :)

theliquu69

I am having some trouble compiling libx264, make just kind of stops on analyse.c and doesn't do anything. I will try to clone from git, configure and run make again. Meanwhile, is there an option in Video Export that will allow me to bundle FFmpeg within the data folder and not bother the user to choose its path? In the end I will use Windows as my dev platform (after I finish my closed beta version of the game) and at that stage (closed beta) the game will be Windows only, so I would like to include FFmpeg with the download but I can't find an option for that in Video Export.

hamoid

I guess I could add an "setFfmpegPath(String path)" method to the library. That should allow you specifying that ffmpeg is inside the data folder, right?

theliquu69

That would be nice ;) If you use a relative path, which you do in the VideoExport constructor, it should allow me to point to a subfolder in the data folder directly

theliquu69

Well... I don't know if I wasted my time compiling libx264, and then FFmpeg once more.

ffmpeg.txt:

/usr/local/bin/ffmpeg: error while loading shared libraries: libx264.so.152: cannot open shared object file: No such file or directory

theliquu69

@hamoid I have tried recording on my PC but for some reason the library thought ffmpeg failed, when it didn't. It made a perfect playable movie, looking as it should, but it crashed my game.

ffmpeg.txt:

[removed because it was pointless]

EDIT: This time it isn't you, but me who made a mistake. So basically I had a boolean isRecording that was turned on when user pressed Shift+\ (or | as I have it in code) and if the boolean was true it would call rec.saveFrame(). What I did wrong is forgot to toggle it off when user presses | again, so it still called rec.saveFrame().

fausto

jumping in this thread for thanking hamoid for writing and sharing this excellent library. It worked fine for the purpose of my project (capturing a time lapse over a certain amount of time, and playing back the time lapse immediately after capturing).

hamoid

You're welcome @fausto ! I'm happy that you found it useful. Thank you for writing! :)

And good to hear you made your program work @theliquu69 :)