Problem with creating video! - Processing Forum

Loading...

Processing Forum

Core Library Questions

fakefaker..

Problem with creating video!

in Core Library Questions • 2 years ago

hey guys,

i just wrote a little music visualization program and now i want to export it as a video.

i used the MovieMaker with addFrame and finish. so far so good. it generates a .mov file if i start it, but it it only becomes 129kb big and when i finish the program this file jumps back to 2kb.

something seems to be written wrong but i can't see where my fault is. i also dont know how much keyframes i should use.
maybe you guys can help me out. here's the code.

import processing.video.*;

import ddf.minim.*;

MovieMaker mm;
Minim minim;
AudioPlayer player;

void setup () {
// Sketch einrichten
size (1920, 1080);
noStroke ();
smooth ();
mm = new MovieMaker(this, width, height, "test.mov",
                     30, MovieMaker.H264, MovieMaker.BEST,5);

minim = new Minim (this);
player = minim.loadFile ("Cragga_-_Please_Mr_Postman__Dubstep_Refix_.mp3");
//player = minim.loadFile ("SATURATE!RECORDS - heRobust - Albumin Ep (STRTEP002) - 10 Shawty Swing My Way (Busted).mp3");
//player = minim.loadFile ("02-mount_kimbie-would_know.mp3");
//player = minim.loadFile ("Kaczinski - Fuck this Melody2.mp3");
player.play ();

}

void draw() {

fill (0,50);
rectMode (CORNER);
rect (0, 0, width, height);

float dimension = player.mix.level () * 600;
rotate(PI/dimension * 10);

println (dimension);

for(int y = 960; y <= height; y = y + y^5){
    fill(dimension/5,3,100);
    triangle(width/2, 0, dimension, dimension, 600, y);

    fill(y/4,y^33,dimension);
    triangle(width/2, 600, 0, y, 600, y/66);

    fill(y,dimension,dimension);
    triangle(dimension * 2, dimension, 0, y, 600, y);

    fill(y + 10,dimension,dimension);
    triangle(dimension + 100, dimension + 18, dimension, y, 350, y);

    fill(y^99,dimension,y^233);
    triangle(width/2, 22, 150, y, 450, y);

    fill(33,33,y^233);
    triangle(dimension + 300, dimension + 22, 100, y, 500, dimension);

    fill(222,dimension,y^233);
    triangle(dimension, dimension, 33, y, 600, y);

    fill(dimension * 2,dimension/4,y^233);
    triangle(dimension * 3, dimension + 20, 0, y, dimension, y);

    mm.addFrame();
}
    import processing.video.*;



}
   void keyPressed() {

if (key == 's' ) {
    mm.finish(); // Finish the movie if space bar is pressed!
}
}

1

Replies(17)

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

probably importing the video library every frame...
search for the second
import processing.video.*;

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

took another look.
as well as adding the frames in the loop and not at the end of draw

fakefaker..

Re: Problem with creating video!

2 years ago

i switched now from codec 264 to 263.

now it records something, but in a very bad quality and ´way too fast.

somehow 264 doesnt work. i don't know why.. i updated my quicktime player.

any ideas whats going on?

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

did you made the mentioned changes?
post your code again please

fakefaker..

Re: Problem with creating video!

2 years ago

import processing.video.*;

import ddf.minim.*;

MovieMaker mm;
Minim minim;
AudioPlayer player;

void setup () {
size (1920, 1080);
noStroke ();
smooth ();
mm = new MovieMaker(this, width, height, "test2.mov",
                     30 , MovieMaker.H264, MovieMaker.BEST);

minim = new Minim (this);
player = minim.loadFile ("Cragga_-_Please_Mr_Postman__Dubstep_Refix_.mp3");
//player = minim.loadFile ("SATURATE!RECORDS - heRobust - Albumin Ep (STRTEP002) - 10 Shawty Swing My Way (Busted).mp3");
//player = minim.loadFile ("02-mount_kimbie-would_know.mp3");
//player = minim.loadFile ("Kaczinski - Fuck this Melody2.mp3");
player.play ();

}

void draw() {

mm.addFrame();
fill (0,50);
rectMode (CORNER);
rect (0, 0, width, height);

float dimension = player.mix.level () * 600;
rotate(PI/dimension * 10);

println (dimension);

for(int y = 960; y <= height; y = y + y^5){
    fill(dimension/5,3,100);
    triangle(width/2, 0, dimension, dimension, 600, y);

    fill(y/4,y^33,dimension);
    triangle(width/2, 600, 0, y, 600, y/66);

    fill(y,dimension,dimension);
    triangle(dimension * 2, dimension, 0, y, 600, y);

    fill(y + 10,dimension,dimension);
    triangle(dimension + 100, dimension + 18, dimension, y, 350, y);

    fill(y^99,dimension,y^233);
    triangle(width/2, 22, 150, y, 450, y);

    fill(33,33,y^233);
    triangle(dimension + 300, dimension + 22, 100, y, 500, dimension);

    fill(222,dimension,y^233);
    triangle(dimension, dimension, 33, y, 600, y);

    fill(dimension * 2,dimension/4,y^233);
    triangle(dimension * 3, dimension + 20, 0, y, dimension, y);


}




}
   void keyPressed() {

if (key == 's' ) {
    mm.finish();
}
}

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

you have to add it at the end. if you do that at the beginning there is nothing drawn i guess

fakefaker..

Re: Problem with creating video!

2 years ago

it already creates a video, but the quality and the speed is completely bad. thats the major problem right now..

amnon.owed

amnon.owed

Re: Problem with creating video!

2 years ago

Then use...

mm = new MovieMaker(this, width, height, "test.mov",
30,MovieMaker.ANIMATION, MovieMaker.LOSSLESS);

...to get higher quality video output.

With regard to speed. Moviemaker is absolutely unsuitable to capture video in realtime at a resolution of 1920 x 1080 if that's what you are expecting. Probably no method will be able to do this. The only method that comes close is saving images in the TGA file format.

There are many discussions on this topic both here and in the old forum. For example here, here, here and here.

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

good point. saving stills using saveFrame() is probably a better idea, but its still slow...

Amnon, do you see a difference in saving TGA and PNGs ? thats what i always use... should be the same in quality...

fakefaker..

Re: Problem with creating video!

2 years ago

hmm what do u mean by slow? slow in rendering?

how would you write it? never made a video like that, and whats with the original speed of the video, because its a music visual and of course the synched speed is very important.

cedrickiefer

cedrickie..

Re: Problem with creating video!

2 years ago

yes thats a problem, but you have that either way.
read that post where we discussed that problem

http://processing.org/discourse/yabb2/YaBB.pl?num=1174169711/0

fakefaker..

Re: Problem with creating video!

2 years ago

alright, i used the animation codec with lossless quality. now the quality is ok for me id say but the video way too fast.
how does that come?

amnon.owed

amnon.owed

Re: Problem with creating video!

2 years ago

@ cedric: In general I also favor PNG because it's a little more compatible for example with windows image viewer, but when it comes to hi-res image sequences I always go with TGA. It has the quality like PNG, but -at least on my system winxp-32bit- it's much faster when saving. So far it's the only image format I've found that approaches real-time image saving at a high resolution. Also see the comparison posted here. Another argument for me is that TGA works well (read: faster) in After Effects when compositing image sequences.

@ fake: with the moviemaker your sketch is probably running at like 10 fps and your output movie has 30 fps. So it will play 3 times as fast as you saw it while recording. This frameRate thing plus the syncing problem is what we have been talking about all along. Check out some of the links above. You can see the frameRate of your sketch by adding this line to the draw loop:

println(int(frameRate));

fakefaker..

Re: Problem with creating video!

2 years ago

yea my fps are jumpin between 5 and 6..

i get the idea of single frame capturing but i'm too new to this program to really understand the entire codes from your link above.

would be very nice if you could use one of the ideas on my code and post it, then i maybe understand better how it's working.

thank you so far for the help.

amnon.owed

amnon.owed

Re: Problem with creating video!

2 years ago

The mighty Dave already made something that should be an excellent starting point for you.

http://www.davebollinger.com/works/p5/FFTOfflineRendererMinim.zip

It's a minim based offline renderer including settings to work in realtime or offline and save to image, movie or turned off.

I just tested it with an mp3 and put the output movie together with the mp3 in quicktime. Perfect sync!

I recommend taking this sketch and using it as a starting point to make your own visualization.

fakefaker..

Re: Problem with creating video!

2 years ago

alright.. i tried to put in my sketch into this offline renderer and i got the same problems again..

when i run it with ANIMATION codec at 1080p i've got ~10fps, and when i use 720p i've got around 17fps..
when i run it with H263 i have ~ 5 fps..

whats wrong? you can try it out with your own audio file.

/**
FFTOfflineRendererMinim.pde
June 2009 Dave Bollinger (for Processing 1.0+)
to demonstrate non-real-time fft analysis and rendering
*/

import processing.opengl.*;
import ddf.minim.*;
import ddf.minim.analysis.*;
import processing.video.*;

String audioFilename = "Cragga_-_Please_Mr_Postman__Dubstep_Refix_.mp3";

static final int RENDER_MODE_REALTIME = 0; // use this when dev'ing your viz
static final int RENDER_MODE_OFFLINE = 1; // use this for final render
int renderMode = RENDER_MODE_OFFLINE; // change this as desired

static final int OUTPUT_TYPE_NONE = 0; // use this when dev'ing
static final int OUTPUT_TYPE_IMAGE = 1; // use this..
static final int OUTPUT_TYPE_MOVIE = 2; // ..or this, for final render
int outputType = OUTPUT_TYPE_MOVIE; // change this as desired

Minim minim;
AudioSample audio;
FFT fft;
MovieMaker movie;
int frameNumber = 0;
float [] audioBuffer; // will hold the entire audio file
float [] fftBuffer; // will hold just a portion for fft analysis
// sync math:
float sampleRate; // we'll query the audio file for this later
float framesPerSecond = 30.0;
float samplesPerFrame; // ==sampleRate/framesPerSecond
int fftBufferSize; // total size of the fft buffer
int fftMixSize; // occupied size of the fft buffer
float exactFramesToRender; // exactly how many (fractional) video frames == audio duration
int wholeFramesToRender; // no such thing as a fractional video frame, so round up

void setup() {
size(1280,720,OPENGL);
if (renderMode==RENDER_MODE_REALTIME)
    frameRate(framesPerSecond); // won't be perfect sync, but we try our best anyway
if (outputType==OUTPUT_TYPE_MOVIE)
    movie = new MovieMaker(this,width,height,"output.mov",(int)framesPerSecond,MovieMaker.ANIMATION,MovieMaker.BEST);
minim = new Minim(this);
audio = minim.loadSample("Cragga_-_Please_Mr_Postman__Dubstep_Refix_.mp3");
if (audio == null) {
    println("oops, failed to load audio file");
    exit();
    return;

}
audioBuffer = audio.getChannel(BufferedAudio.LEFT);
if ((audioBuffer==null) || (audioBuffer.length==0)) {
    println("oops, failed to acquire sample data");
    // try using a mono and/or lower samplerate version
    // to reduce size of file used for analysis
    exit();
    return;
}
// calc values
sampleRate = audio.sampleRate();
samplesPerFrame = sampleRate / framesPerSecond;
fftMixSize = (int)ceil(samplesPerFrame);
if (renderMode == RENDER_MODE_REALTIME) {
    fftBufferSize = audio.bufferSize();
} else { // offline
    fftBufferSize = nextPowerOfTwo(fftMixSize);
}
exactFramesToRender = audioBuffer.length * framesPerSecond / sampleRate;
wholeFramesToRender = (int)ceil(exactFramesToRender);
fft = new FFT(fftBufferSize, sampleRate);
fftBuffer = new float[fftBufferSize];
if (renderMode == RENDER_MODE_REALTIME) {
    audio.trigger();
}
// report values:
println("INFO:");
println("audio length = " + audioBuffer.length);
println("sampleRate = " + sampleRate);
println("audio duration = " + sec2hms(audioBuffer.length/sampleRate));
println("framesPerSecond = " + framesPerSecond);
println("samplesPerFrame = " + samplesPerFrame);
println("fftMixSize = " + fftMixSize);
println("fftBufferSize = " + fftBufferSize);
println("exactFramesToRender = " + exactFramesToRender);
println("wholeFramesToRender = " + wholeFramesToRender);
println("video duration = " + sec2hms(wholeFramesToRender/framesPerSecond));
println("-----");
println("dimension");
}

String sec2hms(float seconds) {
int hours = (int)floor(seconds / 3600.0);
seconds -= (float)(hours * 3600);
int minutes = (int)floor(seconds / 60.0);
seconds -= (float)(minutes * 60);
return hours + ":" + minutes + ":" + seconds;
}

int nextPowerOfTwo(int target) {
int powtwo = 1;
while (powtwo < target) powtwo<<=1;
return powtwo;
}

public void stop() {
println("stop()");
if (movie != null) movie.finish();
if (audio != null) audio.close();
if (minim != null) minim.stop();
super.stop();
}

void draw() {

///rect (1,1,fftBufferSize/2,fftBufferSize/2);

fill (0,50);
rectMode (CORNER);
rect (0, 0, width, height);

if (finished()) {
    exit();
} else {
    progress();
    analyze();
    render();
    store();
    advance();
}
}

boolean finished() {
// this only works properly when offline rendering
// (unfortunately AudioSample doesn't indicate when done playing realtime)
return (frameNumber >= wholeFramesToRender);
}

void progress() {
if ((frameNumber0) == 0) {
    println("Working on frame " + frameNumber + " of " + wholeFramesToRender);
}
}

void analyze() {
switch(renderMode) {
    case RENDER_MODE_REALTIME:
      analyzeRealtime();
      break;
    case RENDER_MODE_OFFLINE:
      analyzeOffline();
      break;
}
}

void analyzeRealtime() {
fft.forward(audio.mix);
}

void analyzeOffline() {
int pos = (int)(frameNumber * sampleRate / framesPerSecond);
if (pos >= audioBuffer.length) {
    // hmm, finished() should've caught this, why are we here?
    exit();
    return;
}
int mixlen = fftMixSize;
// handle last partial frame
if (pos+mixlen >= audioBuffer.length) {
    mixlen = audioBuffer.length - pos;
}
arraycopy(audioBuffer, pos, fftBuffer, 0, mixlen);
// zero remainder (really only need to zero up to fftMixSize,
// since rest is never used, but we do the whole thing jic...)
for (int i=mixlen; i<fftBufferSize; i++)
    fftBuffer[i] = 0f;
fft.forward(fftBuffer);
}

void render() {

   smooth();
   noStroke();
   fill (0,100);
   rectMode (CORNER);
   rect (0, 0, width, height);

//rotate(PI/fft.getBand(y) * 10);

println (int(frameRate));

//for(int y = 960; y <= height; y = y + y^5){
    for (int y = 0; y < fft.specSize(); y = y+500) {
    rotate(PI/fft.getBand(y) * 10);
      fill(fft.getBand(y)/5,3,100);
    triangle(width/2, 0, fft.getBand(y), fft.getBand(y), 600, y);

    fill(y/4,y^33,fft.getBand(y));
    triangle(width/2, 600, 0, y, 600, y/66);

    fill(y,fft.getBand(y),fft.getBand(y));
    triangle(fft.getBand(y) * 2, fft.getBand(y), 0, y, 600, y);

    fill(y + 10,fft.getBand(y),fft.getBand(y));
    triangle(fft.getBand(y) + 100, fft.getBand(y) + 18, fft.getBand(y), y, 350, y);

    fill(y^99,fft.getBand(y),y^233);
    triangle(width/2, 22, 150, y, 450, y);

    fill(33,33,y^233);
    triangle(fft.getBand(y) + 300, fft.getBand(y) + 22, 100, y, 500, fft.getBand(y));

    fill(222,fft.getBand(y),y^233);
    triangle(fft.getBand(y), fft.getBand(y), 33, y, 600, y);

    fill(fft.getBand(y) * 2,fft.getBand(y)/4,y^233);
    triangle(fft.getBand(y) * 3, fft.getBand(y) + 20, 0, y, fft.getBand(y), y);








   //background(0);
//stroke(255);
//for (int i = 0; i < fft.specSize(); i++) {
    //triangle(i, height, i, height - fft.getBand(i) * 2,100,100);
}
}

void store() {
switch(outputType) {
    case OUTPUT_TYPE_NONE :
      break;
    case OUTPUT_TYPE_IMAGE :
      saveFrame("out rame_"+nf(frameNumber,6)+".png"); // or jpg, tif, etc.
      break;
    case OUTPUT_TYPE_MOVIE :
      movie.addFrame();
      break;
}
}

void advance() {
frameNumber++;
}

amnon.owed

amnon.owed

Re: Problem with creating video!

2 years ago

With offline rendering the framerate no longer matters. That's the great strength of this particular solution. The framerate could be 1 fps. The only thing that matters is that the output movie syncs up with the audio, which is the case. What this sketch basically does is analyse the audio in steps based on the framerate of the final output movie in order for them to sync up later on regardless of the framerate during the rendering process. Note that the duration of the output film and the mp3 are equal. Just composite the audio and video and you'll see they sync up.

Top Reply