transforming a video with parameters from audio file

edited February 2014 in How To...

Can a video be transformed, like have its colors changed or being distorted, using parameters from an audio file?



  • Yes it can. You can use the Minim library to load a sound file and get the volume or the frequencies using FFT as it plays. At the same time you can use the Video library to load and play video. You can access the pixels of each video frame and manipulate them as you wish, using the values you obtained from the sound. This would be all live.

    If you don't want live effects, but you want to convert a video file into a new video file, affected by sound properties of a sound file, it can also be done, but it might be more work. First I would analyze the sound file and save the data as a text file, I would use ffmpeg to split the video file into single frames (a ton of images on disk), then write a program that reads the data from the text files and modifies the frames one by one. Finally I would put together all the modified frames as a video file using ffmpeg again.

Sign In or Register to comment.