How to use video to create music...

bmz

Sorry I'm new to programming but I've searched the forums and generally online, with no useful answer to this... I can get videos playing and have managed to use other peoples shared games and generative code to input midi into Logic and record... I'm just having trouble combining the two.

So... how would I go about using light information such as brightness of a video to influence midi note on/off... would anyone share such code with me?

Any info would be greatly appreciated! sorry if this seems obvious to you.

Thanks,

bmz

colouredmirrorball

Do you mean something like if the average brightness of the video reaches a threshold value it sends a note on and when it goes below it sends a note off? Or do you want a specific pixel or region? Or detect a bright flash?

jeremydouglass

@bmz --

I think the term you are looking for is "sonification" -- turning-into-sound (like visualization is turning-into-visuals).

Since you are interested in midi specifically, the library you want is probably:

• The MidiBus

Look at the MidiBus example sketches -- once you can send a note on/off, you can send it based on anything.

Your examples inputs all seem to be based on computing the average of pixels from a video frame or frame the canvas, and then either mapping that to a note (see map()) or using an if.

• https://processing.org/reference/pixels.html
• https://processing.org/reference/PImage_pixels.html
• https://forum.processing.org/two/discussion/15573/get-the-average-rgb-from-pixels

jeremydouglass

For MIDI also see: http://explodingart.com/soundcipher/tutorials.html

bmz

Thanks for all the help!