The Lazy Video player
in
Share your Work
•
2 years ago
Hi All !
I'm really excited to post my first processing sketch! First of all, please excuse the dirtiness of the code (and of the sketch itself - it consumes 100% of my CPU to run), but I'm still a noob :)
I like to watch movies at night before getting asleep, and always wanted the movie to rotate according to my head's own coordinates in 3D space. Therefore, this sketch (aims at) rotating a video according to the viewer's head orientation.
To run this sketch:
- Make sure to have the openCV library installed
- Download the HAAR eye cascade (here: http://www-personal.umich.edu/~shameem/haarcascade_eye.html ). Store the XML file in your sketch folder (not in the data folder).
- Put any movie you want to test this on in your data folder, and make sure to adjust the code accordingly. The movie I used ships with one of the examples; it's called station.mov
// -------------- Sketch Start
import hypermedia.video.*;
import java.awt.Rectangle;
import processing.video.*;
OpenCV face;
PFont font;
Movie myMovie;
int EyeAX, EyeAY, EyeBX, EyeBY, iAngle = 0;
int BigY, SmallY;
float DeltaX, DeltaY, Slope, fAngle = 0;
void setup() {
size( 960, 240 );
font = loadFont("HLN16.vlw");
myMovie = new Movie(this, "station.mov");
myMovie.loop();
face = new OpenCV(this);
face.capture (320,240);
face.cascade( "haarcascade_eye.xml" );
}
void draw() {
face.read();
background(200,200,200);
image ( face.image(), 0,0);
Rectangle[] faces = face.detect();
smooth();
noFill();
stroke(255,20,20,140);
textFont(font,16);
for (int j = 0; j < faces.length; j++){
if ( j ==1 ) {
smooth();
strokeWeight (1);
if ( faces[j].x < faces[j-1].x ) {
EyeAX = (faces[j].x + faces[j].width / 2);
EyeAY = (faces[j].y + faces[j].height /2);
EyeBX = (faces[j-1].x + faces[j-1].width / 2);
EyeBY = (faces[j-1].y + faces[j-1].height / 2);
}
else {
EyeAX = (faces[j-1].x + faces[j-1].width / 2);
EyeAY = (faces[j-1].y + faces[j-1].height /2);
EyeBX = (faces[j].x + faces[j].width / 2);
EyeBY = (faces[j].y + faces[j].height / 2);
}
if (EyeBY < EyeAY ) { BigY = EyeAY ; SmallY = EyeBY;} else { BigY = EyeBY; SmallY = EyeAY; }
DeltaX = EyeBX - EyeAX;
DeltaY = EyeBY - EyeAY;
Slope = DeltaY / DeltaX;
// line between the eyes
line (EyeAX, EyeAY,EyeBX, EyeBY);
// line at the horizon
line(0,BigY, 320, BigY);
// line between high eye and horizon
line (EyeAX, EyeAY, EyeAX, BigY);
line (EyeBX, EyeBY, EyeBX, BigY);
fAngle = asin( Slope );
println (fAngle);
}
}
DrawMovie();
}
void DrawMovie(){
pushMatrix();
translate(640,120);
rotate(-(fAngle)/2);
image(myMovie,0 - myMovie.width/2,0 - myMovie.height /2 );
popMatrix();
}
// ------------- Sketch End
I modified two other sketches to come up with this one. First, an image is acquired from my Macbook's iSight camera. OpenCV is fed with a custom HAAR cascade of eyes, which are detected. Each eye is given an (x,y) coordinate. The sketch then rotates a video according to the slope of the line passing through both eyes.
Some things that I'd like to resolve:
- High CPU utilization ( I really have no clue why this is happening).
- The sketch will create the weirdest results if more than 2 eyes are seen by the cam. I have many ideas that I can formulate in pseudo code, but I find myself unable to achieve them in processing yet. At a future stage, it might be beneficial to track faces first, then try to detect eye features within the boundaries of a face.
- Have the sketch work with acute angles. So far, Eye detection is possible to a ~45 degrees extent. After that, eyes aren't detected anymore. Maybe this could be resolved by progressively rotating the "canvas" in the reverse direction that the eyes are adopting ?
- There's a tiny jitter that constantly appears, even if one tries to hold his head steady. I'd love for rotation to happen based on an average of the past 5 screen captures, instead of real-time.
- Maybe adding a lock feature? something that will do rotations on smooth, 10deg increments.
In the future, I'd like to track noses too. This way, I could possibly calculate the head's rotation around its 3 axes instead of just 1. I could then play the movie that I want on a simple OpenGL polygon that would be located according to my head's orientation.
I'm loving processing so far. I'd really appreciate your comments, suggestions and thoughts.
Thanks guys !
I'm really excited to post my first processing sketch! First of all, please excuse the dirtiness of the code (and of the sketch itself - it consumes 100% of my CPU to run), but I'm still a noob :)
I like to watch movies at night before getting asleep, and always wanted the movie to rotate according to my head's own coordinates in 3D space. Therefore, this sketch (aims at) rotating a video according to the viewer's head orientation.
To run this sketch:
- Make sure to have the openCV library installed
- Download the HAAR eye cascade (here: http://www-personal.umich.edu/~shameem/haarcascade_eye.html ). Store the XML file in your sketch folder (not in the data folder).
- Put any movie you want to test this on in your data folder, and make sure to adjust the code accordingly. The movie I used ships with one of the examples; it's called station.mov
// -------------- Sketch Start
import hypermedia.video.*;
import java.awt.Rectangle;
import processing.video.*;
OpenCV face;
PFont font;
Movie myMovie;
int EyeAX, EyeAY, EyeBX, EyeBY, iAngle = 0;
int BigY, SmallY;
float DeltaX, DeltaY, Slope, fAngle = 0;
void setup() {
size( 960, 240 );
font = loadFont("HLN16.vlw");
myMovie = new Movie(this, "station.mov");
myMovie.loop();
face = new OpenCV(this);
face.capture (320,240);
face.cascade( "haarcascade_eye.xml" );
}
void draw() {
face.read();
background(200,200,200);
image ( face.image(), 0,0);
Rectangle[] faces = face.detect();
smooth();
noFill();
stroke(255,20,20,140);
textFont(font,16);
for (int j = 0; j < faces.length; j++){
if ( j ==1 ) {
smooth();
strokeWeight (1);
if ( faces[j].x < faces[j-1].x ) {
EyeAX = (faces[j].x + faces[j].width / 2);
EyeAY = (faces[j].y + faces[j].height /2);
EyeBX = (faces[j-1].x + faces[j-1].width / 2);
EyeBY = (faces[j-1].y + faces[j-1].height / 2);
}
else {
EyeAX = (faces[j-1].x + faces[j-1].width / 2);
EyeAY = (faces[j-1].y + faces[j-1].height /2);
EyeBX = (faces[j].x + faces[j].width / 2);
EyeBY = (faces[j].y + faces[j].height / 2);
}
if (EyeBY < EyeAY ) { BigY = EyeAY ; SmallY = EyeBY;} else { BigY = EyeBY; SmallY = EyeAY; }
DeltaX = EyeBX - EyeAX;
DeltaY = EyeBY - EyeAY;
Slope = DeltaY / DeltaX;
// line between the eyes
line (EyeAX, EyeAY,EyeBX, EyeBY);
// line at the horizon
line(0,BigY, 320, BigY);
// line between high eye and horizon
line (EyeAX, EyeAY, EyeAX, BigY);
line (EyeBX, EyeBY, EyeBX, BigY);
fAngle = asin( Slope );
println (fAngle);
}
}
DrawMovie();
}
void DrawMovie(){
pushMatrix();
translate(640,120);
rotate(-(fAngle)/2);
image(myMovie,0 - myMovie.width/2,0 - myMovie.height /2 );
popMatrix();
}
// ------------- Sketch End
I modified two other sketches to come up with this one. First, an image is acquired from my Macbook's iSight camera. OpenCV is fed with a custom HAAR cascade of eyes, which are detected. Each eye is given an (x,y) coordinate. The sketch then rotates a video according to the slope of the line passing through both eyes.
Some things that I'd like to resolve:
- High CPU utilization ( I really have no clue why this is happening).
- The sketch will create the weirdest results if more than 2 eyes are seen by the cam. I have many ideas that I can formulate in pseudo code, but I find myself unable to achieve them in processing yet. At a future stage, it might be beneficial to track faces first, then try to detect eye features within the boundaries of a face.
- Have the sketch work with acute angles. So far, Eye detection is possible to a ~45 degrees extent. After that, eyes aren't detected anymore. Maybe this could be resolved by progressively rotating the "canvas" in the reverse direction that the eyes are adopting ?
- There's a tiny jitter that constantly appears, even if one tries to hold his head steady. I'd love for rotation to happen based on an average of the past 5 screen captures, instead of real-time.
- Maybe adding a lock feature? something that will do rotations on smooth, 10deg increments.
In the future, I'd like to track noses too. This way, I could possibly calculate the head's rotation around its 3 axes instead of just 1. I could then play the movie that I want on a simple OpenGL polygon that would be located according to my head's orientation.
I'm loving processing so far. I'd really appreciate your comments, suggestions and thoughts.
Thanks guys !