Processing 1.0 - Processing Discourse - How to crop Movie Class?!

wilsodj

How to crop Movie Class?!
Mar 13^th, 2007, 12:10pm

I have been using crop() with the Capture Class with no problem, but when I want to do exactly the same thing with the Movie Class there is unfortunately no corrolating function.

Do I need to use pixels[] for this? I have played around with it for a while with no success to speak of.

Basically what I want to do is just grab one line of pixels from each frame of video and have only that line drawn to screen. Ideas?

Any help would be really appreicated. Thanks -

fox

Re: How to crop Movie Class?!
Reply #1 - Mar 16^th, 2007, 12:53pm

Yes you're correct - you can access the pixels array to get part of a video frame. Because the Video class extends PImage you may also be able to use the get() method from PImage.

To grab a single horizontal line use something like this:

for(int x = 0; x < movieWidth; x++)
hLine[x] = myMovie.pixels[movieWidth * y + x];

where y is the screen coord of the line you want to get.

To use get() you'd try something like this:

myMovie.get(xPosn, yPosn, xPosn + w, yPosn + h)

where xPosn and yPosn define your starting pixel, w is a width value and h is a height value (i.e. the ending pixel of the area your getting.

Hope this helps.

wilsodj

Re: How to crop Movie Class?!
Reply #2 - Mar 23^rd, 2007, 3:20pm

Yes, that was great. Using get I was able to get it working in just a few minutes.

However pixels is giving me considerably more trouble, but I want to get it working as it would give me a lot more flexibility for what I am trying to do...

I am getting:
java.lang.ArrayIndexOutOfBoundsException: 100

I modified the code you had put down slightly as I wished to take a vertical line from the video:

loadPixels();

for(int y = 0; y < height; y++) {
hLine[y] = myMovie.pixels[(320 * y) + 100];
}

where 320 is the width of the movie, and 100 is the number of pixels I wish to move from the left side of the image. The error is in the second line of this code. I have tried to initialize an array as follows at the top level:

int[] hLine = new int[width*height];

Why is it going out of bounds then?

Then to draw back to screen I have, what I am almost certain is wrong right now, but can't see what is doing since the first part is not working:

for(int i=0; i<height; i++) {
pixels[i] = hLine[i];
}
updatePixels();

And loadPixels(); is going to automatically load each frame as it comes in?

Thanks again for your help.

fox

Re: How to crop Movie Class?!
Reply #3 - Mar 26^th, 2007, 5:17am

Mmmmm... this requires a bit of a head-scratching. The code fragments look OK. hLine[] is actually way to big to be going out of bounds at 100 which is the value the exception lists (you should only need to declare it as the height of the movie or sketch). So I'd say it's myMovie.pixels[] that's the culprit. You can print out its length field to see how big the array is:

println("myMovie pixels array length: " + myMovie.pixels.length);

Maybe it's better if you post up the full code. Also in processing there's a menu item 'Tools > Copy for Discourse' which copies the entire sketch code and formats it to look the same in these forums, so use that if you like too.

Anyway, it sounds like we're both working on something similar. So maybe I'll post up my code when I've got something working in the next day or so.

Also a quick note on coding: where you have hard-coded numbers such as 100 and 320, it would be better to use variables or java 'constants'. That way if you ever change your movie size for example, you would just change the variable assignment in one place only rather than having to search through your code for all occurences of 100 (which can cause problems if you have other unrelated hard coded values that are also 100). You can use the 'final' modifier to declare a variable as a constant in java:

final int xOffset = 100;

Most people use 'static' with these too, which means that only 1 copy of the variable is produced, no matter how many objects are instantiated i.e. it exists at the class level only, not in the individual objects. This is probably irrelevant in this case and processing tends to hide these kinds of modifiers away anyway, but in regular java code it should save a little bit of memory and time whenever an object is instantiated.

fry

Re: How to crop Movie Class?!
Reply #4 - Mar 26^th, 2007, 3:32pm

another option depends on what renderer you're using. if using P3D (which you should with video anyway) or OPENGL, then you can simply treat the video as a texture (see the reference for the texture() function). then you can use beginShape(QUADS) and only draw the necessary section of the image. it's nearly as efficient as drawing an image any other way (except via the set() command).

wilsodj

Re: How to crop Movie Class?!
Reply #5 - Mar 28^th, 2007, 3:05pm

Okay, so apparently my pixel array is 76800 (which makes sense (width*height = 320*240).

Which, still leaves me confused as to why the array is going out of bounds.

Treating the video as a texture and using beginShape(QUADS) works, but seems to run slower than just using myMovie.get and grabbing and then displaying a single line that way.

The real key for me however, is the fact that I would like to be able to get more than just "straight" lines. I would like to be able to draw a line myself on the screen, and then have the pixel coordinates of the line stored and then use those same coordinates to display the "video line". Basically just another form of slit scan imaging, but with the "slit" capable of taking on a user-defined shape.

Anyway, here is the full code:

Quote:

import processing.video.*;

Movie myMovie;
int y;
int[] hLine = new int[76800];
final int xOffset = 100;
final int movieWidth = 320;

void setup()
{
size(1200, 240);

myMovie = new Movie(this, "Escalator.mov");
myMovie.play();
}

void draw()
{
if(myMovie.available()) {
myMovie.read();
}

loadPixels();
for(y = 0; y < height; y++) {
hLine[y] = myMovie.pixels[(movieWidth * y) + xOffset];
}
for (int i=0; i<height; i++) {
pixels[i] = hLine[i];

}
updatePixels();

}

void movieEvent(Movie m) {
m.read();
}

I am, as I mentioned, quite certain that the method I am using to draw the pixels to screen is incorrect, but it is still hanging up at the same place:

Quote:

for(y = 0; y < height; y++) {
hLine[y] = myMovie.pixels[(movieWidth * y) + xOffset];
}

Where I get the error message:

java.lang.ArrayIndexOutOfBoundsException: 100
at Temporary_9883_5945.draw(Temporary_9883_5945.java:25)
at processing.core.PApplet.handleDisplay(PApplet.java:1355)
at processing.core.PGraphics.requestDisplay(PGraphics.java:564)
at processing.core.PApplet.run(PApplet.java:1450)
at java.lang.Thread.run(Thread.java:613)

Any thoughts?

And again, thanks for the help.

fox

Re: How to crop Movie Class?!
Reply #6 - Mar 30^th, 2007, 3:07pm

OK. I think that the problem may be some subtle quirk of how the movie class works.

Anyway, I'd get rid of the movieEvent method because it's not needed if you're using myMovie.available(), so maybe that's one factor.

Also you will need to put all the slit and display code from draw() inside the if(myMovie.available()) code block. I think maybe the draw event was trying to access the myMovie.pixels array before there was anything to read which is maybe why you were getting the array out of bounds exception at 100.

I tried it out on a small movie and the exception wasn't happening (I was getting some other problems but that's another story) but maybe the movie I had was smaller, so it loaded faster, plus there could be so many other differences between our hardware/software which makes it hard to chase these problems down.

fox

Re: How to crop Movie Class?!
Reply #7 - Mar 30^th, 2007, 3:07pm

Here's some code I came up with to do something similar to what I assume you're trying to get happening:

Quote:

//processing libraries
import processing.video.*;

//variables and object declarations
PMovie myMovie;
final int movieWidth = 160;
final int movieHeight = 120;
final int xOffset = 100;
final int windowWidth = 1024;
final int drawingAreaOrigin = windowWidth * movieHeight;
int currSlice;

void setup()
{
//movie instantiation and settings
myMovie = new PMovie(this, "station.mov");
myMovie.speed(1);//set this to 0 if you want sketch to start with the movie paused
myMovie.loop();
myMovie.play();
currSlice = 0;
//println("movieWidth: " + movieWidth + " movieHeight: " + movieHeight);

//sketch settings
size(windowWidth, movieHeight * 2);
background(0);
stroke(255);
frameRate(25);
noCursor();//works, but not always...
}//end setup()

void draw()
{
if(myMovie.available())
{
myMovie.read();
image(myMovie, 0, 0);
//println("frame: " + currSlice);

//draw vertical line for xOffset
//(this shows where each slice is taken from)
line(xOffset, 0, xOffset, movieHeight - 1);

//copy a vertical slice of video into the drawing area below the movie
loadPixels();
for(int i = 0; i < movieHeight; i++)
{
pixels[currSlice + windowWidth * i + drawingAreaOrigin] = myMovie.pixels[movieWidth * i + xOffset];
}//end for loop
updatePixels();

//update the slice counter
currSlice++;
if(currSlice >= windowWidth)
{
currSlice = 0;
}//end if(currSlice >= movieWidth)
}//end if(m.available())

}//end draw()

void keyPressed()
{
//regular keys
switch(key)
{
case 's':
//save the current screen
saveFrame("station" + "-####.tif");
break;
case ESC:
//catch the escape key so the default message is not displayed
break;
default:
//key not assigned
if(key != CODED) println("key '" + key + "' is not assigned");
}//end switch(key)

//CODED keys
//note the keys BACKSPACE, TAB, ENTER, RETURN, ESC, and DELETE
//do NOT require checking - they can be treated as normal keys
//so they go in cases in the switch(key) block above
if(key == CODED)
{
switch(keyCode)
{
case DOWN:
//pause movie
myMovie.speed(0);
break;
case UP:
//resume regular speed
myMovie.speed(1);
break;
default:
//no default
}//end switch(keyCode)
}//end if(key == CODED)

}//end keyPressed()

fox

Re: How to crop Movie Class?!
Reply #8 - Mar 30^th, 2007, 3:08pm

It's got some extra stuff in there, but you'll work it out, BUT most importantly I have used a customised package for the video.processing library which uses the class name PMovie instead of Movie. Just change it wherever it occurs and it should be sweet. If you want the library it can be found
here: http://members.webone.com.au/~fox/video.zip It has a few added functions too which I mention here: http://processing.org/discourse/yabb_beta/YaBB.cgi?board=Video;action=display;num=1174051925;start=0

Hope this helps. You'll note that I used 1 for-loop to pick the slit and draw it, so it's a bit more complicated but it should be a bit quicker.

Also to grab pixels from lines other than straight horizontal and vertical lines you may want to look at Bresenham's line algorithm which is commonly used for drawing angled straight lines (apparently it can be adapted to circles too). The thing is you need to get from a mathematical description of a line which has infintite points, to a set of pixels that lie in a close fit to the line. Sounds like you're working towards much the same thing as I am - I'm thinking of using the superformula which makes supershapes.

wilsodj

Re: How to crop Movie Class?!
Reply #9 - Apr 2^nd, 2007, 6:40pm

Great! It helped a lot.

Finally pixels is working for me. I also used the same to grab from a diagonal line (starting with xOffset=0, and then just adding "1" every time through the pixels loop.

I will look into the line algorithm you mentioned.

One question I still have, though, is if there is a way to draw a NON-straight irregular line and have the pixels it crosses mapped into an array (until it has loaded a number of values equal to the height of the footage)? Then this array would be used to grab the corresponding pixel values from the video.

I imagine some sort of process like this must be happening (very quickly) whenever you draw a line with your mouse on screen...I will look into it myself as well, but if you have any ideas they would of course be appreciated.

And superformula? Supershapes? What do you mean? It sounds great...

fox

Re: How to crop Movie Class?!
Reply #10 - Apr 4^th, 2007, 3:24am

Now you're talking about something I haven't done yet. Basically you need to take a pair of x-y values (or even an angle and distance if using a radial coordinate system) which will be expressed using a float or double, and deciding which pixel it is over which requires an integer value.

If it's just the mouse location then it's easy enough to cast it to int or use the floor/ceiling/rounding methods which processing/java have.

For a mathematical function describing an irregular or curved line you do much the same thing but feed in say the x values as integers, and then you'll get a y value back (or the other way around if you like). That value will need to be somehow converted to an int, so you need a way of deciding. If you were to use the rounding function it's easy - a y value like 34.6 is rounded up to pixel 37, so it takes care of itself pretty much.

Still you need to do 2 things: access the pixels array like we've been doing with a single integer rather than a pair of x-y values; and, watch out for the boundaries where the converted y value will go out of range either because of rounding it up past the screen dimension by a pixel, or because the mathematical function for your line is returning y values outside of the screen area. So just check for values that are out of the screen area before you try to access the pixels array. Too easy.

I discussed this with my tutor at uni and we decided that for curved lines the picking of pixels doesn't have to be as good as a line rendering routine which would use antialiasing to draw a smooth line. This is because the end result being a slice of video shown in a straight line is quite different from its source taken from a curved or irregular line.

As for the superformula, just search the web for Gielis superformula or supershapes.