try this..... the only difference is you are constraining your y value so that you are not performing so many calculations.

I have to tell you that I can see what you are going for but this isn't going to give you the results you expect. At least now though you can eliminate memory as being the problem. Also read http://processing.org/reference/set_.html  Consider using load pixels for speed. (Also try using P3D or OPENGL this made it work for me)

Keep at it it's a cool idea you can make all sorts of crazy custom black/white filters like this.

PImage img = loadImage("example.jpg");

size(512,512);
image(img,0,0,width,height); // fills the screen with the picture

for(int x=0; x<width; x++)
{

     
 for(int y=50; y<70; y++)   #<------------------- change
 {
     color c = get(x,y);
     float red = red(c);
     float green = green(c);
     float blue = blue(c);
     int grey = (int)(red+green+blue)/3;

     set(x,y,grey);
 }
}
println("done");   #<--- another tip i love statements like these you won't have to sit there going is this thing still going

PImage b = loadImage("example.jpg");
image(b, 0, 0);
filter(GRAY);