Processing 1.0 - Processing Discourse - Hi-res images with perspective

surelyyoujest

Hi-res images with perspective
Dec 6^th, 2005, 10:25am

I'm trying to render a very large, hi-res image, bigger than the sketch window can normally be. As far as I can tell from reading posts on the forums, the best way to do this is to break the image down into tiles and render them separately. This technique would work perfectly for my purposes, with one catch: I'm rendering a 3D image with perspective.

Clearly it's not enough to simply move the camera; I need to adjust perspective in each image so that they all have different vanishing points... or rather, so that they all, when assembled, have the same vanishing point. Ideally I'd like to position the camera exactly, not approximately, so that I can later assemble the images together automatically rather than try to fudge things in photoshop.

As far as I can tell, my options for adjusting the perspective are as follows:

using perspective(): doesn't seem to be powerful enough.

using frustum(): I can't tell if this would work, because the documentation for this function is extremely vague.

using applyMatrix(): I assume that if what I'm trying to do can be done at all, it can be done with this function. Any help in figuring out how to use it would be greatly appreciated. I've studied linear algebra and so I understand the theory, but I'm not sure how to apply it in this case. The documentation says that this is very slow, but I imagine it'll only be run once per frame, and in my program it's gonna be the least of my problems, speed-wise.

JohnG

Re: Hi-res images with perspective
Reply #1 - Dec 6^th, 2005, 1:09pm

Sounds like a difficult problem. I would have thought it should be possible with perspective but I'm probably forgetting something that would make it not work. Though it probbaly does require a bit of trig to work out the new view centre, and correct width/height/fov to perspective..

One other possible solution is to create a non-drawn 3D object, render everything to it then save from that. But if your image is sufficiently large, even that may not be possible within memory constraints. If it is however it works something like:

Code:

PGraphics3 g3;
g3=new PGraphics3(2000,2000,null);
g3.background(0);
g3.fill(255);
g3.noStroke();
g3.beginShape(TRIANGLES);
g3.vertex(0,0,0);
g3.vertex(200,200,0);
g3.vertex(200,300,200);
g3.endShape();
g3.save("big.jpg");

The above is just an example, but you should be able to see how it works. Basicly you just prefix everything that would normally affect how/what is drawn to screen with the name of your PGraphics3 object, and it will affect the off-screen drawing rather than what is drawn on screen.

If you need to control parameters before saving, you could either draw everythign twice, once in small scale to screen and second onto the larger one, or draw to the large view only, then copy that view and display it scaled down on the real screen.

surelyyoujest

Re: Hi-res images with perspective
Reply #2 - Dec 6^th, 2005, 9:33pm

Thanks, that's a step in the right direction. It might make big enough images for my purposes, though it still imposes a theoretical upper limit to image size, and a workaround to that limit would be useful for a variety of reasons.

Looking over the documentation agaion, as far as I can tell, there's actually no way to do what I want using perspective(), frustum(), or even changing the camera matrix with applyMatrix(). The only way to use a "weird" perspective would be to change the projection matrix, not the camera matrix. Unfortunately, I can't see any way to directly change the projection matrix to an arbitrary value, only to manipulate it in a limited way using perspective() and frustum(). I can only find a way to print the projection matrix, printProjection().

So how do I set an arbitrary projection matrix?

surelyyoujest

Re: Hi-res images with perspective
Reply #3 - Dec 8^th, 2005, 6:53am

I figured it out!

It turns out that all I need to do is play with the parameters to frustrum().

In figuring it out, I also learned a lot about how 3D graphics programming works.

The code in the next post will first show the entire scene to be rendered, then, each time you click the mouse, step through a series of tiles which can be assembled outside of processing to get a single hi-res image. (I use Graphic Converter on the Mac, which conveniently has a function to do this automatically.)

Uncomment the last line to actually save the files.

With a few changes, this can be adapted to use OpenGL. First, the image would have to be saved on drawing the second frame after updating the perspective, not the first, because it actually saves the previous frame instead of the current one. Second, the near clipping plane is the XY plane in this code, which doesn't matter since P3D doesn't actually clip at the clipping plane. But OpenGL will only draw half the model. To fix it, divide the first 5 parameters to each call to frustrum() by some constant, for example changing this:

frustum(-width/2, width/2, -height/2, height/2, cameraZ, 1000);

to this:

frustum(-width/20, width/20, -height/20, height/20, cameraZ/10, 1000);

surelyyoujest

Re: Hi-res images with perspective
Reply #4 - Dec 8^th, 2005, 6:55am

Code:


float FOV = 60; // Set the initial field of view
int TILES = 3; // Set the number of tiles to use each way

float cameraZ;

void setup() 
{ 
  size(400, 400, P3D); 
  noStroke(); 
  cameraZ = (height/2.0) / tan(PI * FOV / 360.0);
} 
 
int tileX, tileY, mode;
boolean saveThis;
 
void mouseReleased() {
  if (mode == 0) {
    mode = 1;
  } else { 
    if(tileX == TILES - 1) {
	tileX = 0;
	tileY = (tileY + 1) % TILES;
    } else {
	tileX = (tileX + 1);
    }
  }
  saveThis = true;
}

void draw() 
{ 
  background(0); 
  lights(); 
  camera(width/2.0, height/2.0, cameraZ, width/2.0, height/2.0, 0, 0, 1, 0);
  
  if (mode == 0) {
    frustum(-width/2, width/2, -height/2, height/2, cameraZ, 1000);
  } else {
    frustum(  width*(tileX/float(TILES) - .5),  width*((tileX+1)/float(TILES) - .5), 
		 height*(tileY/float(TILES) - .5), height*((tileY+1)/float(TILES) - .5), 
		 cameraZ, 10000 );
  }
  
  // Draw something cool
  translate(width/2, height/2, 0);
  scale(height/20);
  rotateX(radians(-30));
  rotateY(radians(35));
  
  for(int j1 = 0; j1 < 10; j1++) {
    for(int j2 = 0; j2 < 10; j2++) {
	for(int j3 = 0; j3 < 10; j3++) {
	  pushMatrix();
	    translate(j1-4.5, j2-4.5, j3-4.5);
	    fill(240-j1*25, 240-j2*25, 15+j3*25);
	    box(.75);
	  popMatrix();
	}  
    }  
  }
  
  //if (saveThis) { saveThis = false; save(tileY + "-" + tileX + ".tif");} 
}

backspaces

Re: Hi-res images with perspective
Reply #5 - Jan 2^nd, 2006, 8:32pm

Newbie question: Why are the tiles needed? Couldn't the image simply be a single object .. possibly a 3D shape that contains the image or has the image texture mapped onto it?

I ask because I'm starting to look at using Processing for 3D Agent Based Modeling where critters with behavior wander around a 3D landscape which might be a large image.

Owen

Owen -- Mac OS X -- The latest Processing generally.

JohnG Ex Member	Re: Hi-res images with perspective Reply #6 - Jan 2^nd, 2006, 11:22pm I tihnk the main issue is memory usage. If you're wanting to produce print-quality images, you'll be needing images in to 8000x8000 and above range probably and they take up a large ammount of memory, so it is sometime better to produce 4/9/16/however many smaller images, then stitch them together aftrewards.

backspaces Junior Member Offline Posts: 66	Re: Hi-res images with perspective Reply #7 - Jan 3^rd, 2006, 11:35pm Got it! In OpenGL the image sizes are bumped up to the next power of two .. adds up!
	Owen -- Mac OS X -- The latest Processing generally.

watz

Re: Hi-res images with perspective
Reply #8 - Aug 29^th, 2006, 2:42pm

Hi, just managed to hack a OpenGL version of this code to work with a project I'm doing at the moment (see sketches on Flickr). After some weirdness with saving frames twice I got it working, thanks for showing how it's done, I will try to post a cleaned-up version of my code at some point.

Now the second question is: How does one get HUGE files easily stitched together Does anyone know of a utility to do it with, alternatively a way of saving to an image format that would support sizes of over 10k x 10k pixels PSD should be possible, but I haven't seen any code for it.

marius watz // unlekker // code & form // Generator.x

watz Full Member Offline Posts: 202	Re: Hi-res images with perspective Reply #9 - Aug 30^th, 2006, 4:49pm Nevermind, I just tried simply using PImage. Saving as TGA or even PNG I can happily stitch together 10k x 10k images. I'll post code for tiling and stitching when I get this current project done. Thanks again for the tiling code, it's exactly the kind of thing that I am too unadventurous to attempt.
	marius watz // unlekker // code & form // Generator.x

grimus

Re: Hi-res images with perspective
Reply #10 - Aug 31^st, 2006, 8:33am

Here's some openGL code that auto-stitches an 8x8 tile matrix. It should be possible to do a similar thing in processing using pixels[].

Haven't tried this, but there's a program for windows that apparently auto-stitches .bmp files together:
http://www.freedownloadscenter.com/Multimedia_and_Graphics/Misc__Graphics_Tools/TrentStitch.html

If I get a processing version of this code working i'll post it.

Code:



//  It takes 64 passes to produce the high resolution screen shot.
//  The normal frustum extents are [-1, 1], so now we will let them range
//  from -1 to 1 in 8 steps (one step per tile). We do this both
//  horizontally and vertically to end up with 8x8 tiles


for (y = 0; y < 8;y++) {
   for (x = 0; x < 8;x++) {

   left = -1 + (float)x/4;
   right = -1 + (float)(x+1)/4;
   bottom = -1 + (float)y/4;
   top = -1 + (float)(y+1)/4;

   glMatrixMode(GL_PROJECTION);
   glLoadIdentity();
   glFrustum(left, right, bottom, top, 2, 50);

   // Call the rendering function
   RenderFunction();

   unsigned char *pImg;
   // After rendering each tile, 
   // we read the framebuffer contents into our bitmap.

   for (i=0; i < SCREEN_HEIGHT;i++) {
	//Shift to the correct scanline

	pImg = &m_HRScreenShotData[0] + 3*(((y)*SCREEN_HEIGHT)*(SCREEN_WIDTH*8) + (SCREEN_WIDTH*x) + i*(SCREEN_WIDTH*8));


	// Read a line from the framebuffer
	glReadPixels(0, i, SCREEN_WIDTH, 1, GL_RGB, GL_UNSIGNED_BYTE, pImg );
   }
}
}

http://www.digitalstar.net

grimus

Re: Hi-res images with perspective
Reply #11 - Aug 31^st, 2006, 5:33pm

Here's a processing version of the code i posted above.
Change the NUM_TILES variables to alter the size of the final image. A value of 12 produces a 12xwidthx12xheight size image. I just made one of 10248x10248 pixels.

Be sure to increase your VM memory!
http://processing.org/faq/bugs.html#memory

Code:


import processing.opengl.*;


float FOV = 60; // Set the initial field of view 
float cameraZ;
boolean saveThis=false;
int NUM_TILES = 12;


void setup() 
{
  size(850, 850, P3D);
  noStroke();
  fill(0, 102, 153, 26);
  cameraZ = (height/2.0) / tan(PI * FOV / 360.0); 

}


void draw() 
{
  int x=0,y=0;
  float left,right,bottom,top;

  background(0);
  camera(width/2.0, height/2.0, cameraZ, width/2.0, height/2.0, 0, 0, 1, 0); 
    

  frustum(-width/2, width/2, -height/2, height/2, cameraZ, 1000); 
  renderScreen();
  
  if(saveThis) {
    saveThis=false;
    saveScreen();
  }
  

}

void mouseReleased() {
  saveThis=true;
} 

void renderScreen() {
 // Draw something cool
  translate(width/2, height/2, 0);
  scale(height/20);
  rotateX(radians(-30));
  rotateY(radians(35));
   
  for(int j1 = 0; j1 < 10; j1++) {
    for(int j2 = 0; j2 < 10; j2++) {
 for(int j3 = 0; j3 < 10; j3++) {
   pushMatrix();
     translate(j1-4.5, j2-4.5, j3-4.5);
     fill(240-j1*25, 240-j2*25, 15+j3*25);
     box(.75);
   popMatrix();
 }  
    }  
  }

}

void saveScreen() {
int x=0,y=0,index=0;
float left,right,bottom,top;
PImage img = new PImage(width*NUM_TILES,height*NUM_TILES,RGB); 

  for (y = 0; y < NUM_TILES;y++) {
    for (x = 0; x < NUM_TILES;x++) {
	
	println("saving "+x+"-" + y+"...");
	left = (float)(-width/2) + (float)x*width/NUM_TILES;
	right = (float)(-width/2) + (float)((x+1)*width)/NUM_TILES;
	bottom = (float)(-height/2) + (float)(y*height)/NUM_TILES;
	top = (float)(-height/2) + (float)((y+1)*height)/NUM_TILES;
 
	background(0);
	camera(width/2.0, height/2.0, cameraZ, width/2.0, height/2.0, 0, 0, 1, 0);	  
	frustum(left, right, bottom, top, cameraZ, 1000);

	// Call the rendering function
	renderScreen();
	loadPixels();
 
	// After rendering each tile,  
	// we read the framebuffer contents into our bitmap.
 
	for (int i=0; i < height;i++) {
     
	  //Shift to the correct scanline
	  index = (((y)*height)*(width*NUM_TILES) + (width*x) + i*(width*NUM_TILES));

	  // Read a line from the framebuffer
	  for(int j=0; j < width;j++) {
	    img.pixels[index+j] = pixels[i*height+j];
	  }
	}

	  
    }
  }

  img.save("/home/grimus/processing/test.tga");
}

http://www.digitalstar.net

watz Full Member Offline Posts: 202	Re: Hi-res images with perspective Reply #12 - Oct 11^th, 2006, 1:48am I just posted code to put together tiled pieces into one big image: http://workshop.evolutionzone.com/2006/10/11/code-imagestitcherpde/
	marius watz // unlekker // code & form // Generator.x

ssdesign YaBB Newbies Offline Posts: 38	Re: Hi-res images with perspective Reply #13 - Jun 21^st, 2007, 11:51am Hi Watz, When I export image using your code, I get the image of the same size as the processing sketch. How do I enlarge the image?
	:: form follows function ::&&http://www.ssdesigninteractive.com/g2

ssdesign YaBB Newbies Offline Posts: 38	Re: Hi-res images with perspective Reply #14 - Jun 21^st, 2007, 12:01pm Hi, The problem is, the Preview image is same as my sketch image (thats ok) but the TGA image is not one big image, but its a tile of smaller images
	:: form follows function ::&&http://www.ssdesigninteractive.com/g2