Processing 1.0 - Processing Discourse - Creating Mosaics from Photographs

We closed this forum 18 June 2010. It has served us well since 2005 as the ALPHA forum did before it from 2002 to 2005. New discussions are ongoing at the new URL http://forum.processing.org. You'll need to sign up and get a new user account. We're sorry about that inconvenience, but we think it's better in the long run. The content on this forum will remain online.

Index › Discussion › Exhibition › Creating Mosaics from Photographs

‹ Previous Topic | Next Topic ›

Pages: 1

Creating Mosaics from Photographs (Read 2122 times)

Johnny Ow YaBB Newbies Offline Posts: 49	Creating Mosaics from Photographs Feb 13^th, 2009, 4:10pm I did a write up on creating mosaics out of photos, exploring the best way to create a good mosaic. Check it out at: http://bigpixel.com/?p=117. -Jow

gnuskool YaBB Newbies Offline Posts: 18	Re: Creating Mosaics from Photographs Reply #1 - Feb 20^th, 2009, 5:11pm thx for the share

Johnny Ow YaBB Newbies Offline Posts: 49	Re: Creating Mosaics from Photographs Reply #2 - Feb 22^nd, 2009, 9:47pm Part two is here: http://bigpixel.com/?p=158. I take a look at using feature detection (face, edges, color) to improve the mosaics.

Jack Perkins Junior Member Offline Posts: 99	Re: Creating Mosaics from Photographs Reply #3 - Feb 24^th, 2009, 4:43am Thanks, I was just talking about doing this with someone. This is going saving me a lot of research.

Johnny Ow YaBB Newbies Offline Posts: 49	Re: Creating Mosaics from Photographs Reply #4 - Feb 24^th, 2009, 6:43am Happy to share. Feel free to ask if you want some details. I didn't want to bog down the writeup with a bunch of technospeak.

Jack Perkins Junior Member Offline Posts: 99	Re: Creating Mosaics from Photographs Reply #5 - Feb 28^th, 2009, 1:05am Actually, I was wondering what functions you were using to pick up color information from images?

Johnny Ow

Re: Creating Mosaics from Photographs
Reply #6 - Mar 1^st, 2009, 3:51am

Not sure I fully understand this question, but here's a go.

1. The color information is available from an image's pixel array. You get at that by using PImage.loadPixels() and PImage.pixels[].

2. I don't really care about the color of each pixel. What I'm more interested in is the difference in the R, G, and B values of each pixel in the photograph compared to the values of each pixel in a corresponding tile. I do a difference on each R, G, and B value separately and sum up the three differences to get a total difference for that pixel. This is repeated for every pixel of the photograph and every pixel of a tile. By tile, I mean the used to make up the mosaic.

A simpler and faster approach would be to get the average color of each cell of the photograph and each thumbnail by resizing the photograph to C pixels by R pixels and the thumbnail to a 1 pixel by 1 pixel image. (C= columns in grid, R= rows in grid). Then you do a similar difference to find the best color match.

Using my per pixel method, I think, gives you a combination of color-matching and feature (edge) matching.

cloister

Re: Creating Mosaics from Photographs
Reply #7 - Mar 1^st, 2009, 5:04am

Johnny--

I've been reading your write-ups, and three ideas occur to me that would probably help your mosaics a lot:

1. There are a lot of ways you can compare two colors to see which is closer, some of which will work better than others for this and similar tasks. I would encourage you to compare the results you get from different color difference metrics to pick one that works better than simple RGB difference. Sure, RGB is easy because computers fundamentally work in RGB, but the sensitivities of human vision are modeled much better by the HSB color space. In particular, humans are VERY sensitive to changes and alterations in brightness, and considerably less so to variations in hue and saturation. If you don't believe me, try this sometime: separate an image into its hue, saturation, and brightness channels. Apply a mild gaussian blur to the brightness channel only; leave the hue and saturation channels alone. Now reconstitute the image and be amazed at how obvious that minor blur in the brightness channel is. Repeat the experiment with blurs in hue and saturation only. I guarantee that you'll get some insights into how people perceive color and what some better ways are to compare two colors.

2. Instead of looking for the image with the smallest cumulative color difference (no matter what metric you're using), try using a least-squares fit or other mathematical definitions of error that have better statistical properties. Wikipedia has a good write-up of least squares if you don't remember the math from school, or you can e-mail me.

3. Instead of comparing pixel-by-pixel, you could attempt to match sub-regions of each image with the average colors of sub-regions in the target image. This would have two advantages: it should be faster, because you can pre-compute the sub-region color averages for all the images in your pool of pictures, and cache them. When you're working on a pixel-by-pixel basis, it makes no sense to try to cache anything, and as a result your code is going to be really slow. Careful choice of the number of sub-regions (e.g. 2x2 or 3x3 or 4x3 or whatever) should enable you to balance compute time with fidelity in matching, while still doing a good job with matching the source image's structure to the target image's structure.

Feel free to send me a message if you'd like clarification on any of those ideas.

Jack Perkins

Re: Creating Mosaics from Photographs
Reply #8 - Mar 1^st, 2009, 4:45pm

Johnny-

That's exactly what I was looking for, Thanks.

In your writeup you talk about time increases (e.g. doubling because you're doing something twice); what kind of timescales are you talking about here exactly?

I've used MacOSiax (horrible name) to make some photo mosiacs out of my own photos and it takes a LONG TIME ( minimum of five to ten minutes) but it does a pretty decent job.

Johnny Ow

Re: Creating Mosaics from Photographs
Reply #9 - Mar 1^st, 2009, 6:38pm

Hi cloister,

thanks for the excellent suggestions. I am looking at the LMS stuff now and will need to brush up on my maths a bit.

Pertaining to suggestion #1, are there any links or resources about human visual perception and recognition you can recommend? I've just finished reading Eye and Brain by Gregory. It was an interesting read but it's difficult to apply anything from that book to this project.

cloister

Re: Creating Mosaics from Photographs
Reply #10 - Mar 3^rd, 2009, 12:10am

I'd suggest that you just hit wikipedia and research the (weirdly wide) variety of well defined color spaces that are out there. There are some wacky ones.

The thing is, in each one you can take a perceputal color like "that sort of orangy-pink color you see at sunset" and distill it down to a small set of specific values which mean something depending on the coordinate axes of that color system. And therefore, each one has an implicitly different meaning for "color1 - color2." Sometimes those differences do a good job capturing what you're really after, sometimes not. Sometimes one color space is better suited to a given image processing task, sometimes another.

It's best just to be familiar with a wide variety of them, so as to have the mental tools to know what to apply to a given task. Which doesn't mean you need to memorize the conversion formulas from RGB to all of them, just that you should understand how each one works and why it was developed.

Wikipedia, as always, has nice writeups of a lot of this stuff. Look up the pages for "color space" and "color model" in particular, and follow links to pages for the specific color models you haven't heard of.

RGB and HSL/HSB are definitely the heavy hitters for probably 95% of what you'll ever need to do. But the formats used in television and video capture are also interesting to read about because they're really very clever encoding mechanisms (imagine having to design a color model where the image naturally looks "right" whether viewed on a color or a B/W tv, without any software intervention, because this was in the days before software).

THE CIE color space is also a good one to know about, because it is specifically designed to describe the boundaries of human color perception. It's a weird, weird mathematical model but has the nice property that colors which are perceptually "close" to one another are also geometrically close to one another in their locations within CIE space.

Pages: 1

‹ Previous Topic | Next Topic ›